agora inbox for pgsql-hackers@postgresql.org
help / color / mirror / Atom feedBitmapHeapScan streaming read user and prelim refactoring
248+ messages / 18 participants
[nested] [flat]
* BitmapHeapScan streaming read user and prelim refactoring
@ 2024-02-13 23:11 Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 04:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Mark Dilger <mark.dilger@enterprisedb.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-02-13 23:11 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andres Freund <andres@anarazel.de>; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
Hi,
Attached is a patch set which refactors BitmapHeapScan such that it
can use the streaming read API [1]. It also resolves the long-standing
FIXME in the BitmapHeapScan code suggesting that the skip fetch
optimization should be pushed into the table AMs. Additionally, it
moves table scan initialization to after the index scan and bitmap
initialization.
patches 0001-0002 are assorted cleanup needed later in the set.
patches 0003 moves the table scan initialization to after bitmap creation
patch 0004 is, I think, a bug fix. see [2].
patches 0005-0006 push the skip fetch optimization into the table AMs
patches 0007-0009 change the control flow of BitmapHeapNext() to match
that required by the streaming read API
patch 0010 is the streaming read code not yet in master
patch 0011 is the actual bitmapheapscan streaming read user.
patches 0001-0009 apply on top of master but 0010 and 0011 must be
applied on top of a commit before a 21d9c3ee4ef74e2 (until a rebased
version of the streaming read API is on the mailing list).
The caveat is that these patches introduce breaking changes to two
table AM functions for bitmapheapscan: table_scan_bitmap_next_block()
and table_scan_bitmap_next_tuple().
A TBMIterateResult used to be threaded through both of these functions
and used in BitmapHeapNext(). This patch set removes all references to
TBMIterateResults from BitmapHeapNext. Because the streaming read API
requires the callback to specify the next block, BitmapHeapNext() can
no longer pass a TBMIterateResult to table_scan_bitmap_next_block().
More subtly, table_scan_bitmap_next_block() used to return false if
there were no more visible tuples on the page or if the block that was
requested was not valid. With these changes,
table_scan_bitmap_next_block() will only return false when the bitmap
has been exhausted and the scan can end. In order to use the streaming
read API, the user must be able to request the blocks it needs without
requiring synchronous feedback per block. Thus, this table AM function
must change its meaning.
I think the way the patches are split up could be improved. I will
think more about this. There are also probably a few mistakes with
which comments are updated in which patches in the set.
- Melanie
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2...
[2] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.g...
Attachments:
[text/x-patch] v1-0003-BitmapHeapScan-begin-scan-after-bitmap-setup.patch (5.8K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/2-v1-0003-BitmapHeapScan-begin-scan-after-bitmap-setup.patch)
download | inline diff:
From d6dd6eb21dcfbc41208f87d1d81ffe3960130889 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v1 03/11] BitmapHeapScan begin scan after bitmap setup
There is no reason for table_beginscan_bm() to begin the actual scan of
the underlying table in ExecInitBitmapHeapScan(). We can begin the
underlying table scan after the index scan has been completed and the
bitmap built.
The one use of the scan descriptor during initialization was
ExecBitmapHeapInitializeWorker(), which set the scan descriptor snapshot
with one from an array in the parallel state. This overwrote the
snapshot set in table_beginscan_bm().
By saving that worker snapshot as a member in the BitmapHeapScanState
during initialization, it can be restored in table_beginscan_bm() after
returning from the table AM specific begin scan function.
---
src/backend/executor/nodeBitmapHeapscan.c | 27 ++++++++++++++---------
src/include/access/tableam.h | 18 +++++++++------
src/include/nodes/execnodes.h | 2 ++
3 files changed, 30 insertions(+), 17 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 76382c91fd7..fd697d16c72 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -191,6 +191,17 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ node->worker_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -614,7 +625,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -691,7 +703,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
}
/* ----------------------------------------------------------------
@@ -740,6 +753,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->can_skip_fetch = false;
+ scanstate->worker_snapshot = NULL;
/*
* Miscellaneous initialization
@@ -788,11 +802,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
@@ -931,13 +940,11 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelBitmapHeapState *pstate;
- Snapshot snapshot;
Assert(node->ss.ps.state->es_query_dsa != NULL);
pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->pstate = pstate;
- snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
- table_scan_update_snapshot(node->ss.ss_currentScanDesc, snapshot);
+ node->worker_snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 4d495216f07..77f32a7472d 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -931,6 +931,11 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
+/*
+ * Update snapshot used by the scan.
+ */
+extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
+
/*
* table_beginscan_bm is an alternative entry point for setting up a
* TableScanDesc for a bitmap heap scan. Although that scan technology is
@@ -938,12 +943,16 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
* make it worth using the same data structure.
*/
static inline TableScanDesc
-table_beginscan_bm(Relation rel, Snapshot snapshot,
+table_beginscan_bm(Relation rel, Snapshot snapshot, Snapshot worker_snapshot,
int nkeys, struct ScanKeyData *key)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ if (worker_snapshot)
+ table_scan_update_snapshot(result, worker_snapshot);
+ return result;
}
/*
@@ -1033,11 +1042,6 @@ table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
allow_pagemode);
}
-/*
- * Update snapshot used by the scan.
- */
-extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
-
/*
* Return next tuple from `scan`, store in slot.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 444a5f0fd57..00c75fb10e2 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1726,6 +1726,7 @@ typedef struct ParallelBitmapHeapState
* shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * worker_snapshot snapshot for parallel worker
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1750,6 +1751,7 @@ typedef struct BitmapHeapScanState
TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ Snapshot worker_snapshot;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-patch] v1-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/3-v1-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 5f915bc84eae56e52b5a61e9b7e691834fdb9680 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v1 02/11] BitmapHeapScan set can_skip_fetch later
There is no reason for BitmapHeapScan to calculate can_skip_fetch in
ExecInitBitmapHeapScan(). Moving it into BitmapHeapNext() is a
preliminary step toward moving can_skip_fetch into table AM specific
code, as we would need to set it after the scan has begun.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d670939246b..76382c91fd7 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,6 +108,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -729,16 +739,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.37.2
[text/x-patch] v1-0001-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (3.2K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/4-v1-0001-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From 575fb1f93128ebfd8125c769de628f91e0d5c592 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v1 01/11] Remove table_scan_bitmap_next_tuple parameter tbmres
Future commits will remove the input TBMIterateResult from
table_scan_bitmap_next_block() as the streaming read API will be
responsible for iterating through the blocks in the bitmap and not
BitmapHeapNext(). Given that this parameter will not be set from
BitmapHeapNext(), it no longer makes sense to use it as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 7 -------
3 files changed, 1 insertion(+), 9 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d15a02b2be7..716d477e271 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2228,7 +2228,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c1e81ebed63..d670939246b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -304,7 +304,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5f8474871d2..4d495216f07 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -810,15 +810,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -1980,7 +1975,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -1992,7 +1986,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.37.2
[text/x-patch] v1-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/5-v1-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From d56be7741765d93002649ef912ef4b8256a5b9af Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v1 05/11] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index ab97f308a5f..9372b49bfaa 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -55,7 +55,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -226,7 +226,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
/*
* We can skip fetching the heap page if we don't need any fields
@@ -379,7 +379,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -398,7 +398,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.37.2
[text/x-patch] v1-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (1.6K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/6-v1-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From a3f62e4299663d418531ae61bb16ea39f0836fac Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v1 04/11] BitmapPrefetch use prefetch block recheck for skip
fetch
Previously BitmapPrefetch() used the recheck flag for the current block
to determine whether or not it could skip prefetching the proposed
prefetch block. It makes more sense for it to use the recheck flag from
the TBMIterateResult for the prefetch block instead.
---
src/backend/executor/nodeBitmapHeapscan.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index fd697d16c72..ab97f308a5f 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -519,7 +519,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* but is true in many cases.
*/
skip_fetch = (node->can_skip_fetch &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -570,7 +570,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (node->can_skip_fetch &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.37.2
[text/x-patch] v1-0006-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (14.2K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/7-v1-0006-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From 202b16d3a381210e8dbee69e68a8310be8ee11d2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v1 06/11] Push BitmapHeapScan skip fetch optimization into
table AM
This resolves the long-standing FIXME in BitmapHeapNext() which said that
the optmization to skip fetching blocks of the underlying table when
none of the column data was needed should be pushed into the table AM
specific code.
heapam_scan_bitmap_next_block() now does the visibility check and
accounting of empty tuples to be returned; while
heapam_scan_bitmap_next_tuple() prepares the slot to return empty
tuples.
The table AM agnostic functions for prefetching still need to know if
skipping fetching is permitted for this scan. However, this dependency
will be removed when that prefetching code is removed in favor of the
upcoming streaming read API.
---
src/backend/access/heap/heapam.c | 10 +++
src/backend/access/heap/heapam_handler.c | 29 +++++++
src/backend/executor/nodeBitmapHeapscan.c | 100 ++++++----------------
src/include/access/heapam.h | 2 +
src/include/access/tableam.h | 17 ++--
src/include/nodes/execnodes.h | 6 --
6 files changed, 74 insertions(+), 90 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 707460a5364..7aae1ecf0a9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -955,6 +955,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->vmbuffer = InvalidBuffer;
+ scan->empty_tuples = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1043,6 +1045,10 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->vmbuffer))
+ ReleaseBuffer(scan->vmbuffer);
+ scan->vmbuffer = InvalidBuffer;
+
/*
* reinitialize scan descriptor
*/
@@ -1062,6 +1068,10 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->vmbuffer))
+ ReleaseBuffer(scan->vmbuffer);
+ scan->vmbuffer = InvalidBuffer;
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 716d477e271..baba09c87c0 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, and the bitmap entries don't need rechecking, and all tuples on
+ * the page are visible to our transaction.
+ */
+ if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->empty_tuples >= 0);
+
+ hscan->empty_tuples += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2235,6 +2254,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->empty_tuples > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->empty_tuples--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 9372b49bfaa..c0fb06c9688 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,6 +108,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ bool can_skip_fetch;
/*
* We can potentially skip fetching heap pages if we do not need any
* columns of the table, either for checking non-indexable quals or
@@ -115,7 +116,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* the stronger condition that there's no qual or return tlist at all.
* But in most cases it's probably not worth working harder than that.
*/
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
node->ss.ps.plan->targetlist == NIL);
if (!pstate)
@@ -199,7 +200,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ps.state->es_snapshot,
node->worker_snapshot,
0,
- NULL);
+ NULL,
+ can_skip_fetch);
}
node->initialized = true;
@@ -207,8 +209,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -228,32 +228,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!table_scan_bitmap_next_block(scan, tbmres))
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -307,46 +282,30 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +477,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
!tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +529,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
!tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +599,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +608,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +652,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -739,8 +694,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +705,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..2fc369a18ff 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -73,6 +73,8 @@ typedef struct HeapScanDescData
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
/* these fields only used in page-at-a-time mode and for bitmap scans */
+ Buffer vmbuffer; /* for checking if can skip fetch */
+ int empty_tuples; /* count of all NULL tuples to be returned */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
OffsetNumber rs_vistuples[MaxHeapTuplesPerPage]; /* their offsets */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 77f32a7472d..05e700c5055 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,7 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+ SO_CAN_SKIP_FETCH = 1 << 10,
} ScanOptions;
/*
@@ -780,10 +781,8 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time). All work
+ * referencing `tbmres` must be done here.
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -795,11 +794,6 @@ typedef struct TableAmRoutine
* performs prefetching directly using that interface. This probably
* needs to be rectified at a later point.
*
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -944,11 +938,14 @@ extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot, Snapshot worker_snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, bool can_skip_fetch)
{
TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ if (can_skip_fetch)
+ flags |= SO_CAN_SKIP_FETCH;
+
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
if (worker_snapshot)
table_scan_update_snapshot(result, worker_snapshot);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 00c75fb10e2..9392923eb32 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1711,9 +1711,6 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
* pvmbuffer ditto, for prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1736,9 +1733,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.37.2
[text/x-patch] v1-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (3.0K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/8-v1-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From ccd5d688fd5c1dd16908788e0a0abd0f3e64eb77 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v1 08/11] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 19d115de06f..4d55390715c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -76,8 +76,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -90,10 +88,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -111,6 +105,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (!node->initialized)
{
bool can_skip_fetch;
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
/*
* We can potentially skip fetching heap pages if we do not need any
* columns of the table, either for checking non-indexable quals or
@@ -129,7 +125,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -182,8 +178,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +201,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
can_skip_fetch);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -219,9 +216,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.37.2
[text/x-patch] v1-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (17.7K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/9-v1-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 555743e4bc885609d20768f7f2990c6ba69b13a9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:57:07 -0500
Subject: [PATCH v1 09/11] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the known end of the table.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table and for
table_scan_bitmap_next_tuple() to return false if there are no visible
tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API. The table AMs will implement a streaming read API callback
that returns the next block that needs to be fetched. In heap AM's case,
the callback will use the iterator to find the next block to be fetched.
Since choosing the next block will no longer the responsibility of
BitmapHeapNext(), the streaming read control flow requires these changes
to table_scan_bitmap_next_block().
---
src/backend/access/heap/heapam.c | 22 ++++
src/backend/access/heap/heapam_handler.c | 56 ++++++---
src/backend/executor/nodeBitmapHeapscan.c | 132 ++++++++--------------
src/include/access/relscan.h | 3 +
src/include/access/tableam.h | 14 +--
src/include/nodes/execnodes.h | 10 +-
6 files changed, 121 insertions(+), 116 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 88b4aad5820..d8569373987 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -959,6 +959,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->empty_tuples = 0;
scan->rs_base.lossy_pages = 0;
scan->rs_base.exact_pages = 0;
+ scan->rs_base.shared_tbmiterator = NULL;
+ scan->rs_base.tbmiterator = NULL;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1051,6 +1053,18 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
ReleaseBuffer(scan->vmbuffer);
scan->vmbuffer = InvalidBuffer;
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_base.shared_tbmiterator)
+ tbm_end_shared_iterate(scan->rs_base.shared_tbmiterator);
+
+ if (scan->rs_base.tbmiterator)
+ tbm_end_iterate(scan->rs_base.tbmiterator);
+ }
+
+ scan->rs_base.shared_tbmiterator = NULL;
+ scan->rs_base.tbmiterator = NULL;
+
/*
* reinitialize scan descriptor
*/
@@ -1074,6 +1088,14 @@ heap_endscan(TableScanDesc sscan)
ReleaseBuffer(scan->vmbuffer);
scan->vmbuffer = InvalidBuffer;
+ if (sscan->shared_tbmiterator)
+ tbm_end_shared_iterate(sscan->shared_tbmiterator);
+ sscan->shared_tbmiterator = NULL;
+
+ if (sscan->tbmiterator)
+ tbm_end_iterate(sscan->tbmiterator);
+ sscan->tbmiterator = NULL;
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 6e85ef7a946..d55ece23a35 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,17 +2114,49 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ bool *recheck, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->empty_tuples == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, and the bitmap entries don't need rechecking, and all tuples on
@@ -2143,16 +2175,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2251,7 +2274,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
scan->lossy_pages++;
}
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 4d55390715c..efc6952e353 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -76,7 +76,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -88,7 +87,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -126,7 +124,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -179,7 +176,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -201,46 +197,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
can_skip_fetch);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ /* Get the first block. if none, end of scan */
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ goto exit;
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ BitmapAdjustPrefetchTarget(node);
}
for (;;)
{
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- if (!table_scan_bitmap_next_block(scan, tbmres))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
- {
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -262,53 +235,46 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- BitmapPrefetch(node, scan);
-
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
- }
+ /*
+ * We prefetch before fetching the current pages. We expect that a
+ * future streaming read API will do this, so do it now for
+ * consistency.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
{
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
}
+
+ /* OK to return this tuple */
+ return slot;
}
- /* OK to return this tuple */
- return slot;
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ break;
+
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
* if we get here it means we are at the end of the scan..
*/
+exit:
BitmapAccumCounters(node, scan);
return ExecClearTuple(slot);
}
@@ -594,12 +560,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -607,13 +569,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -647,14 +608,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
@@ -697,8 +654,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -707,10 +662,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index b74e08dd745..bf7ee044268 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -16,6 +16,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
+#include "nodes/tidbitmap.h"
#include "port/atomics.h"
#include "storage/buf.h"
#include "storage/spin.h"
@@ -41,6 +42,8 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
+ TBMIterator *tbmiterator;
+ TBMSharedIterator *shared_tbmiterator;
long exact_pages;
long lossy_pages;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 05e700c5055..b90d9b7f3fa 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -798,7 +798,7 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ bool *recheck, BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1942,17 +1942,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ bool *recheck, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1962,8 +1961,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 9392923eb32..03973a3f262 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1709,9 +1709,7 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1720,7 +1718,6 @@ typedef struct ParallelBitmapHeapState
* prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
@@ -1731,8 +1728,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1742,10 +1737,11 @@ typedef struct BitmapHeapScanState
int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
+ bool recheck;
+ BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-patch] v1-0007-BitmapHeapScan-scan-desc-counts-lossy-and-exact-p.patch (3.8K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/10-v1-0007-BitmapHeapScan-scan-desc-counts-lossy-and-exact-p.patch)
download | inline diff:
From 500c84019b982a1e6c8b8dd40240c8510d83c287 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:05:04 -0500
Subject: [PATCH v1 07/11] BitmapHeapScan scan desc counts lossy and exact
pages
Future commits will remove the TBMIterateResult from BitmapHeapNext(),
pushing it into the table AM-specific code. So we will have to keep
track of the number of lossy and exact pages in the scan descriptor.
Doing this change to lossy/exact page counting in a separate commit just
simplifies the diff.
---
src/backend/access/heap/heapam.c | 2 ++
src/backend/access/heap/heapam_handler.c | 9 +++++++++
src/backend/executor/nodeBitmapHeapscan.c | 18 +++++++++++++-----
src/include/access/relscan.h | 4 ++++
4 files changed, 28 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 7aae1ecf0a9..88b4aad5820 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -957,6 +957,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_strategy = NULL; /* set in initscan */
scan->vmbuffer = InvalidBuffer;
scan->empty_tuples = 0;
+ scan->rs_base.lossy_pages = 0;
+ scan->rs_base.exact_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index baba09c87c0..6e85ef7a946 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2242,6 +2242,15 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ /* Only count exact and lossy pages with visible tuples */
+ if (ntup > 0)
+ {
+ if (tbmres->ntuples >= 0)
+ scan->exact_pages++;
+ else
+ scan->lossy_pages++;
+ }
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c0fb06c9688..19d115de06f 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -53,6 +53,8 @@
#include "utils/spccache.h"
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
+static inline void BitmapAccumCounters(BitmapHeapScanState *node,
+ TableScanDesc scan);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
BlockNumber blockno);
@@ -234,11 +236,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
continue;
}
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
- node->lossy_pages++;
-
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
@@ -315,9 +312,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* if we get here it means we are at the end of the scan..
*/
+ BitmapAccumCounters(node, scan);
return ExecClearTuple(slot);
}
+static inline void
+BitmapAccumCounters(BitmapHeapScanState *node,
+ TableScanDesc scan)
+{
+ node->exact_pages += scan->exact_pages;
+ scan->exact_pages = 0;
+ node->lossy_pages += scan->lossy_pages;
+ scan->lossy_pages = 0;
+}
+
/*
* BitmapDoneInitializingSharedState - Shared state is initialized
*
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..b74e08dd745 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -40,6 +40,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ long exact_pages;
+ long lossy_pages;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
--
2.37.2
[text/x-patch] v1-0010-Streaming-Read-API.patch (56.0K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/11-v1-0010-Streaming-Read-API.patch)
download | inline diff:
From 9eb510c1f2fd4d1b3c831f62af1e4c0f422a0922 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 22 Jul 2023 17:31:54 +1200
Subject: [PATCH v1 10/11] Streaming Read API
---
contrib/pg_prewarm/pg_prewarm.c | 40 +-
src/backend/access/transam/xlogutils.c | 2 +-
src/backend/postmaster/bgwriter.c | 8 +-
src/backend/postmaster/checkpointer.c | 15 +-
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 435 ++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 560 +++++++++++++++--------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/backend/storage/smgr/smgr.c | 49 +-
src/include/storage/bufmgr.h | 22 +
src/include/storage/smgr.h | 4 +-
src/include/storage/streaming_read.h | 45 ++
src/include/utils/rel.h | 6 -
src/tools/pgindent/typedefs.list | 2 +
17 files changed, 986 insertions(+), 238 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index 8541e4d6e46..9617bf130bd 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -20,6 +20,7 @@
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
+#include "storage/streaming_read.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
@@ -38,6 +39,25 @@ typedef enum
static PGIOAlignedBlock blockbuffer;
+struct pg_prewarm_streaming_read_private
+{
+ BlockNumber blocknum;
+ int64 last_block;
+};
+
+static BlockNumber
+pg_prewarm_streaming_read_next(PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_data)
+{
+ struct pg_prewarm_streaming_read_private *p = pgsr_private;
+
+ if (p->blocknum <= p->last_block)
+ return p->blocknum++;
+
+ return InvalidBlockNumber;
+}
+
/*
* pg_prewarm(regclass, mode text, fork text,
* first_block int8, last_block int8)
@@ -183,18 +203,36 @@ pg_prewarm(PG_FUNCTION_ARGS)
}
else if (ptype == PREWARM_BUFFER)
{
+ struct pg_prewarm_streaming_read_private p;
+ PgStreamingRead *pgsr;
+
/*
* In buffer mode, we actually pull the data into shared_buffers.
*/
+
+ /* Set up the private state for our streaming buffer read callback. */
+ p.blocknum = first_block;
+ p.last_block = last_block;
+
+ pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ &p,
+ 0,
+ NULL,
+ BMR_REL(rel),
+ forkNumber,
+ pg_prewarm_streaming_read_next);
+
for (block = first_block; block <= last_block; ++block)
{
Buffer buf;
CHECK_FOR_INTERRUPTS();
- buf = ReadBufferExtended(rel, forkNumber, block, RBM_NORMAL, NULL);
+ buf = pg_streaming_read_buffer_get_next(pgsr, NULL);
ReleaseBuffer(buf);
++blocks_done;
}
+ Assert(pg_streaming_read_buffer_get_next(pgsr, NULL) == InvalidBuffer);
+ pg_streaming_read_free(pgsr);
}
/* Close relation, release lock. */
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index aa8667abd10..8775b5789be 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -657,7 +657,7 @@ XLogDropDatabase(Oid dbid)
* This is unnecessarily heavy-handed, as it will close SMgrRelation
* objects for other databases as well. DROP DATABASE occurs seldom enough
* that it's not worth introducing a variant of smgrclose for just this
- * purpose. XXX: Or should we rather leave the smgr entries dangling?
+ * purpose.
*/
smgrcloseall();
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d7d6cc0cd7b..13e5376619e 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -246,10 +246,12 @@ BackgroundWriterMain(void)
if (FirstCallSinceLastCheckpoint())
{
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the bgwriter does
+ * not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
}
/*
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 5e949fc885b..5d843b61426 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -469,10 +469,12 @@ CheckpointerMain(void)
ckpt_performed = CreateRestartPoint(flags);
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the checkpointer
+ * does not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
/*
* Indicate checkpoint completion to any waiting backends.
@@ -958,11 +960,8 @@ RequestCheckpoint(int flags)
*/
CreateCheckPoint(flags | CHECKPOINT_IMMEDIATE);
- /*
- * After any checkpoint, close all smgr files. This is so we won't
- * hang onto smgr references to deleted files indefinitely.
- */
- smgrcloseall();
+ /* Free all smgr objects, as CheckpointerMain() normally would. */
+ smgrdestroyall();
return;
}
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..bcab44c802f
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..39aef2a84a2
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 00000000000..19605090fea
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,435 @@
+#include "postgres.h"
+
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ *
+ * For hits, need_to_complete is false and there is just one block per
+ * range, already pinned and ready for use.
+ *
+ * For misses, need_to_complete is true and buffers[] holds a range of
+ * blocks that are contiguous in storage (though the buffers may not be
+ * contiguous in memory), so we can complete them with a single call to
+ * CompleteReadBuffers().
+ */
+typedef struct PgStreamingReadRange
+{
+ bool advice_issued;
+ bool need_complete;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index[MAX_BUFFERS_PER_TRANSFER];
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+} PgStreamingReadRange;
+
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int ios_in_progress_trigger;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int pinned_buffers_trigger;
+ int next_tail_buffer;
+ bool finished;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ bool advice_enabled;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+ int per_buffer_data_next;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+static PgStreamingRead *
+pg_streaming_read_buffer_alloc_internal(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = maintenance_io_concurrency;
+ else
+ max_ios = effective_io_concurrency;
+
+ /*
+ * The desired level of I/O concurrency controls how far ahead we are
+ * willing to look ahead. We also clamp it to at least
+ * MAX_BUFFER_PER_TRANFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /*
+ * The *_io_concurrency GUCs, we might have 0. We want to allow at least
+ * one, to keep our gating logic simple.
+ */
+ max_ios = Max(max_ios, 1);
+
+ /*
+ * Don't allow this backend to pin too many buffers. For now we'll apply
+ * the limit for the shared buffer pool and the local buffer pool, without
+ * worrying which it is.
+ */
+ LimitAdditionalPins(&max_pinned_buffers);
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * We want to avoid creating ranges that are smaller than they could be
+ * just because we hit max_pinned_buffers. We only look ahead when the
+ * number of pinned buffers falls below this trigger number, or put
+ * another way, we stop looking ahead when we wouldn't be able to build a
+ * "full sized" range.
+ */
+ pgsr->pinned_buffers_trigger =
+ Max(1, (int) max_pinned_buffers - MAX_BUFFERS_PER_TRANSFER);
+
+ /* Space the callback to store extra data along with each block. */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * max_pinned_buffers);
+
+ return pgsr;
+}
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *result;
+
+ result = pg_streaming_read_buffer_alloc_internal(flags,
+ pgsr_private,
+ per_buffer_data_size,
+ strategy);
+ result->callback = next_block_cb;
+ result->bmr = bmr;
+ result->forknum = forknum;
+
+ return result;
+}
+
+/*
+ * Start building a new range. This is called after the previous one
+ * reached maximum size, or the callback's next block can't be merged with it.
+ *
+ * Since the previous head range has now reached its full potential size, this
+ * is also a good time to issue 'prefetch' advice, because we know that'll
+ * soon be reading. In future, we could start an actual I/O here.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_new_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If a call to CompleteReadBuffers() will be needed, and we can issue
+ * advice to the kernel to get the read started. We suppress it if the
+ * access pattern appears to be completely sequential, though, because on
+ * some systems that interfers with the kernel's own sequential read ahead
+ * heurstics and hurts performance.
+ */
+ if (pgsr->advice_enabled)
+ {
+ BlockNumber blocknum = head_range->blocknum;
+ int nblocks = head_range->nblocks;
+
+ if (head_range->need_complete && blocknum != pgsr->seq_blocknum)
+ {
+ SMgrRelation smgr =
+ pgsr->bmr.smgr ? pgsr->bmr.smgr :
+ RelationGetSmgr(pgsr->bmr.rel);
+
+ Assert(!head_range->advice_issued);
+
+ smgrprefetch(smgr, pgsr->forknum, blocknum, nblocks);
+
+ /*
+ * Count this as an I/O that is concurrently in progress, though
+ * we don't really know if the kernel generates a physical I/O.
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ }
+
+ /* Remember the block after this range, for sequence detection. */
+ pgsr->seq_blocknum = blocknum + nblocks;
+ }
+
+ /* Create a new head range. There must be space. */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ head_range = &pgsr->ranges[pgsr->head];
+ head_range->nblocks = 0;
+
+ return head_range;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ /*
+ * If we're finished or can't start more I/O, then don't look ahead.
+ */
+ if (pgsr->finished || pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * We'll also wait until the number of pinned buffers falls below our
+ * trigger level, so that we have the chance to create a full range.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger)
+ return;
+
+ do
+ {
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ Buffer buffer;
+ bool found;
+ bool need_complete;
+ PgStreamingReadRange *head_range;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks == lengthof(head_range->buffers))
+ {
+ Assert(head_range->need_complete);
+ head_range = pg_streaming_read_new_range(pgsr);
+
+ /*
+ * Give up now if I/O is saturated, or we wouldn't be able form
+ * another full range after this due to the pin limit.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger ||
+ pgsr->ios_in_progress == pgsr->max_ios)
+ break;
+ }
+
+ per_buffer_data = (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * pgsr->per_buffer_data_next;
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ pgsr->finished = true;
+ break;
+ }
+ bmr = pgsr->bmr;
+ forknum = pgsr->forknum;
+
+ Assert(pgsr->pinned_buffers < pgsr->max_pinned_buffers);
+
+ buffer = PrepareReadBuffer(bmr,
+ forknum,
+ blocknum,
+ pgsr->strategy,
+ &found);
+ pgsr->pinned_buffers++;
+
+ need_complete = !found;
+
+ /* Is there a head range that we can't extend? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks > 0 &&
+ (!need_complete ||
+ !head_range->need_complete ||
+ head_range->blocknum + head_range->nblocks != blocknum))
+ {
+ /* Yes, time to start building a new one. */
+ head_range = pg_streaming_read_new_range(pgsr);
+ Assert(head_range->nblocks == 0);
+ }
+
+ if (head_range->nblocks == 0)
+ {
+ /* Initialize a new range beginning at this block. */
+ head_range->blocknum = blocknum;
+ head_range->need_complete = need_complete;
+ head_range->advice_issued = false;
+ }
+ else
+ {
+ /* We can extend an existing range by one block. */
+ Assert(head_range->blocknum + head_range->nblocks == blocknum);
+ Assert(head_range->need_complete);
+ }
+
+ head_range->per_buffer_data_index[head_range->nblocks] = pgsr->per_buffer_data_next++;
+ head_range->buffers[head_range->nblocks] = buffer;
+ head_range->nblocks++;
+
+ if (pgsr->per_buffer_data_next == pgsr->max_pinned_buffers)
+ pgsr->per_buffer_data_next = 0;
+
+ } while (pgsr->pinned_buffers < pgsr->max_pinned_buffers &&
+ pgsr->ios_in_progress < pgsr->max_ios);
+
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ pg_streaming_read_new_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* See if we have one buffer to return. */
+ while (pgsr->tail != pgsr->head)
+ {
+ PgStreamingReadRange *tail_range;
+
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to perform an I/O before returning the buffers from this
+ * range?
+ */
+ if (tail_range->need_complete)
+ {
+ CompleteReadBuffers(pgsr->bmr,
+ tail_range->buffers,
+ pgsr->forknum,
+ tail_range->blocknum,
+ tail_range->nblocks,
+ false,
+ pgsr->strategy);
+ tail_range->need_complete = false;
+
+ /*
+ * We don't really know if the kernel generated an physical I/O
+ * when we issued advice, let alone when it finished, but it has
+ * certainly finished after a read call returns.
+ */
+ if (tail_range->advice_issued)
+ pgsr->ios_in_progress--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = (char *) pgsr->per_buffer_data +
+ tail_range->per_buffer_data_index[buffer_index] *
+ pgsr->per_buffer_data_size;
+
+ return buffer;
+ }
+
+ /* Advance tail to next range, if there is one. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead, and unpin anything that wasn't consumed. */
+ pgsr->finished = true;
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7d601bef6dd..2157a97b973 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -472,7 +472,7 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool *hit);
@@ -501,7 +501,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -795,15 +795,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
+ buf = ReadBuffer_common(BMR_REL(reln),
forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+
return buf;
}
@@ -827,8 +821,9 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
SMgrRelation smgr = smgropen(rlocator, InvalidBackendId);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
mode, strategy, &hit);
}
@@ -1002,7 +997,7 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
bool hit;
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
+ buffer = ReadBuffer_common(bmr,
fork, extend_to - 1, mode, strategy,
&hit);
}
@@ -1016,18 +1011,11 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool *hit)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ Buffer buffer;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1046,175 +1034,339 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ *hit = false;
+
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ buffer = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum,
+ strategy,
+ hit);
+
+ /* At this point we do NOT hold any locks. */
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ {
+ /* if we just want zeroes and a lock, we're done */
+ ZeroBuffer(buffer, mode);
+ }
+ else if (!*hit)
+ {
+ /* we might need to perform I/O */
+ CompleteReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ 1,
+ mode == RBM_ZERO_ON_ERROR,
+ strategy);
+ }
+
+ return buffer;
+}
+
+/*
+ * Prepare to read a block. The buffer is pinned. If this is a 'hit', then
+ * the returned buffer can be used immediately. Otherwise, a physical read
+ * should be completed with CompleteReadBuffers(), or the buffer should be
+ * zeroed with ZeroBuffer(). PrepareReadBuffer() followed by
+ * CompleteReadBuffers() or ZeroBuffer() is equivalent to ReadBuffer(), but
+ * the caller has the opportunity to combine reads of multiple neighboring
+ * blocks into one CompleteReadBuffers() call.
+ *
+ * *foundPtr is set to true for a hit, and false for a miss.
+ */
+Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * CompleteReadBuffers() (so, not for hits, and not for buffers that
+ * are zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+static inline bool
+CompleteReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
+ {
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+/*
+ * Complete a set reads prepared with PrepareReadBuffers(). The buffers must
+ * cover a cluster of neighboring block numbers.
+ *
+ * Typically this performs one physical vector read covering the block range,
+ * but if some of the buffers have already been read in the meantime by any
+ * backend, zero or multiple reads may be performed.
+ */
+void
+CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy)
+{
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(strategy);
+ io_object = IOOBJECT_RELATION;
+ }
/*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
*/
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
else
+ pgBufferUsage.shared_blks_read += nblocks;
+
+ for (int i = 0; i < nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+#ifdef USE_ASSERT_CHECKING
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ /*
+ * We could get all the information from buffer headers, but it can be
+ * expensive to access buffer header cache lines so we make the caller
+ * provide all the information we need, and assert that it is
+ * consistent.
+ */
+ {
+ RelFileLocator xlocator;
+ ForkNumber xforknum;
+ BlockNumber xblocknum;
+
+ BufferGetTag(buffers[i], &xlocator, &xforknum, &xblocknum);
+ Assert(RelFileLocatorEquals(bmr.smgr->smgr_rlocator.locator, xlocator));
+ Assert(xforknum == forknum);
+ Assert(xblocknum == blocknum + i);
+ }
+#endif
+
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!CompleteReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ CompleteReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
{
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
}
else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- }
- }
-
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
- {
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
- }
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
- if (isLocalBuf)
- {
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if (zero_on_error || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
- }
- else
- {
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
- }
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
- return BufferDescriptorGetBuffer(bufHdr);
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
@@ -1228,11 +1380,8 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false. A read should be
+ * performed with CompleteReadBuffers().
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1291,19 +1440,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1368,19 +1508,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1412,15 +1543,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1774,7 +1899,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2043,7 +2168,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2066,7 +2191,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2381,7 +2506,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if PrepareReadBuffer() was called and
+ * CompleteReadBuffers() hasn't been called yet. We'll check by
+ * loading the flags without locking. This is racy, but it's OK to
+ * return false spuriously: when CompleteReadBuffers() calls
+ * StartBufferIO(), it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2390,7 +2520,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3458,7 +3588,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -4845,6 +4975,46 @@ ConditionalLockBuffer(Buffer buffer)
LW_EXCLUSIVE);
}
+/*
+ * Zero a buffer, and lock it as RBM_ZERO_AND_LOCK or
+ * RBM_ZERO_AND_CLEANUP_LOCK would. The buffer must be already pinned. It
+ * does not have to be valid, but it is valid and locked on return.
+ */
+void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* Verify that this backend is pinning the buffer exactly once.
*
@@ -5197,9 +5367,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5212,6 +5388,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 1be4f4f8daf..717b8f58daf 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -109,10 +109,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -288,7 +287,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -298,9 +297,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 563a0be5c74..0d7272e796e 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -147,7 +147,9 @@ smgrshutdown(int code, Datum arg)
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
- * This does not attempt to actually open the underlying file.
+ * This does not attempt to actually open the underlying files. The returned
+ * object remains valid at least until AtEOXact_SMgr() is called, or until
+ * smgrdestroy() is called in non-transaction backends.
*/
SMgrRelation
smgropen(RelFileLocator rlocator, BackendId backend)
@@ -259,10 +261,10 @@ smgrexists(SMgrRelation reln, ForkNumber forknum)
}
/*
- * smgrclose() -- Close and delete an SMgrRelation object.
+ * smgrdestroy() -- Delete an SMgrRelation object.
*/
void
-smgrclose(SMgrRelation reln)
+smgrdestroy(SMgrRelation reln)
{
SMgrRelation *owner;
ForkNumber forknum;
@@ -289,12 +291,14 @@ smgrclose(SMgrRelation reln)
}
/*
- * smgrrelease() -- Release all resources used by this object.
+ * smgrclose() -- Release all resources used by this object.
*
- * The object remains valid.
+ * The object remains valid, but is moved to the unknown list where it will
+ * be destroyed by AtEOXact_SMgr(). It may be re-owned if it is accessed by a
+ * relation before then.
*/
void
-smgrrelease(SMgrRelation reln)
+smgrclose(SMgrRelation reln)
{
for (ForkNumber forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
@@ -302,15 +306,20 @@ smgrrelease(SMgrRelation reln)
reln->smgr_cached_nblocks[forknum] = InvalidBlockNumber;
}
reln->smgr_targblock = InvalidBlockNumber;
+
+ if (reln->smgr_owner)
+ {
+ *reln->smgr_owner = NULL;
+ reln->smgr_owner = NULL;
+ dlist_push_tail(&unowned_relns, &reln->node);
+ }
}
/*
- * smgrreleaseall() -- Release resources used by all objects.
- *
- * This is called for PROCSIGNAL_BARRIER_SMGRRELEASE.
+ * smgrcloseall() -- Close all objects.
*/
void
-smgrreleaseall(void)
+smgrcloseall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -322,14 +331,17 @@ smgrreleaseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrrelease(reln);
+ smgrclose(reln);
}
/*
- * smgrcloseall() -- Close all existing SMgrRelation objects.
+ * smgrdestroyall() -- Destroy all SMgrRelation objects.
+ *
+ * It must be known that there are no pointers to SMgrRelations, other than
+ * those registered with smgrsetowner().
*/
void
-smgrcloseall(void)
+smgrdestroyall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -341,7 +353,7 @@ smgrcloseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrclose(reln);
+ smgrdestroy(reln);
}
/*
@@ -733,7 +745,8 @@ smgrimmedsync(SMgrRelation reln, ForkNumber forknum)
* AtEOXact_SMgr
*
* This routine is called during transaction commit or abort (it doesn't
- * particularly care which). All transient SMgrRelation objects are closed.
+ * particularly care which). All transient SMgrRelation objects are
+ * destroyed.
*
* We do this as a compromise between wanting transient SMgrRelations to
* live awhile (to amortize the costs of blind writes of multiple blocks)
@@ -747,7 +760,7 @@ AtEOXact_SMgr(void)
dlist_mutable_iter iter;
/*
- * Zap all unowned SMgrRelations. We rely on smgrclose() to remove each
+ * Zap all unowned SMgrRelations. We rely on smgrdestroy() to remove each
* one from the list.
*/
dlist_foreach_modify(iter, &unowned_relns)
@@ -757,7 +770,7 @@ AtEOXact_SMgr(void)
Assert(rel->smgr_owner == NULL);
- smgrclose(rel);
+ smgrdestroy(rel);
}
}
@@ -768,6 +781,6 @@ AtEOXact_SMgr(void)
bool
ProcessBarrierSmgrRelease(void)
{
- smgrreleaseall();
+ smgrcloseall();
return true;
}
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..a38f1acb37a 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,18 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+extern Buffer PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr);
+extern void CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -247,9 +265,13 @@ extern void LockBufferForCleanup(Buffer buffer);
extern bool ConditionalLockBufferForCleanup(Buffer buffer);
extern bool IsBufferCleanupOK(Buffer buffer);
extern bool HoldingBufferPinThatDelaysRecovery(void);
+extern void ZeroBuffer(Buffer buffer, ReadBufferMode mode);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 527cd2a0568..d8ffe397faf 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -85,8 +85,8 @@ extern void smgrclearowner(SMgrRelation *owner, SMgrRelation reln);
extern void smgrclose(SMgrRelation reln);
extern void smgrcloseall(void);
extern void smgrcloserellocator(RelFileLocatorBackend rlocator);
-extern void smgrrelease(SMgrRelation reln);
-extern void smgrreleaseall(void);
+extern void smgrdestroy(SMgrRelation reln);
+extern void smgrdestroyall(void);
extern void smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo);
extern void smgrdosyncall(SMgrRelation *rels, int nrels);
extern void smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 00000000000..40c3408c541
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,45 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index a584b1ddff3..6636cc82c09 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -561,12 +561,6 @@ typedef struct ViewOptions
*
* Very little code is authorized to touch rel->rd_smgr directly. Instead
* use this function to fetch its value.
- *
- * Note: since a relcache flush can cause the file handle to be closed again,
- * it's unwise to hold onto the pointer returned by this function for any
- * long period. Recommended practice is to just re-execute RelationGetSmgr
- * each time you need to access the SMgrRelation. It's quite cheap in
- * comparison to whatever an smgr function is going to do.
*/
static inline SMgrRelation
RelationGetSmgr(Relation rel)
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 91433d439b7..8007f17320a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2094,6 +2094,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
--
2.37.2
[text/x-patch] v1-0011-BitmapHeapScan-uses-streaming-read-API.patch (36.2K, ../../CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA@mail.gmail.com/12-v1-0011-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From aac60985d6bc70bfedf77a77ee3c512da87bfcb1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:27:57 -0500
Subject: [PATCH v1 11/11] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
---
src/backend/access/gin/ginget.c | 15 +-
src/backend/access/gin/ginscan.c | 7 +
src/backend/access/heap/heapam.c | 71 +++++
src/backend/access/heap/heapam_handler.c | 78 +++--
src/backend/executor/nodeBitmapHeapscan.c | 328 +---------------------
src/backend/nodes/tidbitmap.c | 80 +++---
src/include/access/heapam.h | 2 +
src/include/access/tableam.h | 14 +-
src/include/nodes/execnodes.h | 19 --
src/include/nodes/tidbitmap.h | 8 +-
10 files changed, 178 insertions(+), 444 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3ce28078a6f 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -373,7 +373,10 @@ restartScanEntry:
if (entry->matchBitmap)
{
if (entry->matchIterator)
+ {
tbm_end_iterate(entry->matchIterator);
+ pfree(entry->matchResult);
+ }
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -386,6 +389,7 @@ restartScanEntry:
if (entry->matchBitmap && !tbm_is_empty(entry->matchBitmap))
{
entry->matchIterator = tbm_begin_iterate(entry->matchBitmap);
+ entry->matchResult = palloc0(TBM_ITERATE_RESULT_SIZE);
entry->isFinished = false;
}
}
@@ -823,21 +827,24 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
+ while ((!BlockNumberIsValid(entry->matchResult->blockno)) ||
(entry->matchResult->ntuples >= 0 &&
entry->offset >= entry->matchResult->ntuples) ||
entry->matchResult->blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
entry->matchResult->blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
- if (entry->matchResult == NULL)
+ tbm_iterate(entry->matchIterator, entry->matchResult);
+ if (!BlockNumberIsValid(entry->matchResult->blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
+ pfree(entry->matchResult);
+ entry->matchResult = NULL;
entry->matchIterator = NULL;
entry->isFinished = true;
break;
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..be27f9fe07e 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -246,7 +246,14 @@ ginFreeScanKeys(GinScanOpaque so)
if (entry->list)
pfree(entry->list);
if (entry->matchIterator)
+ {
tbm_end_iterate(entry->matchIterator);
+ if (entry->matchResult)
+ {
+ pfree(entry->matchResult);
+ entry->matchResult = NULL;
+ }
+ }
if (entry->matchBitmap)
tbm_free(entry->matchBitmap);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d8569373987..86484c6c72a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -115,6 +115,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next_single(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -335,6 +337,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->pgsr)
+ pg_streaming_read_free(scan->pgsr);
+
+ scan->pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ TBM_ITERATE_RESULT_SIZE,
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next_single);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -955,6 +973,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->pgsr = NULL;
scan->vmbuffer = InvalidBuffer;
scan->empty_tuples = 0;
scan->rs_base.lossy_pages = 0;
@@ -1113,6 +1132,13 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->pgsr)
+ pg_streaming_read_free(scan->pgsr);
+ scan->pgsr = NULL;
+ }
+
pfree(scan);
}
@@ -10270,3 +10296,48 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next_single(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ if (hdesc->rs_base.shared_tbmiterator)
+ tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
+ else
+ tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+
+ if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->vmbuffer))
+ {
+ hdesc->empty_tuples += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d55ece23a35..0cd586cd4b8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2113,77 +2113,65 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno)
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
TBMIterateResult *tbmres;
+ Assert(hscan->pgsr);
+
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
+
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->pgsr, &io_private);
- if (tbmres == NULL)
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->empty_tuples == 0);
- return false;
+ ReleaseBuffer(hscan->vmbuffer);
+ hscan->vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API will
+ * only return TBMIterateResults for blocks actually fetched. When we
+ * skip fetching a block, we keep track of how many empty tuples to
+ * emit at the end of the BitmapHeapScan. We do not recheck all NULL
+ * tuples.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->empty_tuples > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, and the bitmap entries don't need rechecking, and all tuples on
- * the page are visible to our transaction.
- */
- if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
- Assert(hscan->empty_tuples >= 0);
+ tbmres = (TBMIterateResult *) io_private;
- hscan->empty_tuples += tbmres->ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres->blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index efc6952e353..8b7f87a4779 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,11 +56,6 @@ static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapAccumCounters(BitmapHeapScanState *node,
TableScanDesc scan);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -124,15 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
}
else
{
@@ -155,20 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
- }
-#endif
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
@@ -176,14 +148,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
if (!scan)
@@ -203,46 +167,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->initialized = true;
/* Get the first block. if none, end of scan */
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck))
goto exit;
- BitmapAdjustPrefetchIterator(node, node->blockno);
- BitmapAdjustPrefetchTarget(node);
}
- for (;;)
+ do
{
while (table_scan_bitmap_next_tuple(scan, slot))
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We prefetch before fetching the current pages. We expect that a
- * future streaming read API will do this, so do it now for
- * consistency.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -263,13 +197,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
return slot;
}
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
- break;
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
+ } while (table_scan_bitmap_next_block(scan, &node->recheck));
/*
* if we get here it means we are at the end of the scan..
@@ -304,215 +232,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
- }
- return;
- }
-
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- bool skip_fetch;
-
- if (tbmpre == NULL)
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
- */
-
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult *tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
@@ -559,22 +278,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -606,16 +315,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
ExecEndNode(outerPlanState(node));
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
/*
* close heap scan
@@ -654,19 +357,13 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -706,13 +403,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -796,14 +486,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, node->pscan_len);
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -835,11 +521,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 0f4850065fb..ccb511fb608 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -180,7 +180,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
};
/*
@@ -221,7 +220,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
};
/* Local function prototypes */
@@ -695,8 +693,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
* Create the TBMIterator struct, with enough trailing space to serve the
* needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMIterator *) palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -957,20 +954,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -998,6 +996,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -1008,11 +1007,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1028,18 +1027,20 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
+
/*
* tbm_shared_iterate - scan through next page of a TIDBitmap
*
@@ -1047,10 +1048,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1101,13 +1101,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1117,21 +1117,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1470,8 +1471,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
* Create the TBMSharedIterator struct, with enough trailing space to
* serve the needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2fc369a18ff..33e8a7e0bba 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -73,6 +74,7 @@ typedef struct HeapScanDescData
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
/* these fields only used in page-at-a-time mode and for bitmap scans */
+ PgStreamingRead *pgsr;
Buffer vmbuffer; /* for checking if can skip fetch */
int empty_tuples; /* count of all NULL tuples to be returned */
int rs_cindex; /* current tuple's index in vistuples */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b90d9b7f3fa..adde320d1eb 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -788,17 +788,10 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, BlockNumber *blockno);
+ bool (*scan_bitmap_next_block) (TableScanDesc scan, bool *recheck);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1950,8 +1943,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* used after verifying the presence (at plan time or such).
*/
static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno)
+table_scan_bitmap_next_block(TableScanDesc scan, bool *recheck)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1961,7 +1953,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, blockno);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 03973a3f262..96afabc67e6 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1682,11 +1682,8 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
* mutex mutual exclusion for the prefetching variable
* and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
* phs_snapshot_data snapshot data shared to workers
@@ -1695,10 +1692,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
char phs_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
@@ -1709,16 +1703,10 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
* ----------------
@@ -1728,20 +1716,13 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
bool recheck;
- BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..672608200ba 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -64,12 +64,16 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
dsa_pointer dp);
extern long tbm_calculate_entries(double maxbytes);
+#define TBM_ITERATE_RESULT_SIZE \
+ (offsetof(TBMIterateResult, offsets) + \
+ MaxHeapTuplesPerPage * sizeof(OffsetNumber))
+
#endif /* TIDBITMAP_H */
--
2.37.2
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-14 04:34 ` Mark Dilger <mark.dilger@enterprisedb.com>
2024-02-14 14:47 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Mark Dilger @ 2024-02-14 04:34 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: pgsql-hackers; Andres Freund <andres@anarazel.de>; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
> On Feb 13, 2024, at 3:11 PM, Melanie Plageman <melanieplageman@gmail.com> wrote:
Thanks for the patch...
> Attached is a patch set which refactors BitmapHeapScan such that it
> can use the streaming read API [1]. It also resolves the long-standing
> FIXME in the BitmapHeapScan code suggesting that the skip fetch
> optimization should be pushed into the table AMs. Additionally, it
> moves table scan initialization to after the index scan and bitmap
> initialization.
>
> patches 0001-0002 are assorted cleanup needed later in the set.
> patches 0003 moves the table scan initialization to after bitmap creation
> patch 0004 is, I think, a bug fix. see [2].
> patches 0005-0006 push the skip fetch optimization into the table AMs
> patches 0007-0009 change the control flow of BitmapHeapNext() to match
> that required by the streaming read API
> patch 0010 is the streaming read code not yet in master
> patch 0011 is the actual bitmapheapscan streaming read user.
>
> patches 0001-0009 apply on top of master but 0010 and 0011 must be
> applied on top of a commit before a 21d9c3ee4ef74e2 (until a rebased
> version of the streaming read API is on the mailing list).
I followed your lead and applied them to 6a8ffe812d194ba6f4f26791b6388a4837d17d6c. `make check` worked fine, though I expect you know that already.
> The caveat is that these patches introduce breaking changes to two
> table AM functions for bitmapheapscan: table_scan_bitmap_next_block()
> and table_scan_bitmap_next_tuple().
You might want an independent perspective on how much of a hassle those breaking changes are, so I took a stab at that. Having written a custom proprietary TAM for postgresql 15 here at EDB, and having ported it and released it for postgresql 16, I thought I'd try porting it to the the above commit with your patches. Even without your patches, I already see breaking changes coming from commit f691f5b80a85c66d715b4340ffabb503eb19393e, which creates a similar amount of breakage for me as does your patches. Dealing with the combined breakage might amount to a day of work, including testing, half of which I think I've already finished. In other words, it doesn't seem like a big deal.
Were postgresql 17 shaping up to be compatible with TAMs written for 16, your patch would change that qualitatively, but since things are already incompatible, I think you're in the clear.
> A TBMIterateResult used to be threaded through both of these functions
> and used in BitmapHeapNext(). This patch set removes all references to
> TBMIterateResults from BitmapHeapNext. Because the streaming read API
> requires the callback to specify the next block, BitmapHeapNext() can
> no longer pass a TBMIterateResult to table_scan_bitmap_next_block().
>
> More subtly, table_scan_bitmap_next_block() used to return false if
> there were no more visible tuples on the page or if the block that was
> requested was not valid. With these changes,
> table_scan_bitmap_next_block() will only return false when the bitmap
> has been exhausted and the scan can end. In order to use the streaming
> read API, the user must be able to request the blocks it needs without
> requiring synchronous feedback per block. Thus, this table AM function
> must change its meaning.
>
> I think the way the patches are split up could be improved. I will
> think more about this. There are also probably a few mistakes with
> which comments are updated in which patches in the set.
I look forward to the next version of the patch set. Thanks again for working on this.
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 04:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Mark Dilger <mark.dilger@enterprisedb.com>
@ 2024-02-14 14:47 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 16:41 ` Re: BitmapHeapScan streaming read user and prelim refactoring Mark Dilger <mark.dilger@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-14 14:47 UTC (permalink / raw)
To: Mark Dilger <mark.dilger@enterprisedb.com>; +Cc: pgsql-hackers; Andres Freund <andres@anarazel.de>; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Tue, Feb 13, 2024 at 11:34 PM Mark Dilger
<mark.dilger@enterprisedb.com> wrote:
>
> > On Feb 13, 2024, at 3:11 PM, Melanie Plageman <melanieplageman@gmail.com> wrote:
>
> Thanks for the patch...
>
> > Attached is a patch set which refactors BitmapHeapScan such that it
> > can use the streaming read API [1]. It also resolves the long-standing
> > FIXME in the BitmapHeapScan code suggesting that the skip fetch
> > optimization should be pushed into the table AMs. Additionally, it
> > moves table scan initialization to after the index scan and bitmap
> > initialization.
> >
> > patches 0001-0002 are assorted cleanup needed later in the set.
> > patches 0003 moves the table scan initialization to after bitmap creation
> > patch 0004 is, I think, a bug fix. see [2].
> > patches 0005-0006 push the skip fetch optimization into the table AMs
> > patches 0007-0009 change the control flow of BitmapHeapNext() to match
> > that required by the streaming read API
> > patch 0010 is the streaming read code not yet in master
> > patch 0011 is the actual bitmapheapscan streaming read user.
> >
> > patches 0001-0009 apply on top of master but 0010 and 0011 must be
> > applied on top of a commit before a 21d9c3ee4ef74e2 (until a rebased
> > version of the streaming read API is on the mailing list).
>
> I followed your lead and applied them to 6a8ffe812d194ba6f4f26791b6388a4837d17d6c. `make check` worked fine, though I expect you know that already.
Thanks for taking a look!
> > The caveat is that these patches introduce breaking changes to two
> > table AM functions for bitmapheapscan: table_scan_bitmap_next_block()
> > and table_scan_bitmap_next_tuple().
>
> You might want an independent perspective on how much of a hassle those breaking changes are, so I took a stab at that. Having written a custom proprietary TAM for postgresql 15 here at EDB, and having ported it and released it for postgresql 16, I thought I'd try porting it to the the above commit with your patches. Even without your patches, I already see breaking changes coming from commit f691f5b80a85c66d715b4340ffabb503eb19393e, which creates a similar amount of breakage for me as does your patches. Dealing with the combined breakage might amount to a day of work, including testing, half of which I think I've already finished. In other words, it doesn't seem like a big deal.
>
> Were postgresql 17 shaping up to be compatible with TAMs written for 16, your patch would change that qualitatively, but since things are already incompatible, I think you're in the clear.
Oh, good to know! I'm very happy to have the perspective of a table AM
author. Just curious, did your table AM implement
table_scan_bitmap_next_block() and table_scan_bitmap_next_tuple(),
and, if so, did you use the TBMIterateResult? Since it is not used in
BitmapHeapNext() in my version, table AMs would have to change how
they use TBMIterateResults anyway. But I assume they could add it to a
table AM specific scan descriptor if they want access to a
TBMIterateResult of their own making in both
table_san_bitmap_next_block() and next_tuple()?
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 04:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Mark Dilger <mark.dilger@enterprisedb.com>
2024-02-14 14:47 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-14 16:41 ` Mark Dilger <mark.dilger@enterprisedb.com>
0 siblings, 0 replies; 248+ messages in thread
From: Mark Dilger @ 2024-02-14 16:41 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: pgsql-hackers; Andres Freund <andres@anarazel.de>; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
> On Feb 14, 2024, at 6:47 AM, Melanie Plageman <melanieplageman@gmail.com> wrote:
>
> Just curious, did your table AM implement
> table_scan_bitmap_next_block() and table_scan_bitmap_next_tuple(),
> and, if so, did you use the TBMIterateResult? Since it is not used in
> BitmapHeapNext() in my version, table AMs would have to change how
> they use TBMIterateResults anyway. But I assume they could add it to a
> table AM specific scan descriptor if they want access to a
> TBMIterateResult of their own making in both
> table_san_bitmap_next_block() and next_tuple()?
My table AM does implement those two functions and does use the TBMIterateResult *tbmres argument, yes. I would deal with the issue in very much the same way that your patches modify heapam. I don't really have any additional comments about that.
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-14 19:42 ` Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 2 replies; 248+ messages in thread
From: Andres Freund @ 2024-02-14 19:42 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
Hi,
On 2024-02-13 18:11:25 -0500, Melanie Plageman wrote:
> Attached is a patch set which refactors BitmapHeapScan such that it
> can use the streaming read API [1]. It also resolves the long-standing
> FIXME in the BitmapHeapScan code suggesting that the skip fetch
> optimization should be pushed into the table AMs. Additionally, it
> moves table scan initialization to after the index scan and bitmap
> initialization.
Thanks for working on this! While I have some quibbles with details, I think
this is quite a bit of progress in the right direction.
> patches 0001-0002 are assorted cleanup needed later in the set.
> patches 0003 moves the table scan initialization to after bitmap creation
> patch 0004 is, I think, a bug fix. see [2].
I'd not quite call it a bugfix, it's not like it leads to wrong
behaviour. Seems more like an optimization. But whatever :)
> The caveat is that these patches introduce breaking changes to two
> table AM functions for bitmapheapscan: table_scan_bitmap_next_block()
> and table_scan_bitmap_next_tuple().
That's to be expected, I don't think it's worth worrying about. Right now a
bunch of TAMs can't implement bitmap scans, this goes a fair bit towards
allowing that...
> From d6dd6eb21dcfbc41208f87d1d81ffe3960130889 Mon Sep 17 00:00:00 2001
> From: Melanie Plageman <melanieplageman@gmail.com>
> Date: Mon, 12 Feb 2024 18:50:29 -0500
> Subject: [PATCH v1 03/11] BitmapHeapScan begin scan after bitmap setup
>
> There is no reason for table_beginscan_bm() to begin the actual scan of
> the underlying table in ExecInitBitmapHeapScan(). We can begin the
> underlying table scan after the index scan has been completed and the
> bitmap built.
>
> The one use of the scan descriptor during initialization was
> ExecBitmapHeapInitializeWorker(), which set the scan descriptor snapshot
> with one from an array in the parallel state. This overwrote the
> snapshot set in table_beginscan_bm().
>
> By saving that worker snapshot as a member in the BitmapHeapScanState
> during initialization, it can be restored in table_beginscan_bm() after
> returning from the table AM specific begin scan function.
I don't understand what the point of passing two different snapshots to
table_beginscan_bm() is. What does that even mean? Why can't we just use the
correct snapshot initially?
> From a3f62e4299663d418531ae61bb16ea39f0836fac Mon Sep 17 00:00:00 2001
> From: Melanie Plageman <melanieplageman@gmail.com>
> Date: Mon, 12 Feb 2024 19:03:24 -0500
> Subject: [PATCH v1 04/11] BitmapPrefetch use prefetch block recheck for skip
> fetch
>
> Previously BitmapPrefetch() used the recheck flag for the current block
> to determine whether or not it could skip prefetching the proposed
> prefetch block. It makes more sense for it to use the recheck flag from
> the TBMIterateResult for the prefetch block instead.
I'd mention the commit that introduced the current logic and link to the
the thread that you started about this.
> From d56be7741765d93002649ef912ef4b8256a5b9af Mon Sep 17 00:00:00 2001
> From: Melanie Plageman <melanieplageman@gmail.com>
> Date: Mon, 12 Feb 2024 19:04:48 -0500
> Subject: [PATCH v1 05/11] Update BitmapAdjustPrefetchIterator parameter type
> to BlockNumber
>
> BitmapAdjustPrefetchIterator() only used the blockno member of the
> passed in TBMIterateResult to ensure that the prefetch iterator and
> regular iterator stay in sync. Pass it the BlockNumber only. This will
> allow us to move away from using the TBMIterateResult outside of table
> AM specific code.
Hm - I'm not convinced this is a good direction - doesn't that arguably
*increase* TAM awareness? Perhaps it doesn't make much sense to use bitmap
heap scans in a TAM without blocks, but still.
> From 202b16d3a381210e8dbee69e68a8310be8ee11d2 Mon Sep 17 00:00:00 2001
> From: Melanie Plageman <melanieplageman@gmail.com>
> Date: Mon, 12 Feb 2024 20:15:05 -0500
> Subject: [PATCH v1 06/11] Push BitmapHeapScan skip fetch optimization into
> table AM
>
> This resolves the long-standing FIXME in BitmapHeapNext() which said that
> the optmization to skip fetching blocks of the underlying table when
> none of the column data was needed should be pushed into the table AM
> specific code.
Long-standing? Sure, it's old enough to walk, but we have FIXMEs that are old
enough to drink, at least in some countries. :)
> The table AM agnostic functions for prefetching still need to know if
> skipping fetching is permitted for this scan. However, this dependency
> will be removed when that prefetching code is removed in favor of the
> upcoming streaming read API.
> ---
> src/backend/access/heap/heapam.c | 10 +++
> src/backend/access/heap/heapam_handler.c | 29 +++++++
> src/backend/executor/nodeBitmapHeapscan.c | 100 ++++++----------------
> src/include/access/heapam.h | 2 +
> src/include/access/tableam.h | 17 ++--
> src/include/nodes/execnodes.h | 6 --
> 6 files changed, 74 insertions(+), 90 deletions(-)
>
> diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
> index 707460a5364..7aae1ecf0a9 100644
> --- a/src/backend/access/heap/heapam.c
> +++ b/src/backend/access/heap/heapam.c
> @@ -955,6 +955,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> scan->rs_base.rs_flags = flags;
> scan->rs_base.rs_parallel = parallel_scan;
> scan->rs_strategy = NULL; /* set in initscan */
> + scan->vmbuffer = InvalidBuffer;
> + scan->empty_tuples = 0;
These don't follow the existing naming pattern for HeapScanDescData. While I
explicitly dislike the practice of adding prefixes to struct members, I don't
think mixing conventions within a single struct improves things.
I also think it'd be good to note in comments that the vm buffer currently is
only used for bitmap heap scans, otherwise one might think they'd also be used
for normal scans, where we don't need them, because of the page level flag.
Also, perhaps worth renaming "empty_tuples" to something indicating that it's
the number of empty tuples to be returned later? num_empty_tuples_pending or
such? Or the current "return_empty_tuples".
> @@ -1043,6 +1045,10 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
> if (BufferIsValid(scan->rs_cbuf))
> ReleaseBuffer(scan->rs_cbuf);
>
> + if (BufferIsValid(scan->vmbuffer))
> + ReleaseBuffer(scan->vmbuffer);
> + scan->vmbuffer = InvalidBuffer;
It does not matter one iota here, but personally I prefer moving the write
inside the if, as dirtying the cacheline after we just figured out whe
> diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
> index 9372b49bfaa..c0fb06c9688 100644
> --- a/src/backend/executor/nodeBitmapHeapscan.c
> +++ b/src/backend/executor/nodeBitmapHeapscan.c
> @@ -108,6 +108,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
> */
> if (!node->initialized)
> {
> + bool can_skip_fetch;
> /*
> * We can potentially skip fetching heap pages if we do not need any
> * columns of the table, either for checking non-indexable quals or
Pretty sure pgindent will move this around.
> +++ b/src/include/access/tableam.h
> @@ -62,6 +62,7 @@ typedef enum ScanOptions
>
> /* unregister snapshot at scan end? */
> SO_TEMP_SNAPSHOT = 1 << 9,
> + SO_CAN_SKIP_FETCH = 1 << 10,
> } ScanOptions;
Would be nice to add a comment explaining what this flag means.
> From 500c84019b982a1e6c8b8dd40240c8510d83c287 Mon Sep 17 00:00:00 2001
> From: Melanie Plageman <melanieplageman@gmail.com>
> Date: Tue, 13 Feb 2024 10:05:04 -0500
> Subject: [PATCH v1 07/11] BitmapHeapScan scan desc counts lossy and exact
> pages
>
> Future commits will remove the TBMIterateResult from BitmapHeapNext(),
> pushing it into the table AM-specific code. So we will have to keep
> track of the number of lossy and exact pages in the scan descriptor.
> Doing this change to lossy/exact page counting in a separate commit just
> simplifies the diff.
> ---
> src/backend/access/heap/heapam.c | 2 ++
> src/backend/access/heap/heapam_handler.c | 9 +++++++++
> src/backend/executor/nodeBitmapHeapscan.c | 18 +++++++++++++-----
> src/include/access/relscan.h | 4 ++++
> 4 files changed, 28 insertions(+), 5 deletions(-)
>
> diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
> index 7aae1ecf0a9..88b4aad5820 100644
> --- a/src/backend/access/heap/heapam.c
> +++ b/src/backend/access/heap/heapam.c
> @@ -957,6 +957,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> scan->rs_strategy = NULL; /* set in initscan */
> scan->vmbuffer = InvalidBuffer;
> scan->empty_tuples = 0;
> + scan->rs_base.lossy_pages = 0;
> + scan->rs_base.exact_pages = 0;
>
> /*
> * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
> diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
> index baba09c87c0..6e85ef7a946 100644
> --- a/src/backend/access/heap/heapam_handler.c
> +++ b/src/backend/access/heap/heapam_handler.c
> @@ -2242,6 +2242,15 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
> Assert(ntup <= MaxHeapTuplesPerPage);
> hscan->rs_ntuples = ntup;
>
> + /* Only count exact and lossy pages with visible tuples */
> + if (ntup > 0)
> + {
> + if (tbmres->ntuples >= 0)
> + scan->exact_pages++;
> + else
> + scan->lossy_pages++;
> + }
> +
> return ntup > 0;
> }
>
> diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
> index c0fb06c9688..19d115de06f 100644
> --- a/src/backend/executor/nodeBitmapHeapscan.c
> +++ b/src/backend/executor/nodeBitmapHeapscan.c
> @@ -53,6 +53,8 @@
> #include "utils/spccache.h"
>
> static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
> +static inline void BitmapAccumCounters(BitmapHeapScanState *node,
> + TableScanDesc scan);
> static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
> static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
> BlockNumber blockno);
> @@ -234,11 +236,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
> continue;
> }
>
> - if (tbmres->ntuples >= 0)
> - node->exact_pages++;
> - else
> - node->lossy_pages++;
> -
> /* Adjust the prefetch target */
> BitmapAdjustPrefetchTarget(node);
> }
> @@ -315,9 +312,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
> /*
> * if we get here it means we are at the end of the scan..
> */
> + BitmapAccumCounters(node, scan);
> return ExecClearTuple(slot);
> }
>
> +static inline void
> +BitmapAccumCounters(BitmapHeapScanState *node,
> + TableScanDesc scan)
> +{
> + node->exact_pages += scan->exact_pages;
> + scan->exact_pages = 0;
> + node->lossy_pages += scan->lossy_pages;
> + scan->lossy_pages = 0;
> +}
> +
I don't think this is quite right - you're calling BitmapAccumCounters() only
when the scan doesn't return anything anymore, but there's no guarantee
that'll ever be reached. E.g. a bitmap heap scan below a limit node. I think
this needs to be in a) ExecEndBitmapHeapScan() b) ExecReScanBitmapHeapScan()
> /*
> * BitmapDoneInitializingSharedState - Shared state is initialized
> *
> diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
> index 521043304ab..b74e08dd745 100644
> --- a/src/include/access/relscan.h
> +++ b/src/include/access/relscan.h
> @@ -40,6 +40,10 @@ typedef struct TableScanDescData
> ItemPointerData rs_mintid;
> ItemPointerData rs_maxtid;
>
> + /* Only used for Bitmap table scans */
> + long exact_pages;
> + long lossy_pages;
> +
> /*
> * Information about type and behaviour of the scan, a bitmask of members
> * of the ScanOptions enum (see tableam.h).
I wonder if this really is the best place for the data to be accumulated. This
requires the accounting to be implemented in each AM, which doesn't obviously
seem required. Why can't the accounting continue to live in
nodeBitmapHeapscan.c, to be done after each table_scan_bitmap_next_block()
call?
> From 555743e4bc885609d20768f7f2990c6ba69b13a9 Mon Sep 17 00:00:00 2001
> From: Melanie Plageman <melanieplageman@gmail.com>
> Date: Tue, 13 Feb 2024 10:57:07 -0500
> Subject: [PATCH v1 09/11] Make table_scan_bitmap_next_block() async friendly
>
> table_scan_bitmap_next_block() previously returned false if we did not
> wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
> This could happen when there were no visible tuples on the page or, due
> to concurrent activity on the table, the block returned by the iterator
> is past the known end of the table.
This sounds a bit like the block is actually past the end of the table,
but in reality this happens if the block is past the end of the table as it
was when the scan was started. Somehow that feels significant, but I don't
really know why I think that.
> diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
> index 88b4aad5820..d8569373987 100644
> --- a/src/backend/access/heap/heapam.c
> +++ b/src/backend/access/heap/heapam.c
> @@ -959,6 +959,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> scan->empty_tuples = 0;
> scan->rs_base.lossy_pages = 0;
> scan->rs_base.exact_pages = 0;
> + scan->rs_base.shared_tbmiterator = NULL;
> + scan->rs_base.tbmiterator = NULL;
>
> /*
> * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
> @@ -1051,6 +1053,18 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
> ReleaseBuffer(scan->vmbuffer);
> scan->vmbuffer = InvalidBuffer;
>
> + if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
> + {
> + if (scan->rs_base.shared_tbmiterator)
> + tbm_end_shared_iterate(scan->rs_base.shared_tbmiterator);
> +
> + if (scan->rs_base.tbmiterator)
> + tbm_end_iterate(scan->rs_base.tbmiterator);
> + }
> +
> + scan->rs_base.shared_tbmiterator = NULL;
> + scan->rs_base.tbmiterator = NULL;
> +
> /*
> * reinitialize scan descriptor
> */
If every AM would need to implement this, perhaps this shouldn't be done here,
but in generic code?
> --- a/src/backend/access/heap/heapam_handler.c
> +++ b/src/backend/access/heap/heapam_handler.c
> @@ -2114,17 +2114,49 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
>
> static bool
> heapam_scan_bitmap_next_block(TableScanDesc scan,
> - TBMIterateResult *tbmres)
> + bool *recheck, BlockNumber *blockno)
> {
> HeapScanDesc hscan = (HeapScanDesc) scan;
> - BlockNumber block = tbmres->blockno;
> + BlockNumber block;
> Buffer buffer;
> Snapshot snapshot;
> int ntup;
> + TBMIterateResult *tbmres;
>
> hscan->rs_cindex = 0;
> hscan->rs_ntuples = 0;
>
> + *blockno = InvalidBlockNumber;
> + *recheck = true;
> +
> + do
> + {
> + if (scan->shared_tbmiterator)
> + tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
> + else
> + tbmres = tbm_iterate(scan->tbmiterator);
> +
> + if (tbmres == NULL)
> + {
> + /* no more entries in the bitmap */
> + Assert(hscan->empty_tuples == 0);
> + return false;
> + }
> +
> + /*
> + * Ignore any claimed entries past what we think is the end of the
> + * relation. It may have been extended after the start of our scan (we
> + * only hold an AccessShareLock, and it could be inserts from this
> + * backend). We don't take this optimization in SERIALIZABLE
> + * isolation though, as we need to examine all invisible tuples
> + * reachable by the index.
> + */
> + } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
Hm. Isn't it a problem that we have no CHECK_FOR_INTERRUPTS() in this loop?
> @@ -2251,7 +2274,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
> scan->lossy_pages++;
> }
>
> - return ntup > 0;
> + /*
> + * Return true to indicate that a valid block was found and the bitmap is
> + * not exhausted. If there are no visible tuples on this page,
> + * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
> + * return false returning control to this function to advance to the next
> + * block in the bitmap.
> + */
> + return true;
> }
Why can't we fetch the next block immediately?
> @@ -201,46 +197,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
> can_skip_fetch);
> }
>
> - node->tbmiterator = tbmiterator;
> - node->shared_tbmiterator = shared_tbmiterator;
> + scan->tbmiterator = tbmiterator;
> + scan->shared_tbmiterator = shared_tbmiterator;
It seems a bit odd that this code modifies the scan descriptor, instead of
passing the iterator, or perhaps better the bitmap itself, to
table_beginscan_bm()?
> diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
> index b74e08dd745..bf7ee044268 100644
> --- a/src/include/access/relscan.h
> +++ b/src/include/access/relscan.h
> @@ -16,6 +16,7 @@
>
> #include "access/htup_details.h"
> #include "access/itup.h"
> +#include "nodes/tidbitmap.h"
I'd like to avoid exposing this to everything including relscan.h. I think we
could just forward declare the structs and use them here to avoid that?
> From aac60985d6bc70bfedf77a77ee3c512da87bfcb1 Mon Sep 17 00:00:00 2001
> From: Melanie Plageman <melanieplageman@gmail.com>
> Date: Tue, 13 Feb 2024 14:27:57 -0500
> Subject: [PATCH v1 11/11] BitmapHeapScan uses streaming read API
>
> Remove all of the code to do prefetching from BitmapHeapScan code and
> rely on the streaming read API prefetching. Heap table AM implements a
> streaming read callback which uses the iterator to get the next valid
> block that needs to be fetched for the streaming read API.
> ---
> src/backend/access/gin/ginget.c | 15 +-
> src/backend/access/gin/ginscan.c | 7 +
> src/backend/access/heap/heapam.c | 71 +++++
> src/backend/access/heap/heapam_handler.c | 78 +++--
> src/backend/executor/nodeBitmapHeapscan.c | 328 +---------------------
> src/backend/nodes/tidbitmap.c | 80 +++---
> src/include/access/heapam.h | 2 +
> src/include/access/tableam.h | 14 +-
> src/include/nodes/execnodes.h | 19 --
> src/include/nodes/tidbitmap.h | 8 +-
> 10 files changed, 178 insertions(+), 444 deletions(-)
>
> diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
> index 0b4f2ebadb6..3ce28078a6f 100644
> --- a/src/backend/access/gin/ginget.c
> +++ b/src/backend/access/gin/ginget.c
> @@ -373,7 +373,10 @@ restartScanEntry:
> if (entry->matchBitmap)
> {
> if (entry->matchIterator)
> + {
> tbm_end_iterate(entry->matchIterator);
> + pfree(entry->matchResult);
> + }
> entry->matchIterator = NULL;
> tbm_free(entry->matchBitmap);
> entry->matchBitmap = NULL;
> @@ -386,6 +389,7 @@ restartScanEntry:
> if (entry->matchBitmap && !tbm_is_empty(entry->matchBitmap))
> {
> entry->matchIterator = tbm_begin_iterate(entry->matchBitmap);
> + entry->matchResult = palloc0(TBM_ITERATE_RESULT_SIZE);
Do we actually have to use palloc0? TBM_ITERATE_RESULT_SIZE ain't small, so
zeroing all of it isn't free.
> +static BlockNumber bitmapheap_pgsr_next_single(PgStreamingRead *pgsr, void *pgsr_private,
> + void *per_buffer_data);
Is it correct to have _single in the name here? Aren't we also using for
parallel scans?
> +static BlockNumber
> +bitmapheap_pgsr_next_single(PgStreamingRead *pgsr, void *pgsr_private,
> + void *per_buffer_data)
> +{
> + TBMIterateResult *tbmres = per_buffer_data;
> + HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
> +
> + for (;;)
> + {
> + if (hdesc->rs_base.shared_tbmiterator)
> + tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
> + else
> + tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
> +
> + /* no more entries in the bitmap */
> + if (!BlockNumberIsValid(tbmres->blockno))
> + return InvalidBlockNumber;
> +
> + /*
> + * Ignore any claimed entries past what we think is the end of the
> + * relation. It may have been extended after the start of our scan (we
> + * only hold an AccessShareLock, and it could be inserts from this
> + * backend). We don't take this optimization in SERIALIZABLE
> + * isolation though, as we need to examine all invisible tuples
> + * reachable by the index.
> + */
> + if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
> + continue;
> +
> +
> + if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
> + !tbmres->recheck &&
> + VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->vmbuffer))
> + {
> + hdesc->empty_tuples += tbmres->ntuples;
> + continue;
> + }
> +
> + return tbmres->blockno;
> + }
> +
> + /* not reachable */
> + Assert(false);
> +}
Need to check for interrupts somewhere here.
> @@ -124,15 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
There's still a comment in BitmapHeapNext talking about prefetching with two
iterators etc. That seems outdated now.
> /*
> * tbm_iterate - scan through next page of a TIDBitmap
> *
> - * Returns a TBMIterateResult representing one page, or NULL if there are
> - * no more pages to scan. Pages are guaranteed to be delivered in numerical
> - * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
> - * remember the exact tuples to look at on this page --- the caller must
> - * examine all tuples on the page and check if they meet the intended
> - * condition. If result->recheck is true, only the indicated tuples need
> - * be examined, but the condition must be rechecked anyway. (For ease of
> - * testing, recheck is always set true when ntuples < 0.)
> + * Caller must pass in a TBMIterateResult to be filled.
> + *
> + * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
> + * set to InvalidBlockNumber when there are no more pages to scan. If
> + * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
> + * exact tuples to look at on this page --- the caller must examine all tuples
> + * on the page and check if they meet the intended condition. If
> + * tbmres->recheck is true, only the indicated tuples need be examined, but the
> + * condition must be rechecked anyway. (For ease of testing, recheck is always
> + * set true when ntuples < 0.)
> */
> -TBMIterateResult *
> -tbm_iterate(TBMIterator *iterator)
> +void
> +tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
Hm - it seems a tad odd that we later have to find out if the scan is done
iterating by checking if blockno is valid, when tbm_iterate already knew. But
I guess the code would be a bit uglier if we needed the result of
tbm_[shared_]iterate(), due to the two functions.
Right now ExecEndBitmapHeapScan() frees the tbm before it does table_endscan()
- which seems problematic, as heap_endscan() will do stuff like
tbm_end_iterate(), which imo shouldn't be called after the tbm has been freed,
even if that works today.
It seems a bit confusing that your changes seem to treat
BitmapHeapScanState->initialized as separate from ->scan, even though afaict
scan should be NULL iff initialized is false and vice versa.
Independent of your patches, but brr, it's ugly that
BitmapShouldInitializeSharedState() blocks.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
@ 2024-02-16 03:31 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-16 03:31 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
Thank you so much for this thorough review!!!!
On Wed, Feb 14, 2024 at 2:42 PM Andres Freund <andres@anarazel.de> wrote:
>
>
> On 2024-02-13 18:11:25 -0500, Melanie Plageman wrote:
>
> > From d6dd6eb21dcfbc41208f87d1d81ffe3960130889 Mon Sep 17 00:00:00 2001
> > From: Melanie Plageman <melanieplageman@gmail.com>
> > Date: Mon, 12 Feb 2024 18:50:29 -0500
> > Subject: [PATCH v1 03/11] BitmapHeapScan begin scan after bitmap setup
> >
> > There is no reason for table_beginscan_bm() to begin the actual scan of
> > the underlying table in ExecInitBitmapHeapScan(). We can begin the
> > underlying table scan after the index scan has been completed and the
> > bitmap built.
> >
> > The one use of the scan descriptor during initialization was
> > ExecBitmapHeapInitializeWorker(), which set the scan descriptor snapshot
> > with one from an array in the parallel state. This overwrote the
> > snapshot set in table_beginscan_bm().
> >
> > By saving that worker snapshot as a member in the BitmapHeapScanState
> > during initialization, it can be restored in table_beginscan_bm() after
> > returning from the table AM specific begin scan function.
>
> I don't understand what the point of passing two different snapshots to
> table_beginscan_bm() is. What does that even mean? Why can't we just use the
> correct snapshot initially?
Indeed. Honestly, it was an unlabeled TODO for me. I wasn't quite sure
how to get the same behavior as in master. Fixed in attached v2.
Now the parallel worker still restores and registers that snapshot in
ExecBitmapHeapInitializeWorker() and then saves it in the
BitmapHeapScanState. We then pass SO_TEMP_SNAPSHOT as an extra flag
(to set rs_flags) to table_beginscan_bm() if there is a parallel
worker snapshot saved in the BitmapHeapScanState.
> > From a3f62e4299663d418531ae61bb16ea39f0836fac Mon Sep 17 00:00:00 2001
> > From: Melanie Plageman <melanieplageman@gmail.com>
> > Date: Mon, 12 Feb 2024 19:03:24 -0500
> > Subject: [PATCH v1 04/11] BitmapPrefetch use prefetch block recheck for skip
> > fetch
> >
> > Previously BitmapPrefetch() used the recheck flag for the current block
> > to determine whether or not it could skip prefetching the proposed
> > prefetch block. It makes more sense for it to use the recheck flag from
> > the TBMIterateResult for the prefetch block instead.
>
> I'd mention the commit that introduced the current logic and link to the
> the thread that you started about this.
Done
> > From d56be7741765d93002649ef912ef4b8256a5b9af Mon Sep 17 00:00:00 2001
> > From: Melanie Plageman <melanieplageman@gmail.com>
> > Date: Mon, 12 Feb 2024 19:04:48 -0500
> > Subject: [PATCH v1 05/11] Update BitmapAdjustPrefetchIterator parameter type
> > to BlockNumber
> >
> > BitmapAdjustPrefetchIterator() only used the blockno member of the
> > passed in TBMIterateResult to ensure that the prefetch iterator and
> > regular iterator stay in sync. Pass it the BlockNumber only. This will
> > allow us to move away from using the TBMIterateResult outside of table
> > AM specific code.
>
> Hm - I'm not convinced this is a good direction - doesn't that arguably
> *increase* TAM awareness? Perhaps it doesn't make much sense to use bitmap
> heap scans in a TAM without blocks, but still.
This is removed in later commits and is an intermediate state to try
and move the TBMIterateResult out of BitmapHeapNext(). I can find
another way to achieve this if it is important.
> > From 202b16d3a381210e8dbee69e68a8310be8ee11d2 Mon Sep 17 00:00:00 2001
> > From: Melanie Plageman <melanieplageman@gmail.com>
> > Date: Mon, 12 Feb 2024 20:15:05 -0500
> > Subject: [PATCH v1 06/11] Push BitmapHeapScan skip fetch optimization into
> > table AM
> >
> > This resolves the long-standing FIXME in BitmapHeapNext() which said that
> > the optmization to skip fetching blocks of the underlying table when
> > none of the column data was needed should be pushed into the table AM
> > specific code.
>
> Long-standing? Sure, it's old enough to walk, but we have FIXMEs that are old
> enough to drink, at least in some countries. :)
;) I've updated the commit message. Though it is longstanding in that
it predates Melanie + Postgres.
> > The table AM agnostic functions for prefetching still need to know if
> > skipping fetching is permitted for this scan. However, this dependency
> > will be removed when that prefetching code is removed in favor of the
> > upcoming streaming read API.
>
> > ---
> > src/backend/access/heap/heapam.c | 10 +++
> > src/backend/access/heap/heapam_handler.c | 29 +++++++
> > src/backend/executor/nodeBitmapHeapscan.c | 100 ++++++----------------
> > src/include/access/heapam.h | 2 +
> > src/include/access/tableam.h | 17 ++--
> > src/include/nodes/execnodes.h | 6 --
> > 6 files changed, 74 insertions(+), 90 deletions(-)
> >
> > diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
> > index 707460a5364..7aae1ecf0a9 100644
> > --- a/src/backend/access/heap/heapam.c
> > +++ b/src/backend/access/heap/heapam.c
> > @@ -955,6 +955,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> > scan->rs_base.rs_flags = flags;
> > scan->rs_base.rs_parallel = parallel_scan;
> > scan->rs_strategy = NULL; /* set in initscan */
> > + scan->vmbuffer = InvalidBuffer;
> > + scan->empty_tuples = 0;
>
> These don't follow the existing naming pattern for HeapScanDescData. While I
> explicitly dislike the practice of adding prefixes to struct members, I don't
> think mixing conventions within a single struct improves things.
I've updated the names. What does rs even stand for?
> I also think it'd be good to note in comments that the vm buffer currently is
> only used for bitmap heap scans, otherwise one might think they'd also be used
> for normal scans, where we don't need them, because of the page level flag.
Done.
> Also, perhaps worth renaming "empty_tuples" to something indicating that it's
> the number of empty tuples to be returned later? num_empty_tuples_pending or
> such? Or the current "return_empty_tuples".
Done.
> > @@ -1043,6 +1045,10 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
> > if (BufferIsValid(scan->rs_cbuf))
> > ReleaseBuffer(scan->rs_cbuf);
> >
> > + if (BufferIsValid(scan->vmbuffer))
> > + ReleaseBuffer(scan->vmbuffer);
> > + scan->vmbuffer = InvalidBuffer;
>
> It does not matter one iota here, but personally I prefer moving the write
> inside the if, as dirtying the cacheline after we just figured out whe
I've now followed this convention throughout my patchset in the places
where I noticed it.
> > diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
> > index 9372b49bfaa..c0fb06c9688 100644
> > --- a/src/backend/executor/nodeBitmapHeapscan.c
> > +++ b/src/backend/executor/nodeBitmapHeapscan.c
> > @@ -108,6 +108,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
> > */
> > if (!node->initialized)
> > {
> > + bool can_skip_fetch;
> > /*
> > * We can potentially skip fetching heap pages if we do not need any
> > * columns of the table, either for checking non-indexable quals or
>
> Pretty sure pgindent will move this around.
This is gone now, but I have pgindented all the commits so it
shouldn't be a problem again.
> > +++ b/src/include/access/tableam.h
> > @@ -62,6 +62,7 @@ typedef enum ScanOptions
> >
> > /* unregister snapshot at scan end? */
> > SO_TEMP_SNAPSHOT = 1 << 9,
> > + SO_CAN_SKIP_FETCH = 1 << 10,
> > } ScanOptions;
>
> Would be nice to add a comment explaining what this flag means.
Done.
> > From 500c84019b982a1e6c8b8dd40240c8510d83c287 Mon Sep 17 00:00:00 2001
> > From: Melanie Plageman <melanieplageman@gmail.com>
> > Date: Tue, 13 Feb 2024 10:05:04 -0500
> > Subject: [PATCH v1 07/11] BitmapHeapScan scan desc counts lossy and exact
> > pages
> >
> > Future commits will remove the TBMIterateResult from BitmapHeapNext(),
> > pushing it into the table AM-specific code. So we will have to keep
> > track of the number of lossy and exact pages in the scan descriptor.
> > Doing this change to lossy/exact page counting in a separate commit just
> > simplifies the diff.
>
> > ---
> > src/backend/access/heap/heapam.c | 2 ++
> > src/backend/access/heap/heapam_handler.c | 9 +++++++++
> > src/backend/executor/nodeBitmapHeapscan.c | 18 +++++++++++++-----
> > src/include/access/relscan.h | 4 ++++
> > 4 files changed, 28 insertions(+), 5 deletions(-)
> >
> > diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
> > index 7aae1ecf0a9..88b4aad5820 100644
> > --- a/src/backend/access/heap/heapam.c
> > +++ b/src/backend/access/heap/heapam.c
> > @@ -957,6 +957,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> > scan->rs_strategy = NULL; /* set in initscan */
> > scan->vmbuffer = InvalidBuffer;
> > scan->empty_tuples = 0;
> > + scan->rs_base.lossy_pages = 0;
> > + scan->rs_base.exact_pages = 0;
> >
> > /*
> > * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
> > diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
> > index baba09c87c0..6e85ef7a946 100644
> > --- a/src/backend/access/heap/heapam_handler.c
> > +++ b/src/backend/access/heap/heapam_handler.c
> > @@ -2242,6 +2242,15 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
> > Assert(ntup <= MaxHeapTuplesPerPage);
> > hscan->rs_ntuples = ntup;
> >
> > + /* Only count exact and lossy pages with visible tuples */
> > + if (ntup > 0)
> > + {
> > + if (tbmres->ntuples >= 0)
> > + scan->exact_pages++;
> > + else
> > + scan->lossy_pages++;
> > + }
> > +
> > return ntup > 0;
> > }
> >
> > diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
> > index c0fb06c9688..19d115de06f 100644
> > --- a/src/backend/executor/nodeBitmapHeapscan.c
> > +++ b/src/backend/executor/nodeBitmapHeapscan.c
> > @@ -53,6 +53,8 @@
> > #include "utils/spccache.h"
> >
> > static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
> > +static inline void BitmapAccumCounters(BitmapHeapScanState *node,
> > + TableScanDesc scan);
> > static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
> > static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
> > BlockNumber blockno);
> > @@ -234,11 +236,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
> > continue;
> > }
> >
> > - if (tbmres->ntuples >= 0)
> > - node->exact_pages++;
> > - else
> > - node->lossy_pages++;
> > -
> > /* Adjust the prefetch target */
> > BitmapAdjustPrefetchTarget(node);
> > }
> > @@ -315,9 +312,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
> > /*
> > * if we get here it means we are at the end of the scan..
> > */
> > + BitmapAccumCounters(node, scan);
> > return ExecClearTuple(slot);
> > }
> >
> > +static inline void
> > +BitmapAccumCounters(BitmapHeapScanState *node,
> > + TableScanDesc scan)
> > +{
> > + node->exact_pages += scan->exact_pages;
> > + scan->exact_pages = 0;
> > + node->lossy_pages += scan->lossy_pages;
> > + scan->lossy_pages = 0;
> > +}
> > +
>
> I don't think this is quite right - you're calling BitmapAccumCounters() only
> when the scan doesn't return anything anymore, but there's no guarantee
> that'll ever be reached. E.g. a bitmap heap scan below a limit node. I think
> this needs to be in a) ExecEndBitmapHeapScan() b) ExecReScanBitmapHeapScan()
The scan descriptor isn't available in ExecEnd/ReScanBitmapHeapScan().
So, if we count in the scan descriptor we can't accumulate into the
BitmapHeapScanState there. The reason to count in the scan descriptor
is that it is in the table AM where we know if we have a lossy or
exact page -- and we only have the scan descriptor not the
BitmapHeapScanState in the table AM.
I added a call to BitmapAccumCounters before the tuple is returned for
correctness in this version (not ideal, I realize). See below for
thoughts about what we could do instead.
> > /*
> > * BitmapDoneInitializingSharedState - Shared state is initialized
> > *
> > diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
> > index 521043304ab..b74e08dd745 100644
> > --- a/src/include/access/relscan.h
> > +++ b/src/include/access/relscan.h
> > @@ -40,6 +40,10 @@ typedef struct TableScanDescData
> > ItemPointerData rs_mintid;
> > ItemPointerData rs_maxtid;
> >
> > + /* Only used for Bitmap table scans */
> > + long exact_pages;
> > + long lossy_pages;
> > +
> > /*
> > * Information about type and behaviour of the scan, a bitmask of members
> > * of the ScanOptions enum (see tableam.h).
>
> I wonder if this really is the best place for the data to be accumulated. This
> requires the accounting to be implemented in each AM, which doesn't obviously
> seem required. Why can't the accounting continue to live in
> nodeBitmapHeapscan.c, to be done after each table_scan_bitmap_next_block()
> call?
Yes, I would really prefer not to do it in the table AM. But, we only
count exact and lossy pages for which at least one or more tuples were
visible (change this and you'll see tests fail). So, we need to decide
if we are going to increment the counters somewhere where we have
access to that information. In the case of heap, that is really only
once I have the value of ntup in heapam_scan_bitmap_next_block(). To
get that information back out to BitmapHeapNext(), I considered adding
another parameter to heapam_scan_bitmap_next_block() -- maybe an enum
like this:
/*
* BitmapHeapScans's bitmaps can choose to store per page information in a
* lossy or exact way. Exact pages in the bitmap have the individual tuple
* offsets that need to be visited while lossy pages in the bitmap have only the
* block number of the page.
*/
typedef enum BitmapBlockResolution
{
BITMAP_BLOCK_NO_VISIBLE,
BITMAP_BLOCK_LOSSY,
BITMAP_BLOCK_EXACT,
} BitmapBlockResolution;
which we then use to increment the counter. But while I was writing
this code, I found myself narrating in the comment that the reason
this had to be set inside of the table AM is that only the table AM
knows if it wants to count the block as lossy, exact, or not count it.
So, that made me question if it really should be in the
BitmapHeapScanState.
I also explored passing the table scan descriptor to
show_tidbitmap_info() -- but that had its own problems.
> > From 555743e4bc885609d20768f7f2990c6ba69b13a9 Mon Sep 17 00:00:00 2001
> > From: Melanie Plageman <melanieplageman@gmail.com>
> > Date: Tue, 13 Feb 2024 10:57:07 -0500
> > Subject: [PATCH v1 09/11] Make table_scan_bitmap_next_block() async friendly
> >
> > table_scan_bitmap_next_block() previously returned false if we did not
> > wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
> > This could happen when there were no visible tuples on the page or, due
> > to concurrent activity on the table, the block returned by the iterator
> > is past the known end of the table.
>
> This sounds a bit like the block is actually past the end of the table,
> but in reality this happens if the block is past the end of the table as it
> was when the scan was started. Somehow that feels significant, but I don't
> really know why I think that.
I have tried to update the commit message to make it clearer. I was
actually wondering: now that we do table_beginscan_bm() in
BitmapHeapNext() instead of ExecInitBitmapHeapScan(), have we reduced
or eliminated the opportunity for this to be true? initscan() sets
rs_nblocks and that now happens much later.
> > diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
> > index 88b4aad5820..d8569373987 100644
> > --- a/src/backend/access/heap/heapam.c
> > +++ b/src/backend/access/heap/heapam.c
> > @@ -959,6 +959,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
> > scan->empty_tuples = 0;
> > scan->rs_base.lossy_pages = 0;
> > scan->rs_base.exact_pages = 0;
> > + scan->rs_base.shared_tbmiterator = NULL;
> > + scan->rs_base.tbmiterator = NULL;
> >
> > /*
> > * Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
> > @@ -1051,6 +1053,18 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
> > ReleaseBuffer(scan->vmbuffer);
> > scan->vmbuffer = InvalidBuffer;
> >
> > + if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
> > + {
> > + if (scan->rs_base.shared_tbmiterator)
> > + tbm_end_shared_iterate(scan->rs_base.shared_tbmiterator);
> > +
> > + if (scan->rs_base.tbmiterator)
> > + tbm_end_iterate(scan->rs_base.tbmiterator);
> > + }
> > +
> > + scan->rs_base.shared_tbmiterator = NULL;
> > + scan->rs_base.tbmiterator = NULL;
> > +
> > /*
> > * reinitialize scan descriptor
> > */
>
> If every AM would need to implement this, perhaps this shouldn't be done here,
> but in generic code?
Fixed.
> > --- a/src/backend/access/heap/heapam_handler.c
> > +++ b/src/backend/access/heap/heapam_handler.c
> > @@ -2114,17 +2114,49 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
> >
> > static bool
> > heapam_scan_bitmap_next_block(TableScanDesc scan,
> > - TBMIterateResult *tbmres)
> > + bool *recheck, BlockNumber *blockno)
> > {
> > HeapScanDesc hscan = (HeapScanDesc) scan;
> > - BlockNumber block = tbmres->blockno;
> > + BlockNumber block;
> > Buffer buffer;
> > Snapshot snapshot;
> > int ntup;
> > + TBMIterateResult *tbmres;
> >
> > hscan->rs_cindex = 0;
> > hscan->rs_ntuples = 0;
> >
> > + *blockno = InvalidBlockNumber;
> > + *recheck = true;
> > +
> > + do
> > + {
> > + if (scan->shared_tbmiterator)
> > + tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
> > + else
> > + tbmres = tbm_iterate(scan->tbmiterator);
> > +
> > + if (tbmres == NULL)
> > + {
> > + /* no more entries in the bitmap */
> > + Assert(hscan->empty_tuples == 0);
> > + return false;
> > + }
> > +
> > + /*
> > + * Ignore any claimed entries past what we think is the end of the
> > + * relation. It may have been extended after the start of our scan (we
> > + * only hold an AccessShareLock, and it could be inserts from this
> > + * backend). We don't take this optimization in SERIALIZABLE
> > + * isolation though, as we need to examine all invisible tuples
> > + * reachable by the index.
> > + */
> > + } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
>
> Hm. Isn't it a problem that we have no CHECK_FOR_INTERRUPTS() in this loop?
Yes. fixed.
> > @@ -2251,7 +2274,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
> > scan->lossy_pages++;
> > }
> >
> > - return ntup > 0;
> > + /*
> > + * Return true to indicate that a valid block was found and the bitmap is
> > + * not exhausted. If there are no visible tuples on this page,
> > + * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
> > + * return false returning control to this function to advance to the next
> > + * block in the bitmap.
> > + */
> > + return true;
> > }
>
> Why can't we fetch the next block immediately?
We don't know that we want another block until we've gone through this
page and seen there were no visible tuples, so we'd somehow have to
jump back up to the top of the function to get the next block -- which
is basically what is happening in my revised control flow. We call
heapam_scan_bitmap_next_tuple() and rs_ntuples is 0, so we end up
calling heapam_scan_bitmap_next_block() right away.
> > @@ -201,46 +197,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
> > can_skip_fetch);
> > }
> >
> > - node->tbmiterator = tbmiterator;
> > - node->shared_tbmiterator = shared_tbmiterator;
> > + scan->tbmiterator = tbmiterator;
> > + scan->shared_tbmiterator = shared_tbmiterator;
>
> It seems a bit odd that this code modifies the scan descriptor, instead of
> passing the iterator, or perhaps better the bitmap itself, to
> table_beginscan_bm()?
On rescan we actually will have initialized = false and make new
iterators but have the old scan descriptor. So, we need to be able to
set the iterator in the scan to the new iterator.
> > diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
> > index b74e08dd745..bf7ee044268 100644
> > --- a/src/include/access/relscan.h
> > +++ b/src/include/access/relscan.h
> > @@ -16,6 +16,7 @@
> >
> > #include "access/htup_details.h"
> > #include "access/itup.h"
> > +#include "nodes/tidbitmap.h"
>
> I'd like to avoid exposing this to everything including relscan.h. I think we
> could just forward declare the structs and use them here to avoid that?
Done
> > From aac60985d6bc70bfedf77a77ee3c512da87bfcb1 Mon Sep 17 00:00:00 2001
> > From: Melanie Plageman <melanieplageman@gmail.com>
> > Date: Tue, 13 Feb 2024 14:27:57 -0500
> > Subject: [PATCH v1 11/11] BitmapHeapScan uses streaming read API
> >
> > Remove all of the code to do prefetching from BitmapHeapScan code and
> > rely on the streaming read API prefetching. Heap table AM implements a
> > streaming read callback which uses the iterator to get the next valid
> > block that needs to be fetched for the streaming read API.
> > ---
> > src/backend/access/gin/ginget.c | 15 +-
> > src/backend/access/gin/ginscan.c | 7 +
> > src/backend/access/heap/heapam.c | 71 +++++
> > src/backend/access/heap/heapam_handler.c | 78 +++--
> > src/backend/executor/nodeBitmapHeapscan.c | 328 +---------------------
> > src/backend/nodes/tidbitmap.c | 80 +++---
> > src/include/access/heapam.h | 2 +
> > src/include/access/tableam.h | 14 +-
> > src/include/nodes/execnodes.h | 19 --
> > src/include/nodes/tidbitmap.h | 8 +-
> > 10 files changed, 178 insertions(+), 444 deletions(-)
> >
> > diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
> > index 0b4f2ebadb6..3ce28078a6f 100644
> > --- a/src/backend/access/gin/ginget.c
> > +++ b/src/backend/access/gin/ginget.c
> > @@ -373,7 +373,10 @@ restartScanEntry:
> > if (entry->matchBitmap)
> > {
> > if (entry->matchIterator)
> > + {
> > tbm_end_iterate(entry->matchIterator);
> > + pfree(entry->matchResult);
> > + }
> > entry->matchIterator = NULL;
> > tbm_free(entry->matchBitmap);
> > entry->matchBitmap = NULL;
> > @@ -386,6 +389,7 @@ restartScanEntry:
> > if (entry->matchBitmap && !tbm_is_empty(entry->matchBitmap))
> > {
> > entry->matchIterator = tbm_begin_iterate(entry->matchBitmap);
> > + entry->matchResult = palloc0(TBM_ITERATE_RESULT_SIZE);
>
> Do we actually have to use palloc0? TBM_ITERATE_RESULT_SIZE ain't small, so
> zeroing all of it isn't free.
Tests actually did fail when I didn't use palloc0.
This code is different now though. There are a few new patches in v2
that 1) make the offsets array in the TBMIterateResult fixed size and
then this makes it possible to 2) make matchResult an inline member of
the GinScanEntry. I have a TODO in the code asking if setting blockno
in the TBMIterateResult to InvalidBlockNumber is sufficient
"resetting".
> > +static BlockNumber bitmapheap_pgsr_next_single(PgStreamingRead *pgsr, void *pgsr_private,
> > + void *per_buffer_data);
>
> Is it correct to have _single in the name here? Aren't we also using for
> parallel scans?
Right. I had a separate parallel version and then deleted it. This is now fixed.
> > +static BlockNumber
> > +bitmapheap_pgsr_next_single(PgStreamingRead *pgsr, void *pgsr_private,
> > + void *per_buffer_data)
> > +{
> > + TBMIterateResult *tbmres = per_buffer_data;
> > + HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
> > +
> > + for (;;)
> > + {
> > + if (hdesc->rs_base.shared_tbmiterator)
> > + tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
> > + else
> > + tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
> > +
> > + /* no more entries in the bitmap */
> > + if (!BlockNumberIsValid(tbmres->blockno))
> > + return InvalidBlockNumber;
> > +
> > + /*
> > + * Ignore any claimed entries past what we think is the end of the
> > + * relation. It may have been extended after the start of our scan (we
> > + * only hold an AccessShareLock, and it could be inserts from this
> > + * backend). We don't take this optimization in SERIALIZABLE
> > + * isolation though, as we need to examine all invisible tuples
> > + * reachable by the index.
> > + */
> > + if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
> > + continue;
> > +
> > +
> > + if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
> > + !tbmres->recheck &&
> > + VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->vmbuffer))
> > + {
> > + hdesc->empty_tuples += tbmres->ntuples;
> > + continue;
> > + }
> > +
> > + return tbmres->blockno;
> > + }
> > +
> > + /* not reachable */
> > + Assert(false);
> > +}
>
> Need to check for interrupts somewhere here.
Done.
> > @@ -124,15 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
>
> There's still a comment in BitmapHeapNext talking about prefetching with two
> iterators etc. That seems outdated now.
Fixed.
> > /*
> > * tbm_iterate - scan through next page of a TIDBitmap
> > *
> > - * Returns a TBMIterateResult representing one page, or NULL if there are
> > - * no more pages to scan. Pages are guaranteed to be delivered in numerical
> > - * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
> > - * remember the exact tuples to look at on this page --- the caller must
> > - * examine all tuples on the page and check if they meet the intended
> > - * condition. If result->recheck is true, only the indicated tuples need
> > - * be examined, but the condition must be rechecked anyway. (For ease of
> > - * testing, recheck is always set true when ntuples < 0.)
> > + * Caller must pass in a TBMIterateResult to be filled.
> > + *
> > + * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
> > + * set to InvalidBlockNumber when there are no more pages to scan. If
> > + * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
> > + * exact tuples to look at on this page --- the caller must examine all tuples
> > + * on the page and check if they meet the intended condition. If
> > + * tbmres->recheck is true, only the indicated tuples need be examined, but the
> > + * condition must be rechecked anyway. (For ease of testing, recheck is always
> > + * set true when ntuples < 0.)
> > */
> > -TBMIterateResult *
> > -tbm_iterate(TBMIterator *iterator)
> > +void
> > +tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
>
> Hm - it seems a tad odd that we later have to find out if the scan is done
> iterating by checking if blockno is valid, when tbm_iterate already knew. But
> I guess the code would be a bit uglier if we needed the result of
> tbm_[shared_]iterate(), due to the two functions.
Yes.
> Right now ExecEndBitmapHeapScan() frees the tbm before it does table_endscan()
> - which seems problematic, as heap_endscan() will do stuff like
> tbm_end_iterate(), which imo shouldn't be called after the tbm has been freed,
> even if that works today.
I've flipped the order -- I end the scan then free the bitmap.
> It seems a bit confusing that your changes seem to treat
> BitmapHeapScanState->initialized as separate from ->scan, even though afaict
> scan should be NULL iff initialized is false and vice versa.
I thought so too, but it seems on rescan that the node->initialized is
set to false but the scan is reused. So, we want to only make a new
scan descriptor if it is truly the beginning of a new scan.
- Melanie
Attachments:
[text/x-patch] v2-0003-BitmapHeapScan-begin-scan-after-bitmap-setup.patch (6.4K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/2-v2-0003-BitmapHeapScan-begin-scan-after-bitmap-setup.patch)
download | inline diff:
From 6cfa8fee46a83936789062eedc37d0c06b59dc46 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v2 03/13] BitmapHeapScan begin scan after bitmap setup
There is no reason for table_beginscan_bm() to begin the actual scan of
the underlying table in ExecInitBitmapHeapScan(). We can begin the
underlying table scan after the index scan has been completed and the
bitmap built.
The one use of the scan descriptor during initialization was
ExecBitmapHeapInitializeWorker(), which set the scan descriptor snapshot
with one from an array in the parallel state. This overwrote the
snapshot set in table_beginscan_bm().
By saving that worker snapshot as a member in the BitmapHeapScanState
during initialization, it can be restored in table_beginscan_bm() after
returning from the table AM specific begin scan function.
---
src/backend/access/table/tableam.c | 11 ------
src/backend/executor/nodeBitmapHeapscan.c | 43 +++++++++++++++++------
src/include/access/tableam.h | 10 ++----
src/include/nodes/execnodes.h | 2 ++
4 files changed, 38 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 6ed8cca05a1..e78d793f69c 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,17 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-void
-table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot)
-{
- Assert(IsMVCCSnapshot(snapshot));
-
- RegisterSnapshot(snapshot);
- scan->rs_snapshot = snapshot;
- scan->rs_flags |= SO_TEMP_SNAPSHOT;
-}
-
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 76382c91fd7..be08bd785ae 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -191,6 +191,30 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ if (!scan)
+ {
+ Snapshot snapshot = node->ss.ps.state->es_snapshot;
+ uint32 extra_flags = 0;
+
+ /*
+ * Parallel workers must use the snapshot initialized by the
+ * parallel leader.
+ */
+ if (node->worker_snapshot)
+ {
+ snapshot = node->worker_snapshot;
+ extra_flags |= SO_TEMP_SNAPSHOT;
+ }
+
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ snapshot,
+ 0,
+ NULL,
+ extra_flags);
+ }
+
node->initialized = true;
}
@@ -614,7 +638,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -691,7 +716,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -740,6 +767,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->can_skip_fetch = false;
+ scanstate->worker_snapshot = NULL;
/*
* Miscellaneous initialization
@@ -788,11 +816,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
@@ -931,13 +954,13 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelBitmapHeapState *pstate;
- Snapshot snapshot;
Assert(node->ss.ps.state->es_query_dsa != NULL);
pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->pstate = pstate;
- snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
- table_scan_update_snapshot(node->ss.ss_currentScanDesc, snapshot);
+ node->worker_snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
+ Assert(IsMVCCSnapshot(node->worker_snapshot));
+ RegisterSnapshot(node->worker_snapshot);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 4d495216f07..8ef6b5ca25b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -939,9 +939,10 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key,
+ uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
@@ -1033,11 +1034,6 @@ table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
allow_pagemode);
}
-/*
- * Update snapshot used by the scan.
- */
-extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
-
/*
* Return next tuple from `scan`, store in slot.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 444a5f0fd57..00c75fb10e2 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1726,6 +1726,7 @@ typedef struct ParallelBitmapHeapState
* shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * worker_snapshot snapshot for parallel worker
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1750,6 +1751,7 @@ typedef struct BitmapHeapScanState
TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ Snapshot worker_snapshot;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-patch] v2-0001-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (3.2K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/3-v2-0001-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From 575fb1f93128ebfd8125c769de628f91e0d5c592 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v2 01/13] Remove table_scan_bitmap_next_tuple parameter tbmres
Future commits will remove the input TBMIterateResult from
table_scan_bitmap_next_block() as the streaming read API will be
responsible for iterating through the blocks in the bitmap and not
BitmapHeapNext(). Given that this parameter will not be set from
BitmapHeapNext(), it no longer makes sense to use it as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 7 -------
3 files changed, 1 insertion(+), 9 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d15a02b2be7..716d477e271 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2228,7 +2228,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c1e81ebed63..d670939246b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -304,7 +304,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5f8474871d2..4d495216f07 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -810,15 +810,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -1980,7 +1975,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -1992,7 +1986,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.37.2
[text/x-patch] v2-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (1.8K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/4-v2-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From 2a11dc5ef62408bb455277eb73c83abc8f9d2bf3 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v2 04/13] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index be08bd785ae..c6dfdf8cae9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -532,7 +532,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* but is true in many cases.
*/
skip_fetch = (node->can_skip_fetch &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -583,7 +583,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (node->can_skip_fetch &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.37.2
[text/x-patch] v2-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/5-v2-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 5f915bc84eae56e52b5a61e9b7e691834fdb9680 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v2 02/13] BitmapHeapScan set can_skip_fetch later
There is no reason for BitmapHeapScan to calculate can_skip_fetch in
ExecInitBitmapHeapScan(). Moving it into BitmapHeapNext() is a
preliminary step toward moving can_skip_fetch into table AM specific
code, as we would need to set it after the scan has begun.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d670939246b..76382c91fd7 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,6 +108,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -729,16 +739,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.37.2
[text/x-patch] v2-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/6-v2-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From d578ca2c47857794622c49319087f53918fe4c6c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v2 05/13] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c6dfdf8cae9..07a218ec03e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -55,7 +55,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -239,7 +239,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
/*
* We can skip fetching the heap page if we don't need any fields
@@ -392,7 +392,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -411,7 +411,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.37.2
[text/x-patch] v2-0007-BitmapHeapScan-scan-desc-counts-lossy-and-exact-p.patch (4.0K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/7-v2-0007-BitmapHeapScan-scan-desc-counts-lossy-and-exact-p.patch)
download | inline diff:
From 224a7e4e8eb7106c1e7159df8ca3d7ede6732be8 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:05:04 -0500
Subject: [PATCH v2 07/13] BitmapHeapScan scan desc counts lossy and exact
pages
Future commits will remove the TBMIterateResult from BitmapHeapNext(),
pushing it into the table AM-specific code. So we will have to keep
track of the number of lossy and exact pages in the scan descriptor.
Doing this change to lossy/exact page counting in a separate commit just
simplifies the diff.
---
src/backend/access/heap/heapam_handler.c | 9 +++++++++
src/backend/executor/nodeBitmapHeapscan.c | 19 ++++++++++++++-----
src/include/access/relscan.h | 4 ++++
src/include/access/tableam.h | 6 +++++-
4 files changed, 32 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d775756fa53..3af9466b9ca 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2242,6 +2242,15 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ /* Only count exact and lossy pages with visible tuples */
+ if (ntup > 0)
+ {
+ if (tbmres->ntuples >= 0)
+ scan->exact_pages++;
+ else
+ scan->lossy_pages++;
+ }
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c884771e826..3b89e7e6c63 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -53,6 +53,8 @@
#include "utils/spccache.h"
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
+static inline void BitmapAccumCounters(BitmapHeapScanState *node,
+ TableScanDesc scan);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
BlockNumber blockno);
@@ -246,11 +248,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
continue;
}
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
- node->lossy_pages++;
-
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
@@ -321,15 +318,27 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* OK to return this tuple */
+ BitmapAccumCounters(node, scan);
return slot;
}
/*
* if we get here it means we are at the end of the scan..
*/
+ BitmapAccumCounters(node, scan);
return ExecClearTuple(slot);
}
+static inline void
+BitmapAccumCounters(BitmapHeapScanState *node,
+ TableScanDesc scan)
+{
+ node->exact_pages += scan->exact_pages;
+ scan->exact_pages = 0;
+ node->lossy_pages += scan->lossy_pages;
+ scan->lossy_pages = 0;
+}
+
/*
* BitmapDoneInitializingSharedState - Shared state is initialized
*
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..b74e08dd745 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -40,6 +40,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ long exact_pages;
+ long lossy_pages;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b5d65a9528c..a3e30c4eda7 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -942,9 +942,13 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key,
uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->lossy_pages = 0;
+ result->exact_pages = 0;
+ return result;
}
/*
--
2.37.2
[text/x-patch] v2-0006-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (14.5K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/8-v2-0006-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From 9104d9c36462119a1875d7620b21d37b994216f9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v2 06/13] Push BitmapHeapScan skip fetch optimization into
table AM
This resolves the FIXME in BitmapHeapNext() which said that the
optmization to skip fetching blocks of the underlying table when none of
the column data was needed should be pushed into the table AM specific
code.
heapam_scan_bitmap_next_block() now does the visibility check and
accounting of empty tuples to be returned; while
heapam_scan_bitmap_next_tuple() prepares the slot to return empty
tuples.
The table AM agnostic functions for prefetching still need to know if
skipping fetching is permitted for this scan. However, this dependency
will be removed when that prefetching code is removed in favor of the
upcoming streaming read API.
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 ++++++
src/backend/executor/nodeBitmapHeapscan.c | 115 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 18 ++--
src/include/nodes/execnodes.h | 6 --
6 files changed, 95 insertions(+), 97 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 707460a5364..b93f243c282 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -955,6 +955,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1043,6 +1045,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1062,6 +1070,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 716d477e271..d775756fa53 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, and the bitmap entries don't need rechecking, and all tuples on
+ * the page are visible to our transaction.
+ */
+ if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2235,6 +2254,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 07a218ec03e..c884771e826 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,16 +108,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -207,6 +197,17 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags |= SO_TEMP_SNAPSHOT;
}
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
+ extra_flags |= SO_CAN_SKIP_FETCH;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
snapshot,
@@ -220,8 +221,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -241,32 +240,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!table_scan_bitmap_next_block(scan, tbmres))
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -320,46 +294,30 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -531,7 +489,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
!tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -582,7 +541,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
!tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -652,8 +611,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -663,7 +620,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -708,8 +664,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -753,8 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -766,7 +718,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..3dfb19ec7d5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8ef6b5ca25b..b5d65a9528c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed.
+ */
+ SO_CAN_SKIP_FETCH = 1 << 10,
} ScanOptions;
/*
@@ -780,10 +787,8 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time). All work
+ * referencing `tbmres` must be done here.
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -795,11 +800,6 @@ typedef struct TableAmRoutine
* performs prefetching directly using that interface. This probably
* needs to be rectified at a later point.
*
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 00c75fb10e2..9392923eb32 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1711,9 +1711,6 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
* pvmbuffer ditto, for prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1736,9 +1733,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.37.2
[text/x-patch] v2-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/9-v2-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From 8632e2b57bd4b532e5cbe94df89f2c1123fed62c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v2 08/13] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3b89e7e6c63..c62f978f5d7 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -76,8 +76,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -90,10 +88,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -110,6 +104,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -118,7 +115,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -171,8 +168,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -218,6 +214,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -231,9 +229,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.37.2
[text/x-patch] v2-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (18.1K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/10-v2-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 59fdb0423ddc9032380247f987b682944a52d476 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:57:07 -0500
Subject: [PATCH v2 09/13] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the known end of the table when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API. The table AMs will implement a streaming read API callback
that returns the next block that needs to be fetched. In heap AM's case,
the callback will use the iterator to find the next block to be fetched.
Since choosing the next block will no longer the responsibility of
BitmapHeapNext(), the streaming read control flow requires these changes
to table_scan_bitmap_next_block().
---
src/backend/access/heap/heapam_handler.c | 58 +++++++--
src/backend/executor/nodeBitmapHeapscan.c | 148 ++++++++--------------
src/include/access/relscan.h | 5 +
src/include/access/tableam.h | 47 +++++--
src/include/nodes/execnodes.h | 11 +-
5 files changed, 145 insertions(+), 124 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3af9466b9ca..c8da3def645 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,17 +2114,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ bool *recheck, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, and the bitmap entries don't need rechecking, and all tuples on
@@ -2143,16 +2177,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2251,7 +2276,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
scan->lossy_pages++;
}
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c62f978f5d7..ae837785116 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -76,7 +76,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -88,7 +87,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -116,7 +114,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -169,7 +166,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -214,46 +210,24 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ /* Get the first block. if none, end of scan */
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ return ExecClearTuple(slot);
+
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ BitmapAdjustPrefetchTarget(node);
}
for (;;)
{
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- if (!table_scan_bitmap_next_block(scan, tbmres))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -275,49 +249,41 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- BitmapPrefetch(node, scan);
-
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
- }
+ /*
+ * We prefetch before fetching the current pages. We expect that a
+ * future streaming read API will do this, so do it now for
+ * consistency.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
{
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
}
+
+ /* OK to return this tuple */
+ BitmapAccumCounters(node, scan);
+ return slot;
}
- /* OK to return this tuple */
- BitmapAccumCounters(node, scan);
- return slot;
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ break;
+
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -608,12 +574,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -621,13 +583,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -658,28 +619,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -712,8 +669,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -722,10 +677,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index b74e08dd745..5dea9c7a03d 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -41,6 +44,8 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
long exact_pages;
long lossy_pages;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a3e30c4eda7..ef1fcc02b1a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -804,7 +805,7 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ bool *recheck, BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -948,6 +949,8 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->lossy_pages = 0;
result->exact_pages = 0;
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
return result;
}
@@ -1008,6 +1011,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1018,6 +1036,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1941,17 +1974,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ bool *recheck, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1961,8 +1993,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 9392923eb32..a59df51dd69 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1709,9 +1709,7 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1720,10 +1718,10 @@ typedef struct ParallelBitmapHeapState
* prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
+ * recheck do current page's tuples need recheck
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1731,8 +1729,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1742,10 +1738,11 @@ typedef struct BitmapHeapScanState
int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
+ bool recheck;
+ BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-patch] v2-0011-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.4K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/11-v2-0011-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From b0417f661ffab18058a392327370eb8690b49c38 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v2 11/13] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 33 +++++-----
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 107 insertions(+), 87 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..831941271c4 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno to
+ * InvalidBlockNumber. It seems like this should be okay because that is
+ * usually what we check before using the matchResult members. But it might
+ * be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c8da3def645..ba6793a749c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2121,7 +2121,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2134,11 +2134,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2153,11 +2153,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2165,19 +2165,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* the page are visible to our transaction.
*/
if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2206,7 +2206,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2215,9 +2215,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2270,7 +2270,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/* Only count exact and lossy pages with visible tuples */
if (ntup > 0)
{
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
scan->exact_pages++;
else
scan->lossy_pages++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index ae837785116..284641fa8ea 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -340,9 +340,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
+ tbm_iterate(prefetch_iterator, &tbmpre);
- if (tbmpre == NULL || tbmpre->blockno != blockno)
+ if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
@@ -360,6 +361,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
}
else
{
+ TBMIterateResult tbmpre;
+
/* Release the mutex before iterating */
SpinLockRelease(&pstate->mutex);
@@ -372,7 +375,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
}
}
#endif /* USE_PREFETCH */
@@ -439,10 +442,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -464,13 +469,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
*/
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -485,7 +490,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -504,8 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -515,13 +520,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index f711c056143..b4dcb1cbb88 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -171,7 +171,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -212,7 +211,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
};
/* Local function prototypes */
@@ -943,20 +941,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -984,6 +983,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -994,11 +994,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1014,16 +1014,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1033,10 +1034,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1087,13 +1087,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1103,21 +1103,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 51d0c74a6b0..e423d92b41c 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -352,7 +352,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.37.2
[text/x-patch] v2-0012-Streaming-Read-API.patch (56.0K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/12-v2-0012-Streaming-Read-API.patch)
download | inline diff:
From d84a520da846da83717b748a2bd30f4185d36ebe Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 22 Jul 2023 17:31:54 +1200
Subject: [PATCH v2 12/13] Streaming Read API
---
contrib/pg_prewarm/pg_prewarm.c | 40 +-
src/backend/access/transam/xlogutils.c | 2 +-
src/backend/postmaster/bgwriter.c | 8 +-
src/backend/postmaster/checkpointer.c | 15 +-
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 435 ++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 560 +++++++++++++++--------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/backend/storage/smgr/smgr.c | 49 +-
src/include/storage/bufmgr.h | 22 +
src/include/storage/smgr.h | 4 +-
src/include/storage/streaming_read.h | 45 ++
src/include/utils/rel.h | 6 -
src/tools/pgindent/typedefs.list | 2 +
17 files changed, 986 insertions(+), 238 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index 8541e4d6e46..9617bf130bd 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -20,6 +20,7 @@
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
+#include "storage/streaming_read.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
@@ -38,6 +39,25 @@ typedef enum
static PGIOAlignedBlock blockbuffer;
+struct pg_prewarm_streaming_read_private
+{
+ BlockNumber blocknum;
+ int64 last_block;
+};
+
+static BlockNumber
+pg_prewarm_streaming_read_next(PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_data)
+{
+ struct pg_prewarm_streaming_read_private *p = pgsr_private;
+
+ if (p->blocknum <= p->last_block)
+ return p->blocknum++;
+
+ return InvalidBlockNumber;
+}
+
/*
* pg_prewarm(regclass, mode text, fork text,
* first_block int8, last_block int8)
@@ -183,18 +203,36 @@ pg_prewarm(PG_FUNCTION_ARGS)
}
else if (ptype == PREWARM_BUFFER)
{
+ struct pg_prewarm_streaming_read_private p;
+ PgStreamingRead *pgsr;
+
/*
* In buffer mode, we actually pull the data into shared_buffers.
*/
+
+ /* Set up the private state for our streaming buffer read callback. */
+ p.blocknum = first_block;
+ p.last_block = last_block;
+
+ pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ &p,
+ 0,
+ NULL,
+ BMR_REL(rel),
+ forkNumber,
+ pg_prewarm_streaming_read_next);
+
for (block = first_block; block <= last_block; ++block)
{
Buffer buf;
CHECK_FOR_INTERRUPTS();
- buf = ReadBufferExtended(rel, forkNumber, block, RBM_NORMAL, NULL);
+ buf = pg_streaming_read_buffer_get_next(pgsr, NULL);
ReleaseBuffer(buf);
++blocks_done;
}
+ Assert(pg_streaming_read_buffer_get_next(pgsr, NULL) == InvalidBuffer);
+ pg_streaming_read_free(pgsr);
}
/* Close relation, release lock. */
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index aa8667abd10..8775b5789be 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -657,7 +657,7 @@ XLogDropDatabase(Oid dbid)
* This is unnecessarily heavy-handed, as it will close SMgrRelation
* objects for other databases as well. DROP DATABASE occurs seldom enough
* that it's not worth introducing a variant of smgrclose for just this
- * purpose. XXX: Or should we rather leave the smgr entries dangling?
+ * purpose.
*/
smgrcloseall();
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d7d6cc0cd7b..13e5376619e 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -246,10 +246,12 @@ BackgroundWriterMain(void)
if (FirstCallSinceLastCheckpoint())
{
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the bgwriter does
+ * not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
}
/*
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 5e949fc885b..5d843b61426 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -469,10 +469,12 @@ CheckpointerMain(void)
ckpt_performed = CreateRestartPoint(flags);
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the checkpointer
+ * does not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
/*
* Indicate checkpoint completion to any waiting backends.
@@ -958,11 +960,8 @@ RequestCheckpoint(int flags)
*/
CreateCheckPoint(flags | CHECKPOINT_IMMEDIATE);
- /*
- * After any checkpoint, close all smgr files. This is so we won't
- * hang onto smgr references to deleted files indefinitely.
- */
- smgrcloseall();
+ /* Free all smgr objects, as CheckpointerMain() normally would. */
+ smgrdestroyall();
return;
}
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..bcab44c802f
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..39aef2a84a2
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 00000000000..19605090fea
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,435 @@
+#include "postgres.h"
+
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ *
+ * For hits, need_to_complete is false and there is just one block per
+ * range, already pinned and ready for use.
+ *
+ * For misses, need_to_complete is true and buffers[] holds a range of
+ * blocks that are contiguous in storage (though the buffers may not be
+ * contiguous in memory), so we can complete them with a single call to
+ * CompleteReadBuffers().
+ */
+typedef struct PgStreamingReadRange
+{
+ bool advice_issued;
+ bool need_complete;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index[MAX_BUFFERS_PER_TRANSFER];
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+} PgStreamingReadRange;
+
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int ios_in_progress_trigger;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int pinned_buffers_trigger;
+ int next_tail_buffer;
+ bool finished;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ bool advice_enabled;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+ int per_buffer_data_next;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+static PgStreamingRead *
+pg_streaming_read_buffer_alloc_internal(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = maintenance_io_concurrency;
+ else
+ max_ios = effective_io_concurrency;
+
+ /*
+ * The desired level of I/O concurrency controls how far ahead we are
+ * willing to look ahead. We also clamp it to at least
+ * MAX_BUFFER_PER_TRANFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /*
+ * The *_io_concurrency GUCs, we might have 0. We want to allow at least
+ * one, to keep our gating logic simple.
+ */
+ max_ios = Max(max_ios, 1);
+
+ /*
+ * Don't allow this backend to pin too many buffers. For now we'll apply
+ * the limit for the shared buffer pool and the local buffer pool, without
+ * worrying which it is.
+ */
+ LimitAdditionalPins(&max_pinned_buffers);
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * We want to avoid creating ranges that are smaller than they could be
+ * just because we hit max_pinned_buffers. We only look ahead when the
+ * number of pinned buffers falls below this trigger number, or put
+ * another way, we stop looking ahead when we wouldn't be able to build a
+ * "full sized" range.
+ */
+ pgsr->pinned_buffers_trigger =
+ Max(1, (int) max_pinned_buffers - MAX_BUFFERS_PER_TRANSFER);
+
+ /* Space the callback to store extra data along with each block. */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * max_pinned_buffers);
+
+ return pgsr;
+}
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *result;
+
+ result = pg_streaming_read_buffer_alloc_internal(flags,
+ pgsr_private,
+ per_buffer_data_size,
+ strategy);
+ result->callback = next_block_cb;
+ result->bmr = bmr;
+ result->forknum = forknum;
+
+ return result;
+}
+
+/*
+ * Start building a new range. This is called after the previous one
+ * reached maximum size, or the callback's next block can't be merged with it.
+ *
+ * Since the previous head range has now reached its full potential size, this
+ * is also a good time to issue 'prefetch' advice, because we know that'll
+ * soon be reading. In future, we could start an actual I/O here.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_new_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If a call to CompleteReadBuffers() will be needed, and we can issue
+ * advice to the kernel to get the read started. We suppress it if the
+ * access pattern appears to be completely sequential, though, because on
+ * some systems that interfers with the kernel's own sequential read ahead
+ * heurstics and hurts performance.
+ */
+ if (pgsr->advice_enabled)
+ {
+ BlockNumber blocknum = head_range->blocknum;
+ int nblocks = head_range->nblocks;
+
+ if (head_range->need_complete && blocknum != pgsr->seq_blocknum)
+ {
+ SMgrRelation smgr =
+ pgsr->bmr.smgr ? pgsr->bmr.smgr :
+ RelationGetSmgr(pgsr->bmr.rel);
+
+ Assert(!head_range->advice_issued);
+
+ smgrprefetch(smgr, pgsr->forknum, blocknum, nblocks);
+
+ /*
+ * Count this as an I/O that is concurrently in progress, though
+ * we don't really know if the kernel generates a physical I/O.
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ }
+
+ /* Remember the block after this range, for sequence detection. */
+ pgsr->seq_blocknum = blocknum + nblocks;
+ }
+
+ /* Create a new head range. There must be space. */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ head_range = &pgsr->ranges[pgsr->head];
+ head_range->nblocks = 0;
+
+ return head_range;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ /*
+ * If we're finished or can't start more I/O, then don't look ahead.
+ */
+ if (pgsr->finished || pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * We'll also wait until the number of pinned buffers falls below our
+ * trigger level, so that we have the chance to create a full range.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger)
+ return;
+
+ do
+ {
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ Buffer buffer;
+ bool found;
+ bool need_complete;
+ PgStreamingReadRange *head_range;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks == lengthof(head_range->buffers))
+ {
+ Assert(head_range->need_complete);
+ head_range = pg_streaming_read_new_range(pgsr);
+
+ /*
+ * Give up now if I/O is saturated, or we wouldn't be able form
+ * another full range after this due to the pin limit.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger ||
+ pgsr->ios_in_progress == pgsr->max_ios)
+ break;
+ }
+
+ per_buffer_data = (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * pgsr->per_buffer_data_next;
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ pgsr->finished = true;
+ break;
+ }
+ bmr = pgsr->bmr;
+ forknum = pgsr->forknum;
+
+ Assert(pgsr->pinned_buffers < pgsr->max_pinned_buffers);
+
+ buffer = PrepareReadBuffer(bmr,
+ forknum,
+ blocknum,
+ pgsr->strategy,
+ &found);
+ pgsr->pinned_buffers++;
+
+ need_complete = !found;
+
+ /* Is there a head range that we can't extend? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks > 0 &&
+ (!need_complete ||
+ !head_range->need_complete ||
+ head_range->blocknum + head_range->nblocks != blocknum))
+ {
+ /* Yes, time to start building a new one. */
+ head_range = pg_streaming_read_new_range(pgsr);
+ Assert(head_range->nblocks == 0);
+ }
+
+ if (head_range->nblocks == 0)
+ {
+ /* Initialize a new range beginning at this block. */
+ head_range->blocknum = blocknum;
+ head_range->need_complete = need_complete;
+ head_range->advice_issued = false;
+ }
+ else
+ {
+ /* We can extend an existing range by one block. */
+ Assert(head_range->blocknum + head_range->nblocks == blocknum);
+ Assert(head_range->need_complete);
+ }
+
+ head_range->per_buffer_data_index[head_range->nblocks] = pgsr->per_buffer_data_next++;
+ head_range->buffers[head_range->nblocks] = buffer;
+ head_range->nblocks++;
+
+ if (pgsr->per_buffer_data_next == pgsr->max_pinned_buffers)
+ pgsr->per_buffer_data_next = 0;
+
+ } while (pgsr->pinned_buffers < pgsr->max_pinned_buffers &&
+ pgsr->ios_in_progress < pgsr->max_ios);
+
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ pg_streaming_read_new_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* See if we have one buffer to return. */
+ while (pgsr->tail != pgsr->head)
+ {
+ PgStreamingReadRange *tail_range;
+
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to perform an I/O before returning the buffers from this
+ * range?
+ */
+ if (tail_range->need_complete)
+ {
+ CompleteReadBuffers(pgsr->bmr,
+ tail_range->buffers,
+ pgsr->forknum,
+ tail_range->blocknum,
+ tail_range->nblocks,
+ false,
+ pgsr->strategy);
+ tail_range->need_complete = false;
+
+ /*
+ * We don't really know if the kernel generated an physical I/O
+ * when we issued advice, let alone when it finished, but it has
+ * certainly finished after a read call returns.
+ */
+ if (tail_range->advice_issued)
+ pgsr->ios_in_progress--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = (char *) pgsr->per_buffer_data +
+ tail_range->per_buffer_data_index[buffer_index] *
+ pgsr->per_buffer_data_size;
+
+ return buffer;
+ }
+
+ /* Advance tail to next range, if there is one. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead, and unpin anything that wasn't consumed. */
+ pgsr->finished = true;
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7d601bef6dd..2157a97b973 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -472,7 +472,7 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool *hit);
@@ -501,7 +501,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -795,15 +795,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
+ buf = ReadBuffer_common(BMR_REL(reln),
forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+
return buf;
}
@@ -827,8 +821,9 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
SMgrRelation smgr = smgropen(rlocator, InvalidBackendId);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
mode, strategy, &hit);
}
@@ -1002,7 +997,7 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
bool hit;
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
+ buffer = ReadBuffer_common(bmr,
fork, extend_to - 1, mode, strategy,
&hit);
}
@@ -1016,18 +1011,11 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool *hit)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ Buffer buffer;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1046,175 +1034,339 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ *hit = false;
+
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ buffer = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum,
+ strategy,
+ hit);
+
+ /* At this point we do NOT hold any locks. */
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ {
+ /* if we just want zeroes and a lock, we're done */
+ ZeroBuffer(buffer, mode);
+ }
+ else if (!*hit)
+ {
+ /* we might need to perform I/O */
+ CompleteReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ 1,
+ mode == RBM_ZERO_ON_ERROR,
+ strategy);
+ }
+
+ return buffer;
+}
+
+/*
+ * Prepare to read a block. The buffer is pinned. If this is a 'hit', then
+ * the returned buffer can be used immediately. Otherwise, a physical read
+ * should be completed with CompleteReadBuffers(), or the buffer should be
+ * zeroed with ZeroBuffer(). PrepareReadBuffer() followed by
+ * CompleteReadBuffers() or ZeroBuffer() is equivalent to ReadBuffer(), but
+ * the caller has the opportunity to combine reads of multiple neighboring
+ * blocks into one CompleteReadBuffers() call.
+ *
+ * *foundPtr is set to true for a hit, and false for a miss.
+ */
+Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * CompleteReadBuffers() (so, not for hits, and not for buffers that
+ * are zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+static inline bool
+CompleteReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
+ {
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+/*
+ * Complete a set reads prepared with PrepareReadBuffers(). The buffers must
+ * cover a cluster of neighboring block numbers.
+ *
+ * Typically this performs one physical vector read covering the block range,
+ * but if some of the buffers have already been read in the meantime by any
+ * backend, zero or multiple reads may be performed.
+ */
+void
+CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy)
+{
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(strategy);
+ io_object = IOOBJECT_RELATION;
+ }
/*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
*/
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
else
+ pgBufferUsage.shared_blks_read += nblocks;
+
+ for (int i = 0; i < nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+#ifdef USE_ASSERT_CHECKING
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ /*
+ * We could get all the information from buffer headers, but it can be
+ * expensive to access buffer header cache lines so we make the caller
+ * provide all the information we need, and assert that it is
+ * consistent.
+ */
+ {
+ RelFileLocator xlocator;
+ ForkNumber xforknum;
+ BlockNumber xblocknum;
+
+ BufferGetTag(buffers[i], &xlocator, &xforknum, &xblocknum);
+ Assert(RelFileLocatorEquals(bmr.smgr->smgr_rlocator.locator, xlocator));
+ Assert(xforknum == forknum);
+ Assert(xblocknum == blocknum + i);
+ }
+#endif
+
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!CompleteReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ CompleteReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
{
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
}
else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- }
- }
-
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
- {
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
- }
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
- if (isLocalBuf)
- {
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if (zero_on_error || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
- }
- else
- {
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
- }
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
- return BufferDescriptorGetBuffer(bufHdr);
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
@@ -1228,11 +1380,8 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false. A read should be
+ * performed with CompleteReadBuffers().
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1291,19 +1440,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1368,19 +1508,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1412,15 +1543,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1774,7 +1899,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2043,7 +2168,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2066,7 +2191,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2381,7 +2506,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if PrepareReadBuffer() was called and
+ * CompleteReadBuffers() hasn't been called yet. We'll check by
+ * loading the flags without locking. This is racy, but it's OK to
+ * return false spuriously: when CompleteReadBuffers() calls
+ * StartBufferIO(), it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2390,7 +2520,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3458,7 +3588,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -4845,6 +4975,46 @@ ConditionalLockBuffer(Buffer buffer)
LW_EXCLUSIVE);
}
+/*
+ * Zero a buffer, and lock it as RBM_ZERO_AND_LOCK or
+ * RBM_ZERO_AND_CLEANUP_LOCK would. The buffer must be already pinned. It
+ * does not have to be valid, but it is valid and locked on return.
+ */
+void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* Verify that this backend is pinning the buffer exactly once.
*
@@ -5197,9 +5367,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5212,6 +5388,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 1be4f4f8daf..717b8f58daf 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -109,10 +109,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -288,7 +287,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -298,9 +297,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 563a0be5c74..0d7272e796e 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -147,7 +147,9 @@ smgrshutdown(int code, Datum arg)
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
- * This does not attempt to actually open the underlying file.
+ * This does not attempt to actually open the underlying files. The returned
+ * object remains valid at least until AtEOXact_SMgr() is called, or until
+ * smgrdestroy() is called in non-transaction backends.
*/
SMgrRelation
smgropen(RelFileLocator rlocator, BackendId backend)
@@ -259,10 +261,10 @@ smgrexists(SMgrRelation reln, ForkNumber forknum)
}
/*
- * smgrclose() -- Close and delete an SMgrRelation object.
+ * smgrdestroy() -- Delete an SMgrRelation object.
*/
void
-smgrclose(SMgrRelation reln)
+smgrdestroy(SMgrRelation reln)
{
SMgrRelation *owner;
ForkNumber forknum;
@@ -289,12 +291,14 @@ smgrclose(SMgrRelation reln)
}
/*
- * smgrrelease() -- Release all resources used by this object.
+ * smgrclose() -- Release all resources used by this object.
*
- * The object remains valid.
+ * The object remains valid, but is moved to the unknown list where it will
+ * be destroyed by AtEOXact_SMgr(). It may be re-owned if it is accessed by a
+ * relation before then.
*/
void
-smgrrelease(SMgrRelation reln)
+smgrclose(SMgrRelation reln)
{
for (ForkNumber forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
@@ -302,15 +306,20 @@ smgrrelease(SMgrRelation reln)
reln->smgr_cached_nblocks[forknum] = InvalidBlockNumber;
}
reln->smgr_targblock = InvalidBlockNumber;
+
+ if (reln->smgr_owner)
+ {
+ *reln->smgr_owner = NULL;
+ reln->smgr_owner = NULL;
+ dlist_push_tail(&unowned_relns, &reln->node);
+ }
}
/*
- * smgrreleaseall() -- Release resources used by all objects.
- *
- * This is called for PROCSIGNAL_BARRIER_SMGRRELEASE.
+ * smgrcloseall() -- Close all objects.
*/
void
-smgrreleaseall(void)
+smgrcloseall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -322,14 +331,17 @@ smgrreleaseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrrelease(reln);
+ smgrclose(reln);
}
/*
- * smgrcloseall() -- Close all existing SMgrRelation objects.
+ * smgrdestroyall() -- Destroy all SMgrRelation objects.
+ *
+ * It must be known that there are no pointers to SMgrRelations, other than
+ * those registered with smgrsetowner().
*/
void
-smgrcloseall(void)
+smgrdestroyall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -341,7 +353,7 @@ smgrcloseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrclose(reln);
+ smgrdestroy(reln);
}
/*
@@ -733,7 +745,8 @@ smgrimmedsync(SMgrRelation reln, ForkNumber forknum)
* AtEOXact_SMgr
*
* This routine is called during transaction commit or abort (it doesn't
- * particularly care which). All transient SMgrRelation objects are closed.
+ * particularly care which). All transient SMgrRelation objects are
+ * destroyed.
*
* We do this as a compromise between wanting transient SMgrRelations to
* live awhile (to amortize the costs of blind writes of multiple blocks)
@@ -747,7 +760,7 @@ AtEOXact_SMgr(void)
dlist_mutable_iter iter;
/*
- * Zap all unowned SMgrRelations. We rely on smgrclose() to remove each
+ * Zap all unowned SMgrRelations. We rely on smgrdestroy() to remove each
* one from the list.
*/
dlist_foreach_modify(iter, &unowned_relns)
@@ -757,7 +770,7 @@ AtEOXact_SMgr(void)
Assert(rel->smgr_owner == NULL);
- smgrclose(rel);
+ smgrdestroy(rel);
}
}
@@ -768,6 +781,6 @@ AtEOXact_SMgr(void)
bool
ProcessBarrierSmgrRelease(void)
{
- smgrreleaseall();
+ smgrcloseall();
return true;
}
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..a38f1acb37a 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,18 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+extern Buffer PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr);
+extern void CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -247,9 +265,13 @@ extern void LockBufferForCleanup(Buffer buffer);
extern bool ConditionalLockBufferForCleanup(Buffer buffer);
extern bool IsBufferCleanupOK(Buffer buffer);
extern bool HoldingBufferPinThatDelaysRecovery(void);
+extern void ZeroBuffer(Buffer buffer, ReadBufferMode mode);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 527cd2a0568..d8ffe397faf 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -85,8 +85,8 @@ extern void smgrclearowner(SMgrRelation *owner, SMgrRelation reln);
extern void smgrclose(SMgrRelation reln);
extern void smgrcloseall(void);
extern void smgrcloserellocator(RelFileLocatorBackend rlocator);
-extern void smgrrelease(SMgrRelation reln);
-extern void smgrreleaseall(void);
+extern void smgrdestroy(SMgrRelation reln);
+extern void smgrdestroyall(void);
extern void smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo);
extern void smgrdosyncall(SMgrRelation *rels, int nrels);
extern void smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 00000000000..40c3408c541
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,45 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index a584b1ddff3..6636cc82c09 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -561,12 +561,6 @@ typedef struct ViewOptions
*
* Very little code is authorized to touch rel->rd_smgr directly. Instead
* use this function to fetch its value.
- *
- * Note: since a relcache flush can cause the file handle to be closed again,
- * it's unwise to hold onto the pointer returned by this function for any
- * long period. Recommended practice is to just re-execute RelationGetSmgr
- * each time you need to access the SMgrRelation. It's quite cheap in
- * comparison to whatever an smgr function is going to do.
*/
static inline SMgrRelation
RelationGetSmgr(Relation rel)
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 91433d439b7..8007f17320a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2094,6 +2094,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
--
2.37.2
[text/x-patch] v2-0010-Hard-code-TBMIterateResult-offsets-array-size.patch (4.9K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/13-v2-0010-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From 0ee5aaf02cd59bec3f42a319d37f7b8755a53554 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v2 10/13] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 27 ++++++---------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 16 insertions(+), 23 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 0f4850065fb..f711c056143 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,21 +40,12 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "nodes/bitmapset.h"
#include "nodes/tidbitmap.h"
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -66,7 +57,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -78,7 +69,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -180,7 +171,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -389,7 +380,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -691,12 +682,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1470,8 +1456,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
* Create the TBMSharedIterator struct, with enough trailing space to
* serve the needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.37.2
[text/x-patch] v2-0013-BitmapHeapScan-uses-streaming-read-API.patch (28.2K, ../../CAAKRu_bQ9a2dB42Tz-mrsR+2MuG-ojRmqbX-x-TC08Uo_RunNw@mail.gmail.com/14-v2-0013-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From 9473ddcf05c4c3142fbc3fbc2371df2b8a8113e8 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:04:18 -0500
Subject: [PATCH v2 13/13] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
---
src/backend/access/heap/heapam.c | 68 +++++
src/backend/access/heap/heapam_handler.c | 88 +++---
src/backend/executor/nodeBitmapHeapscan.c | 343 +---------------------
src/include/access/heapam.h | 4 +
src/include/access/tableam.h | 14 +-
src/include/nodes/execnodes.h | 19 --
6 files changed, 117 insertions(+), 419 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b93f243c282..c965048af60 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -115,6 +115,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -335,6 +337,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -955,6 +973,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
@@ -1093,6 +1112,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10250,3 +10272,49 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hdesc->rs_base.shared_tbmiterator)
+ tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
+ else
+ tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ba6793a749c..53812584774 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2113,79 +2113,65 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno)
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API will
+ * only return TBMIterateResults for blocks actually fetched. When we
+ * skip fetching a block, we keep track of how many empty tuples to
+ * emit at the end of the BitmapHeapScan. We do not recheck all NULL
+ * tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, and the bitmap entries don't need rechecking, and all tuples on
- * the page are visible to our transaction.
- */
- if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2206,7 +2192,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2215,9 +2201,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2270,7 +2256,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/* Only count exact and lossy pages with visible tuples */
if (ntup > 0)
{
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
scan->exact_pages++;
else
scan->lossy_pages++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 284641fa8ea..128621f1306 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,11 +56,6 @@ static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapAccumCounters(BitmapHeapScanState *node,
TableScanDesc scan);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -91,14 +86,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -114,15 +101,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
}
else
{
@@ -145,20 +123,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
- }
-#endif
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
@@ -166,14 +130,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
if (!scan)
@@ -216,47 +172,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->initialized = true;
/* Get the first block. if none, end of scan */
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck))
return ExecClearTuple(slot);
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- BitmapAdjustPrefetchTarget(node);
}
- for (;;)
+ do
{
while (table_scan_bitmap_next_tuple(scan, slot))
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We prefetch before fetching the current pages. We expect that a
- * future streaming read API will do this, so do it now for
- * consistency.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -278,13 +203,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
return slot;
}
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
- break;
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
+ } while (table_scan_bitmap_next_block(scan, &node->recheck));
/*
* if we get here it means we are at the end of the scan..
@@ -318,221 +237,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult tbmpre;
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
- }
- return;
- }
-
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- TBMIterateResult tbmpre;
-
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
- */
-
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -578,22 +282,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -632,16 +326,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -674,19 +362,13 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -726,13 +408,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -816,14 +491,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, node->pscan_len);
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -855,11 +526,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d5..1cad9c04f01 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index ef1fcc02b1a..56683f9e4aa 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -795,17 +795,10 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, BlockNumber *blockno);
+ bool (*scan_bitmap_next_block) (TableScanDesc scan, bool *recheck);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1982,8 +1975,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* used after verifying the presence (at plan time or such).
*/
static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno)
+table_scan_bitmap_next_block(TableScanDesc scan, bool *recheck)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1993,7 +1985,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, blockno);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a59df51dd69..d41a3e134d8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1682,11 +1682,8 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
* mutex mutual exclusion for the prefetching variable
* and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
* phs_snapshot_data snapshot data shared to workers
@@ -1695,10 +1692,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
char phs_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
@@ -1709,16 +1703,10 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
* recheck do current page's tuples need recheck
@@ -1729,20 +1717,13 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
bool recheck;
- BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-16 17:35 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-16 17:35 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
In the attached v3, I've reordered the commits, updated some errant
comments, and improved the commit messages.
I've also made some updates to the TIDBitmap API that seem like a
clarity improvement to the API in general. These also reduce the diff
for GIN when separating the TBMIterateResult from the
TBM[Shared]Iterator. And these TIDBitmap API changes are now all in
their own commits (previously those were in the same commit as adding
the BitmapHeapScan streaming read user).
The three outstanding issues I see in the patch set are:
1) the lossy and exact page counters issue described in my previous
email
2) the TODO in the TIDBitmap API changes about being sure that setting
TBMIterateResult->blockno to InvalidBlockNumber is sufficient for
indicating an invalid TBMIterateResult (and an exhausted bitmap)
3) the streaming read API is not committed yet, so the last two patches
are not "done"
- Melanie
Attachments:
[text/x-diff] v3-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (6.3K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/2-v3-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From e0cee301b81400209a0e727a3d7daa1f435ba999 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v3 01/13] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
ExecBitmapHeapInitializeWorker() overwrote the snapshot in the scan
descriptor with the correct one provided by the parallel leader. Since
ExecBitmapHeapInitializeWorker() is now called before the scan
descriptor has been created, save the worker's snapshot in the
BitmapHeapScanState and pass it to table_beginscan_bm().
---
src/backend/access/table/tableam.c | 11 ------
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++++++++++++-----
src/include/access/tableam.h | 10 ++---
src/include/nodes/execnodes.h | 2 +
4 files changed, 42 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 6ed8cca05a1..e78d793f69c 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,17 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-void
-table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot)
-{
- Assert(IsMVCCSnapshot(snapshot));
-
- RegisterSnapshot(snapshot);
- scan->rs_snapshot = snapshot;
- scan->rs_flags |= SO_TEMP_SNAPSHOT;
-}
-
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c1e81ebed63..44bf38be3c9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -181,6 +181,34 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ Snapshot snapshot = node->ss.ps.state->es_snapshot;
+ uint32 extra_flags = 0;
+
+ /*
+ * Parallel workers must use the snapshot initialized by the
+ * parallel leader.
+ */
+ if (node->worker_snapshot)
+ {
+ snapshot = node->worker_snapshot;
+ extra_flags |= SO_TEMP_SNAPSHOT;
+ }
+
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ snapshot,
+ 0,
+ NULL,
+ extra_flags);
+ }
+
node->initialized = true;
}
@@ -604,7 +632,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -681,7 +710,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -739,6 +770,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
*/
scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
node->scan.plan.targetlist == NIL);
+ scanstate->worker_snapshot = NULL;
/*
* Miscellaneous initialization
@@ -787,11 +819,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
@@ -930,13 +957,13 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelBitmapHeapState *pstate;
- Snapshot snapshot;
Assert(node->ss.ps.state->es_query_dsa != NULL);
pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->pstate = pstate;
- snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
- table_scan_update_snapshot(node->ss.ss_currentScanDesc, snapshot);
+ node->worker_snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
+ Assert(IsMVCCSnapshot(node->worker_snapshot));
+ RegisterSnapshot(node->worker_snapshot);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5f8474871d2..5375dd7150f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -944,9 +944,10 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key,
+ uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
@@ -1038,11 +1039,6 @@ table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
allow_pagemode);
}
-/*
- * Update snapshot used by the scan.
- */
-extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
-
/*
* Return next tuple from `scan`, store in slot.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 444a5f0fd57..00c75fb10e2 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1726,6 +1726,7 @@ typedef struct ParallelBitmapHeapState
* shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * worker_snapshot snapshot for parallel worker
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1750,6 +1751,7 @@ typedef struct BitmapHeapScanState
TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ Snapshot worker_snapshot;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-diff] v3-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/3-v3-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 69cd001bcdade976a51985e714d1b30b090bb388 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v3 02/13] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 44bf38be3c9..a9ba2bdfb88 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,6 +108,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -760,16 +770,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
--
2.37.2
[text/x-diff] v3-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (14.2K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/4-v3-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From b29df9592f8b3a3966cf6fab40f56a0c113f3d57 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v3 03/13] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 ++++++
src/backend/executor/nodeBitmapHeapscan.c | 118 ++++++----------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 7 ++
src/include/nodes/execnodes.h | 8 +-
6 files changed, 94 insertions(+), 92 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 707460a5364..b93f243c282 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -955,6 +955,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1043,6 +1045,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1062,6 +1070,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d15a02b2be7..7661acac3a8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, and the bitmap entries don't need rechecking, and all tuples on
+ * the page are visible to our transaction.
+ */
+ if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2236,6 +2255,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a9ba2bdfb88..2e4f87ea3a3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,16 +108,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -211,6 +201,17 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags |= SO_TEMP_SNAPSHOT;
}
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
+ extra_flags |= SO_CAN_SKIP_FETCH;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
snapshot,
@@ -224,8 +225,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -245,32 +244,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!table_scan_bitmap_next_block(scan, tbmres))
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -318,52 +292,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -535,7 +490,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -586,7 +542,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -656,8 +612,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -667,7 +621,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -712,8 +665,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -757,8 +708,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -770,7 +719,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..3dfb19ec7d5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5375dd7150f..c193ea5db43 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed.
+ */
+ SO_CAN_SKIP_FETCH = 1 << 10,
} ScanOptions;
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 00c75fb10e2..6fb4ec07c5f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1711,10 +1711,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1736,9 +1733,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.37.2
[text/x-diff] v3-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (2.2K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/5-v3-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From 17fc9d4c35e42b6e870b7e7f7c3495114e393e8a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v3 04/13] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2e4f87ea3a3..35ef26221ba 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -484,15 +484,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
-
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -543,7 +537,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.37.2
[text/x-diff] v3-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/6-v3-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From 67a9fb1848718cabfcfd5c98368ab2aa79a6b213 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v3 05/13] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 35ef26221ba..3439c02e989 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -55,7 +55,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -242,7 +242,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
if (!table_scan_bitmap_next_block(scan, tbmres))
{
@@ -351,7 +351,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -370,7 +370,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.37.2
[text/x-diff] v3-0006-BitmapHeapScan-scan-desc-counts-lossy-and-exact-p.patch (4.0K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/7-v3-0006-BitmapHeapScan-scan-desc-counts-lossy-and-exact-p.patch)
download | inline diff:
From efbb311eddc765dd761154e1460e337fc2d29323 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:05:04 -0500
Subject: [PATCH v3 06/13] BitmapHeapScan scan desc counts lossy and exact
pages
Future commits will remove the TBMIterateResult from BitmapHeapNext(),
pushing it into the table AM-specific code. So we will have to keep
track of the number of lossy and exact pages in the scan descriptor.
Doing this change to lossy/exact page counting in a separate commit just
simplifies the diff.
---
src/backend/access/heap/heapam_handler.c | 9 +++++++++
src/backend/executor/nodeBitmapHeapscan.c | 19 ++++++++++++++-----
src/include/access/relscan.h | 4 ++++
src/include/access/tableam.h | 6 +++++-
4 files changed, 32 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 7661acac3a8..9fc99a87fdf 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2242,6 +2242,15 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ /* Only count exact and lossy pages with visible tuples */
+ if (ntup > 0)
+ {
+ if (tbmres->ntuples >= 0)
+ scan->exact_pages++;
+ else
+ scan->lossy_pages++;
+ }
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3439c02e989..eee90b8785b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -53,6 +53,8 @@
#include "utils/spccache.h"
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
+static inline void BitmapAccumCounters(BitmapHeapScanState *node,
+ TableScanDesc scan);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
BlockNumber blockno);
@@ -250,11 +252,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
continue;
}
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
- node->lossy_pages++;
-
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
@@ -322,15 +319,27 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* OK to return this tuple */
+ BitmapAccumCounters(node, scan);
return slot;
}
/*
* if we get here it means we are at the end of the scan..
*/
+ BitmapAccumCounters(node, scan);
return ExecClearTuple(slot);
}
+static inline void
+BitmapAccumCounters(BitmapHeapScanState *node,
+ TableScanDesc scan)
+{
+ node->exact_pages += scan->exact_pages;
+ scan->exact_pages = 0;
+ node->lossy_pages += scan->lossy_pages;
+ scan->lossy_pages = 0;
+}
+
/*
* BitmapDoneInitializingSharedState - Shared state is initialized
*
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..b74e08dd745 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -40,6 +40,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ long exact_pages;
+ long lossy_pages;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index c193ea5db43..7dfb291800c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -954,9 +954,13 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key,
uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->lossy_pages = 0;
+ result->exact_pages = 0;
+ return result;
}
/*
--
2.37.2
[text/x-diff] v3-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/8-v3-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From e42eea35eb863303eb0a914b96fe33103e3afcd9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v3 07/13] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index eee90b8785b..a0fe65fde58 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -76,8 +76,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -90,10 +88,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -110,6 +104,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -118,7 +115,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -171,8 +168,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -222,6 +218,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -235,9 +233,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.37.2
[text/x-diff] v3-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (4.1K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/9-v3-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From eee14d6a4cd7191201b158ed77e79abbefe6349f Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v3 08/13] Remove table_scan_bitmap_next_tuple parameter tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 9fc99a87fdf..3af9466b9ca 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2256,7 +2256,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a0fe65fde58..b4333184576 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -293,7 +293,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 7dfb291800c..2dc79583bcf 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -787,10 +787,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -817,15 +814,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -1987,7 +1979,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -1999,7 +1990,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.37.2
[text/x-diff] v3-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (19.3K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/10-v3-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From d579faa35292c4d3730a7fd112606fc419b7886a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:57:07 -0500
Subject: [PATCH v3 09/13] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 58 +++++++--
src/backend/executor/nodeBitmapHeapscan.c | 148 ++++++++--------------
src/include/access/relscan.h | 5 +
src/include/access/tableam.h | 58 ++++++---
src/include/nodes/execnodes.h | 9 +-
5 files changed, 150 insertions(+), 128 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3af9466b9ca..c8da3def645 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,17 +2114,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ bool *recheck, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, and the bitmap entries don't need rechecking, and all tuples on
@@ -2143,16 +2177,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2251,7 +2276,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
scan->lossy_pages++;
}
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index b4333184576..9109e8ddddf 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -76,7 +76,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -88,7 +87,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -116,7 +114,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -169,7 +166,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -218,46 +214,24 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ /* Get the first block. if none, end of scan */
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ return ExecClearTuple(slot);
+
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ BitmapAdjustPrefetchTarget(node);
}
for (;;)
{
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- if (!table_scan_bitmap_next_block(scan, tbmres))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
- {
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -279,46 +253,44 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
-
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
- }
+ /*
+ * We prefetch before fetching the current pages. We expect that a
+ * future streaming read API will do this, so do it this way now
+ * for consistency. Also, this should happen only when we have
+ * determined there is still something to do on the current page,
+ * else we may uselessly prefetch the same page we are just about
+ * to request for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
{
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
}
+
+ /* OK to return this tuple */
+ BitmapAccumCounters(node, scan);
+ return slot;
}
- /* OK to return this tuple */
- BitmapAccumCounters(node, scan);
- return slot;
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ break;
+
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -603,12 +575,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -616,13 +584,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -653,28 +620,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -707,8 +670,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -717,10 +678,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index b74e08dd745..5dea9c7a03d 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -41,6 +44,8 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
long exact_pages;
long lossy_pages;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2dc79583bcf..f1f5b7ab1d0 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -780,19 +781,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -808,7 +804,7 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ bool *recheck, BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -952,6 +948,8 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->lossy_pages = 0;
result->exact_pages = 0;
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
return result;
}
@@ -1012,6 +1010,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1022,6 +1035,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1945,17 +1973,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ bool *recheck, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1965,8 +1992,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6fb4ec07c5f..a59df51dd69 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1709,8 +1709,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1720,10 +1718,10 @@ typedef struct ParallelBitmapHeapState
* prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
+ * recheck do current page's tuples need recheck
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1731,8 +1729,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1742,10 +1738,11 @@ typedef struct BitmapHeapScanState
int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
+ bool recheck;
+ BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-diff] v3-0010-Hard-code-TBMIterateResult-offsets-array-size.patch (5.3K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/11-v3-0010-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From c2ba82cb19d21f79090598b81aee3184cd45a654 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v3 10/13] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 29 +++++++----------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 0f4850065fb..689a959b467 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,21 +40,12 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "nodes/bitmapset.h"
#include "nodes/tidbitmap.h"
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -66,7 +57,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -78,7 +69,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -180,7 +171,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -221,7 +212,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -389,7 +380,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -691,12 +682,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1470,8 +1456,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
* Create the TBMSharedIterator struct, with enough trailing space to
* serve the needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.37.2
[text/x-diff] v3-0011-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.3K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/12-v3-0011-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From 3f763a0fb8b16a84ef666cd9086402cb01171fab Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v3 11/13] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 33 +++++-----
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 107 insertions(+), 87 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3aa457a29e1 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c8da3def645..ba6793a749c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2121,7 +2121,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2134,11 +2134,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2153,11 +2153,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2165,19 +2165,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* the page are visible to our transaction.
*/
if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2206,7 +2206,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2215,9 +2215,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2270,7 +2270,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/* Only count exact and lossy pages with visible tuples */
if (ntup > 0)
{
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
scan->exact_pages++;
else
scan->lossy_pages++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 9109e8ddddf..bcc60d3cf98 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -347,9 +347,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
+ tbm_iterate(prefetch_iterator, &tbmpre);
- if (tbmpre == NULL || tbmpre->blockno != blockno)
+ if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
@@ -367,6 +368,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
}
else
{
+ TBMIterateResult tbmpre;
+
/* Release the mutex before iterating */
SpinLockRelease(&pstate->mutex);
@@ -379,7 +382,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
}
}
#endif /* USE_PREFETCH */
@@ -446,10 +449,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -465,13 +470,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -486,7 +491,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -505,8 +510,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -516,13 +521,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 689a959b467..b4dcb1cbb88 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -171,7 +171,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -212,7 +211,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -943,20 +941,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -984,6 +983,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -994,11 +994,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1014,16 +1014,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1033,10 +1034,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1087,13 +1087,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1103,21 +1103,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 51d0c74a6b0..e423d92b41c 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -352,7 +352,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.37.2
[text/x-diff] v3-0012-Streaming-Read-API.patch (56.0K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/13-v3-0012-Streaming-Read-API.patch)
download | inline diff:
From 6b9989da160c8a96a8e70ae276796b460c205ff0 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 22 Jul 2023 17:31:54 +1200
Subject: [PATCH v3 12/13] Streaming Read API
---
contrib/pg_prewarm/pg_prewarm.c | 40 +-
src/backend/access/transam/xlogutils.c | 2 +-
src/backend/postmaster/bgwriter.c | 8 +-
src/backend/postmaster/checkpointer.c | 15 +-
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 435 ++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 560 +++++++++++++++--------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/backend/storage/smgr/smgr.c | 49 +-
src/include/storage/bufmgr.h | 22 +
src/include/storage/smgr.h | 4 +-
src/include/storage/streaming_read.h | 45 ++
src/include/utils/rel.h | 6 -
src/tools/pgindent/typedefs.list | 2 +
17 files changed, 986 insertions(+), 238 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index 8541e4d6e46..9617bf130bd 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -20,6 +20,7 @@
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
+#include "storage/streaming_read.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
@@ -38,6 +39,25 @@ typedef enum
static PGIOAlignedBlock blockbuffer;
+struct pg_prewarm_streaming_read_private
+{
+ BlockNumber blocknum;
+ int64 last_block;
+};
+
+static BlockNumber
+pg_prewarm_streaming_read_next(PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_data)
+{
+ struct pg_prewarm_streaming_read_private *p = pgsr_private;
+
+ if (p->blocknum <= p->last_block)
+ return p->blocknum++;
+
+ return InvalidBlockNumber;
+}
+
/*
* pg_prewarm(regclass, mode text, fork text,
* first_block int8, last_block int8)
@@ -183,18 +203,36 @@ pg_prewarm(PG_FUNCTION_ARGS)
}
else if (ptype == PREWARM_BUFFER)
{
+ struct pg_prewarm_streaming_read_private p;
+ PgStreamingRead *pgsr;
+
/*
* In buffer mode, we actually pull the data into shared_buffers.
*/
+
+ /* Set up the private state for our streaming buffer read callback. */
+ p.blocknum = first_block;
+ p.last_block = last_block;
+
+ pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ &p,
+ 0,
+ NULL,
+ BMR_REL(rel),
+ forkNumber,
+ pg_prewarm_streaming_read_next);
+
for (block = first_block; block <= last_block; ++block)
{
Buffer buf;
CHECK_FOR_INTERRUPTS();
- buf = ReadBufferExtended(rel, forkNumber, block, RBM_NORMAL, NULL);
+ buf = pg_streaming_read_buffer_get_next(pgsr, NULL);
ReleaseBuffer(buf);
++blocks_done;
}
+ Assert(pg_streaming_read_buffer_get_next(pgsr, NULL) == InvalidBuffer);
+ pg_streaming_read_free(pgsr);
}
/* Close relation, release lock. */
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index aa8667abd10..8775b5789be 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -657,7 +657,7 @@ XLogDropDatabase(Oid dbid)
* This is unnecessarily heavy-handed, as it will close SMgrRelation
* objects for other databases as well. DROP DATABASE occurs seldom enough
* that it's not worth introducing a variant of smgrclose for just this
- * purpose. XXX: Or should we rather leave the smgr entries dangling?
+ * purpose.
*/
smgrcloseall();
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d7d6cc0cd7b..13e5376619e 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -246,10 +246,12 @@ BackgroundWriterMain(void)
if (FirstCallSinceLastCheckpoint())
{
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the bgwriter does
+ * not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
}
/*
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 5e949fc885b..5d843b61426 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -469,10 +469,12 @@ CheckpointerMain(void)
ckpt_performed = CreateRestartPoint(flags);
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the checkpointer
+ * does not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
/*
* Indicate checkpoint completion to any waiting backends.
@@ -958,11 +960,8 @@ RequestCheckpoint(int flags)
*/
CreateCheckPoint(flags | CHECKPOINT_IMMEDIATE);
- /*
- * After any checkpoint, close all smgr files. This is so we won't
- * hang onto smgr references to deleted files indefinitely.
- */
- smgrcloseall();
+ /* Free all smgr objects, as CheckpointerMain() normally would. */
+ smgrdestroyall();
return;
}
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..bcab44c802f
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..39aef2a84a2
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 00000000000..19605090fea
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,435 @@
+#include "postgres.h"
+
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ *
+ * For hits, need_to_complete is false and there is just one block per
+ * range, already pinned and ready for use.
+ *
+ * For misses, need_to_complete is true and buffers[] holds a range of
+ * blocks that are contiguous in storage (though the buffers may not be
+ * contiguous in memory), so we can complete them with a single call to
+ * CompleteReadBuffers().
+ */
+typedef struct PgStreamingReadRange
+{
+ bool advice_issued;
+ bool need_complete;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index[MAX_BUFFERS_PER_TRANSFER];
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+} PgStreamingReadRange;
+
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int ios_in_progress_trigger;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int pinned_buffers_trigger;
+ int next_tail_buffer;
+ bool finished;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ bool advice_enabled;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+ int per_buffer_data_next;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+static PgStreamingRead *
+pg_streaming_read_buffer_alloc_internal(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = maintenance_io_concurrency;
+ else
+ max_ios = effective_io_concurrency;
+
+ /*
+ * The desired level of I/O concurrency controls how far ahead we are
+ * willing to look ahead. We also clamp it to at least
+ * MAX_BUFFER_PER_TRANFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /*
+ * The *_io_concurrency GUCs, we might have 0. We want to allow at least
+ * one, to keep our gating logic simple.
+ */
+ max_ios = Max(max_ios, 1);
+
+ /*
+ * Don't allow this backend to pin too many buffers. For now we'll apply
+ * the limit for the shared buffer pool and the local buffer pool, without
+ * worrying which it is.
+ */
+ LimitAdditionalPins(&max_pinned_buffers);
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * We want to avoid creating ranges that are smaller than they could be
+ * just because we hit max_pinned_buffers. We only look ahead when the
+ * number of pinned buffers falls below this trigger number, or put
+ * another way, we stop looking ahead when we wouldn't be able to build a
+ * "full sized" range.
+ */
+ pgsr->pinned_buffers_trigger =
+ Max(1, (int) max_pinned_buffers - MAX_BUFFERS_PER_TRANSFER);
+
+ /* Space the callback to store extra data along with each block. */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * max_pinned_buffers);
+
+ return pgsr;
+}
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *result;
+
+ result = pg_streaming_read_buffer_alloc_internal(flags,
+ pgsr_private,
+ per_buffer_data_size,
+ strategy);
+ result->callback = next_block_cb;
+ result->bmr = bmr;
+ result->forknum = forknum;
+
+ return result;
+}
+
+/*
+ * Start building a new range. This is called after the previous one
+ * reached maximum size, or the callback's next block can't be merged with it.
+ *
+ * Since the previous head range has now reached its full potential size, this
+ * is also a good time to issue 'prefetch' advice, because we know that'll
+ * soon be reading. In future, we could start an actual I/O here.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_new_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If a call to CompleteReadBuffers() will be needed, and we can issue
+ * advice to the kernel to get the read started. We suppress it if the
+ * access pattern appears to be completely sequential, though, because on
+ * some systems that interfers with the kernel's own sequential read ahead
+ * heurstics and hurts performance.
+ */
+ if (pgsr->advice_enabled)
+ {
+ BlockNumber blocknum = head_range->blocknum;
+ int nblocks = head_range->nblocks;
+
+ if (head_range->need_complete && blocknum != pgsr->seq_blocknum)
+ {
+ SMgrRelation smgr =
+ pgsr->bmr.smgr ? pgsr->bmr.smgr :
+ RelationGetSmgr(pgsr->bmr.rel);
+
+ Assert(!head_range->advice_issued);
+
+ smgrprefetch(smgr, pgsr->forknum, blocknum, nblocks);
+
+ /*
+ * Count this as an I/O that is concurrently in progress, though
+ * we don't really know if the kernel generates a physical I/O.
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ }
+
+ /* Remember the block after this range, for sequence detection. */
+ pgsr->seq_blocknum = blocknum + nblocks;
+ }
+
+ /* Create a new head range. There must be space. */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ head_range = &pgsr->ranges[pgsr->head];
+ head_range->nblocks = 0;
+
+ return head_range;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ /*
+ * If we're finished or can't start more I/O, then don't look ahead.
+ */
+ if (pgsr->finished || pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * We'll also wait until the number of pinned buffers falls below our
+ * trigger level, so that we have the chance to create a full range.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger)
+ return;
+
+ do
+ {
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ Buffer buffer;
+ bool found;
+ bool need_complete;
+ PgStreamingReadRange *head_range;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks == lengthof(head_range->buffers))
+ {
+ Assert(head_range->need_complete);
+ head_range = pg_streaming_read_new_range(pgsr);
+
+ /*
+ * Give up now if I/O is saturated, or we wouldn't be able form
+ * another full range after this due to the pin limit.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger ||
+ pgsr->ios_in_progress == pgsr->max_ios)
+ break;
+ }
+
+ per_buffer_data = (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * pgsr->per_buffer_data_next;
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ pgsr->finished = true;
+ break;
+ }
+ bmr = pgsr->bmr;
+ forknum = pgsr->forknum;
+
+ Assert(pgsr->pinned_buffers < pgsr->max_pinned_buffers);
+
+ buffer = PrepareReadBuffer(bmr,
+ forknum,
+ blocknum,
+ pgsr->strategy,
+ &found);
+ pgsr->pinned_buffers++;
+
+ need_complete = !found;
+
+ /* Is there a head range that we can't extend? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks > 0 &&
+ (!need_complete ||
+ !head_range->need_complete ||
+ head_range->blocknum + head_range->nblocks != blocknum))
+ {
+ /* Yes, time to start building a new one. */
+ head_range = pg_streaming_read_new_range(pgsr);
+ Assert(head_range->nblocks == 0);
+ }
+
+ if (head_range->nblocks == 0)
+ {
+ /* Initialize a new range beginning at this block. */
+ head_range->blocknum = blocknum;
+ head_range->need_complete = need_complete;
+ head_range->advice_issued = false;
+ }
+ else
+ {
+ /* We can extend an existing range by one block. */
+ Assert(head_range->blocknum + head_range->nblocks == blocknum);
+ Assert(head_range->need_complete);
+ }
+
+ head_range->per_buffer_data_index[head_range->nblocks] = pgsr->per_buffer_data_next++;
+ head_range->buffers[head_range->nblocks] = buffer;
+ head_range->nblocks++;
+
+ if (pgsr->per_buffer_data_next == pgsr->max_pinned_buffers)
+ pgsr->per_buffer_data_next = 0;
+
+ } while (pgsr->pinned_buffers < pgsr->max_pinned_buffers &&
+ pgsr->ios_in_progress < pgsr->max_ios);
+
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ pg_streaming_read_new_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* See if we have one buffer to return. */
+ while (pgsr->tail != pgsr->head)
+ {
+ PgStreamingReadRange *tail_range;
+
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to perform an I/O before returning the buffers from this
+ * range?
+ */
+ if (tail_range->need_complete)
+ {
+ CompleteReadBuffers(pgsr->bmr,
+ tail_range->buffers,
+ pgsr->forknum,
+ tail_range->blocknum,
+ tail_range->nblocks,
+ false,
+ pgsr->strategy);
+ tail_range->need_complete = false;
+
+ /*
+ * We don't really know if the kernel generated an physical I/O
+ * when we issued advice, let alone when it finished, but it has
+ * certainly finished after a read call returns.
+ */
+ if (tail_range->advice_issued)
+ pgsr->ios_in_progress--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = (char *) pgsr->per_buffer_data +
+ tail_range->per_buffer_data_index[buffer_index] *
+ pgsr->per_buffer_data_size;
+
+ return buffer;
+ }
+
+ /* Advance tail to next range, if there is one. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead, and unpin anything that wasn't consumed. */
+ pgsr->finished = true;
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7d601bef6dd..2157a97b973 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -472,7 +472,7 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool *hit);
@@ -501,7 +501,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -795,15 +795,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
+ buf = ReadBuffer_common(BMR_REL(reln),
forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+
return buf;
}
@@ -827,8 +821,9 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
SMgrRelation smgr = smgropen(rlocator, InvalidBackendId);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
mode, strategy, &hit);
}
@@ -1002,7 +997,7 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
bool hit;
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
+ buffer = ReadBuffer_common(bmr,
fork, extend_to - 1, mode, strategy,
&hit);
}
@@ -1016,18 +1011,11 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool *hit)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ Buffer buffer;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1046,175 +1034,339 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ *hit = false;
+
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ buffer = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum,
+ strategy,
+ hit);
+
+ /* At this point we do NOT hold any locks. */
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ {
+ /* if we just want zeroes and a lock, we're done */
+ ZeroBuffer(buffer, mode);
+ }
+ else if (!*hit)
+ {
+ /* we might need to perform I/O */
+ CompleteReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ 1,
+ mode == RBM_ZERO_ON_ERROR,
+ strategy);
+ }
+
+ return buffer;
+}
+
+/*
+ * Prepare to read a block. The buffer is pinned. If this is a 'hit', then
+ * the returned buffer can be used immediately. Otherwise, a physical read
+ * should be completed with CompleteReadBuffers(), or the buffer should be
+ * zeroed with ZeroBuffer(). PrepareReadBuffer() followed by
+ * CompleteReadBuffers() or ZeroBuffer() is equivalent to ReadBuffer(), but
+ * the caller has the opportunity to combine reads of multiple neighboring
+ * blocks into one CompleteReadBuffers() call.
+ *
+ * *foundPtr is set to true for a hit, and false for a miss.
+ */
+Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * CompleteReadBuffers() (so, not for hits, and not for buffers that
+ * are zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+static inline bool
+CompleteReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
+ {
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+/*
+ * Complete a set reads prepared with PrepareReadBuffers(). The buffers must
+ * cover a cluster of neighboring block numbers.
+ *
+ * Typically this performs one physical vector read covering the block range,
+ * but if some of the buffers have already been read in the meantime by any
+ * backend, zero or multiple reads may be performed.
+ */
+void
+CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy)
+{
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(strategy);
+ io_object = IOOBJECT_RELATION;
+ }
/*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
*/
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
else
+ pgBufferUsage.shared_blks_read += nblocks;
+
+ for (int i = 0; i < nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+#ifdef USE_ASSERT_CHECKING
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ /*
+ * We could get all the information from buffer headers, but it can be
+ * expensive to access buffer header cache lines so we make the caller
+ * provide all the information we need, and assert that it is
+ * consistent.
+ */
+ {
+ RelFileLocator xlocator;
+ ForkNumber xforknum;
+ BlockNumber xblocknum;
+
+ BufferGetTag(buffers[i], &xlocator, &xforknum, &xblocknum);
+ Assert(RelFileLocatorEquals(bmr.smgr->smgr_rlocator.locator, xlocator));
+ Assert(xforknum == forknum);
+ Assert(xblocknum == blocknum + i);
+ }
+#endif
+
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!CompleteReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ CompleteReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
{
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
}
else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- }
- }
-
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
- {
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
- }
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
- if (isLocalBuf)
- {
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if (zero_on_error || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
- }
- else
- {
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
- }
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
- return BufferDescriptorGetBuffer(bufHdr);
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
@@ -1228,11 +1380,8 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false. A read should be
+ * performed with CompleteReadBuffers().
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1291,19 +1440,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1368,19 +1508,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1412,15 +1543,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1774,7 +1899,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2043,7 +2168,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2066,7 +2191,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2381,7 +2506,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if PrepareReadBuffer() was called and
+ * CompleteReadBuffers() hasn't been called yet. We'll check by
+ * loading the flags without locking. This is racy, but it's OK to
+ * return false spuriously: when CompleteReadBuffers() calls
+ * StartBufferIO(), it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2390,7 +2520,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3458,7 +3588,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -4845,6 +4975,46 @@ ConditionalLockBuffer(Buffer buffer)
LW_EXCLUSIVE);
}
+/*
+ * Zero a buffer, and lock it as RBM_ZERO_AND_LOCK or
+ * RBM_ZERO_AND_CLEANUP_LOCK would. The buffer must be already pinned. It
+ * does not have to be valid, but it is valid and locked on return.
+ */
+void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* Verify that this backend is pinning the buffer exactly once.
*
@@ -5197,9 +5367,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5212,6 +5388,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 1be4f4f8daf..717b8f58daf 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -109,10 +109,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -288,7 +287,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -298,9 +297,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 563a0be5c74..0d7272e796e 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -147,7 +147,9 @@ smgrshutdown(int code, Datum arg)
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
- * This does not attempt to actually open the underlying file.
+ * This does not attempt to actually open the underlying files. The returned
+ * object remains valid at least until AtEOXact_SMgr() is called, or until
+ * smgrdestroy() is called in non-transaction backends.
*/
SMgrRelation
smgropen(RelFileLocator rlocator, BackendId backend)
@@ -259,10 +261,10 @@ smgrexists(SMgrRelation reln, ForkNumber forknum)
}
/*
- * smgrclose() -- Close and delete an SMgrRelation object.
+ * smgrdestroy() -- Delete an SMgrRelation object.
*/
void
-smgrclose(SMgrRelation reln)
+smgrdestroy(SMgrRelation reln)
{
SMgrRelation *owner;
ForkNumber forknum;
@@ -289,12 +291,14 @@ smgrclose(SMgrRelation reln)
}
/*
- * smgrrelease() -- Release all resources used by this object.
+ * smgrclose() -- Release all resources used by this object.
*
- * The object remains valid.
+ * The object remains valid, but is moved to the unknown list where it will
+ * be destroyed by AtEOXact_SMgr(). It may be re-owned if it is accessed by a
+ * relation before then.
*/
void
-smgrrelease(SMgrRelation reln)
+smgrclose(SMgrRelation reln)
{
for (ForkNumber forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
@@ -302,15 +306,20 @@ smgrrelease(SMgrRelation reln)
reln->smgr_cached_nblocks[forknum] = InvalidBlockNumber;
}
reln->smgr_targblock = InvalidBlockNumber;
+
+ if (reln->smgr_owner)
+ {
+ *reln->smgr_owner = NULL;
+ reln->smgr_owner = NULL;
+ dlist_push_tail(&unowned_relns, &reln->node);
+ }
}
/*
- * smgrreleaseall() -- Release resources used by all objects.
- *
- * This is called for PROCSIGNAL_BARRIER_SMGRRELEASE.
+ * smgrcloseall() -- Close all objects.
*/
void
-smgrreleaseall(void)
+smgrcloseall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -322,14 +331,17 @@ smgrreleaseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrrelease(reln);
+ smgrclose(reln);
}
/*
- * smgrcloseall() -- Close all existing SMgrRelation objects.
+ * smgrdestroyall() -- Destroy all SMgrRelation objects.
+ *
+ * It must be known that there are no pointers to SMgrRelations, other than
+ * those registered with smgrsetowner().
*/
void
-smgrcloseall(void)
+smgrdestroyall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -341,7 +353,7 @@ smgrcloseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrclose(reln);
+ smgrdestroy(reln);
}
/*
@@ -733,7 +745,8 @@ smgrimmedsync(SMgrRelation reln, ForkNumber forknum)
* AtEOXact_SMgr
*
* This routine is called during transaction commit or abort (it doesn't
- * particularly care which). All transient SMgrRelation objects are closed.
+ * particularly care which). All transient SMgrRelation objects are
+ * destroyed.
*
* We do this as a compromise between wanting transient SMgrRelations to
* live awhile (to amortize the costs of blind writes of multiple blocks)
@@ -747,7 +760,7 @@ AtEOXact_SMgr(void)
dlist_mutable_iter iter;
/*
- * Zap all unowned SMgrRelations. We rely on smgrclose() to remove each
+ * Zap all unowned SMgrRelations. We rely on smgrdestroy() to remove each
* one from the list.
*/
dlist_foreach_modify(iter, &unowned_relns)
@@ -757,7 +770,7 @@ AtEOXact_SMgr(void)
Assert(rel->smgr_owner == NULL);
- smgrclose(rel);
+ smgrdestroy(rel);
}
}
@@ -768,6 +781,6 @@ AtEOXact_SMgr(void)
bool
ProcessBarrierSmgrRelease(void)
{
- smgrreleaseall();
+ smgrcloseall();
return true;
}
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..a38f1acb37a 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,18 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+extern Buffer PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr);
+extern void CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -247,9 +265,13 @@ extern void LockBufferForCleanup(Buffer buffer);
extern bool ConditionalLockBufferForCleanup(Buffer buffer);
extern bool IsBufferCleanupOK(Buffer buffer);
extern bool HoldingBufferPinThatDelaysRecovery(void);
+extern void ZeroBuffer(Buffer buffer, ReadBufferMode mode);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 527cd2a0568..d8ffe397faf 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -85,8 +85,8 @@ extern void smgrclearowner(SMgrRelation *owner, SMgrRelation reln);
extern void smgrclose(SMgrRelation reln);
extern void smgrcloseall(void);
extern void smgrcloserellocator(RelFileLocatorBackend rlocator);
-extern void smgrrelease(SMgrRelation reln);
-extern void smgrreleaseall(void);
+extern void smgrdestroy(SMgrRelation reln);
+extern void smgrdestroyall(void);
extern void smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo);
extern void smgrdosyncall(SMgrRelation *rels, int nrels);
extern void smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 00000000000..40c3408c541
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,45 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index a584b1ddff3..6636cc82c09 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -561,12 +561,6 @@ typedef struct ViewOptions
*
* Very little code is authorized to touch rel->rd_smgr directly. Instead
* use this function to fetch its value.
- *
- * Note: since a relcache flush can cause the file handle to be closed again,
- * it's unwise to hold onto the pointer returned by this function for any
- * long period. Recommended practice is to just re-execute RelationGetSmgr
- * each time you need to access the SMgrRelation. It's quite cheap in
- * comparison to whatever an smgr function is going to do.
*/
static inline SMgrRelation
RelationGetSmgr(Relation rel)
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 91433d439b7..8007f17320a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2094,6 +2094,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
--
2.37.2
[text/x-diff] v3-0013-BitmapHeapScan-uses-streaming-read-API.patch (28.4K, ../../20240216173559.xiy5xcl5dqmsprns@liskov/14-v3-0013-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From 6469df2a68926093e40f82df15d85ceacc6e0ca5 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:04:18 -0500
Subject: [PATCH v3 13/13] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
ci-os-only:
---
src/backend/access/heap/heapam.c | 68 +++++
src/backend/access/heap/heapam_handler.c | 88 +++---
src/backend/executor/nodeBitmapHeapscan.c | 340 +---------------------
src/include/access/heapam.h | 4 +
src/include/access/tableam.h | 19 +-
src/include/nodes/execnodes.h | 19 --
6 files changed, 117 insertions(+), 421 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b93f243c282..c965048af60 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -115,6 +115,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -335,6 +337,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -955,6 +973,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
@@ -1093,6 +1112,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10250,3 +10272,49 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hdesc->rs_base.shared_tbmiterator)
+ tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
+ else
+ tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ba6793a749c..0237cd52b61 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2113,79 +2113,65 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno)
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, and the bitmap entries don't need rechecking, and all tuples on
- * the page are visible to our transaction.
- */
- if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2206,7 +2192,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2215,9 +2201,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2270,7 +2256,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/* Only count exact and lossy pages with visible tuples */
if (ntup > 0)
{
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
scan->exact_pages++;
else
scan->lossy_pages++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index bcc60d3cf98..5fd760a0f66 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,11 +56,6 @@ static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapAccumCounters(BitmapHeapScanState *node,
TableScanDesc scan);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -91,14 +86,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -114,15 +101,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
}
else
{
@@ -145,20 +123,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
- }
-#endif
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
@@ -166,14 +130,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -220,50 +176,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->initialized = true;
/* Get the first block. if none, end of scan */
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck))
return ExecClearTuple(slot);
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- BitmapAdjustPrefetchTarget(node);
}
- for (;;)
+ do
{
while (table_scan_bitmap_next_tuple(scan, slot))
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We prefetch before fetching the current pages. We expect that a
- * future streaming read API will do this, so do it this way now
- * for consistency. Also, this should happen only when we have
- * determined there is still something to do on the current page,
- * else we may uselessly prefetch the same page we are just about
- * to request for real.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -285,13 +207,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
return slot;
}
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno))
- break;
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
+ } while (table_scan_bitmap_next_block(scan, &node->recheck));
/*
* if we get here it means we are at the end of the scan..
@@ -325,215 +241,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult tbmpre;
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
- }
- return;
- }
-
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- TBMIterateResult tbmpre;
-
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -579,22 +286,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -633,16 +330,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -675,19 +366,13 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -727,13 +412,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -817,14 +495,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, node->pscan_len);
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -856,11 +530,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d5..1cad9c04f01 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index f1f5b7ab1d0..9fad92675f4 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -789,22 +789,10 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, BlockNumber *blockno);
+ bool (*scan_bitmap_next_block) (TableScanDesc scan, bool *recheck);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1981,8 +1969,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* used after verifying the presence (at plan time or such).
*/
static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno)
+table_scan_bitmap_next_block(TableScanDesc scan, bool *recheck)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1992,7 +1979,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, blockno);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a59df51dd69..d41a3e134d8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1682,11 +1682,8 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
* mutex mutual exclusion for the prefetching variable
* and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
* phs_snapshot_data snapshot data shared to workers
@@ -1695,10 +1692,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
char phs_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
@@ -1709,16 +1703,10 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
* recheck do current page's tuples need recheck
@@ -1729,20 +1717,13 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
bool recheck;
- BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-27 01:50 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-27 01:50 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Fri, Feb 16, 2024 at 12:35:59PM -0500, Melanie Plageman wrote:
> In the attached v3, I've reordered the commits, updated some errant
> comments, and improved the commit messages.
>
> I've also made some updates to the TIDBitmap API that seem like a
> clarity improvement to the API in general. These also reduce the diff
> for GIN when separating the TBMIterateResult from the
> TBM[Shared]Iterator. And these TIDBitmap API changes are now all in
> their own commits (previously those were in the same commit as adding
> the BitmapHeapScan streaming read user).
>
> The three outstanding issues I see in the patch set are:
> 1) the lossy and exact page counters issue described in my previous
> email
I've resolved this. I added a new patch to the set which starts counting
even pages with no visible tuples toward lossy and exact pages. After an
off-list conversation with Andres, it seems that this omission in master
may not have been intentional.
Once we have only two types of pages to differentiate between (lossy and
exact [no longer have to care about "has no visible tuples"]), it is
easy enough to pass a "lossy" boolean paramater to
table_scan_bitmap_next_block(). I've done this in the attached v4.
- Melanie
Attachments:
[text/x-diff] v4-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (6.3K, ../../20240227015028.knohvy3spaqwk7lf@liskov/2-v4-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From e0cee301b81400209a0e727a3d7daa1f435ba999 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v4 01/14] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
ExecBitmapHeapInitializeWorker() overwrote the snapshot in the scan
descriptor with the correct one provided by the parallel leader. Since
ExecBitmapHeapInitializeWorker() is now called before the scan
descriptor has been created, save the worker's snapshot in the
BitmapHeapScanState and pass it to table_beginscan_bm().
---
src/backend/access/table/tableam.c | 11 ------
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++++++++++++-----
src/include/access/tableam.h | 10 ++---
src/include/nodes/execnodes.h | 2 +
4 files changed, 42 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 6ed8cca05a1..e78d793f69c 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,17 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-void
-table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot)
-{
- Assert(IsMVCCSnapshot(snapshot));
-
- RegisterSnapshot(snapshot);
- scan->rs_snapshot = snapshot;
- scan->rs_flags |= SO_TEMP_SNAPSHOT;
-}
-
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c1e81ebed63..44bf38be3c9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -181,6 +181,34 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ Snapshot snapshot = node->ss.ps.state->es_snapshot;
+ uint32 extra_flags = 0;
+
+ /*
+ * Parallel workers must use the snapshot initialized by the
+ * parallel leader.
+ */
+ if (node->worker_snapshot)
+ {
+ snapshot = node->worker_snapshot;
+ extra_flags |= SO_TEMP_SNAPSHOT;
+ }
+
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ snapshot,
+ 0,
+ NULL,
+ extra_flags);
+ }
+
node->initialized = true;
}
@@ -604,7 +632,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -681,7 +710,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -739,6 +770,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
*/
scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
node->scan.plan.targetlist == NIL);
+ scanstate->worker_snapshot = NULL;
/*
* Miscellaneous initialization
@@ -787,11 +819,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
@@ -930,13 +957,13 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelBitmapHeapState *pstate;
- Snapshot snapshot;
Assert(node->ss.ps.state->es_query_dsa != NULL);
pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->pstate = pstate;
- snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
- table_scan_update_snapshot(node->ss.ss_currentScanDesc, snapshot);
+ node->worker_snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
+ Assert(IsMVCCSnapshot(node->worker_snapshot));
+ RegisterSnapshot(node->worker_snapshot);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5f8474871d2..5375dd7150f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -944,9 +944,10 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key,
+ uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
@@ -1038,11 +1039,6 @@ table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
allow_pagemode);
}
-/*
- * Update snapshot used by the scan.
- */
-extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
-
/*
* Return next tuple from `scan`, store in slot.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 444a5f0fd57..00c75fb10e2 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1726,6 +1726,7 @@ typedef struct ParallelBitmapHeapState
* shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * worker_snapshot snapshot for parallel worker
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1750,6 +1751,7 @@ typedef struct BitmapHeapScanState
TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ Snapshot worker_snapshot;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-diff] v4-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240227015028.knohvy3spaqwk7lf@liskov/3-v4-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 69cd001bcdade976a51985e714d1b30b090bb388 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v4 02/14] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 44bf38be3c9..a9ba2bdfb88 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,6 +108,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -760,16 +770,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
--
2.37.2
[text/x-diff] v4-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (14.2K, ../../20240227015028.knohvy3spaqwk7lf@liskov/4-v4-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From b29df9592f8b3a3966cf6fab40f56a0c113f3d57 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v4 03/14] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 ++++++
src/backend/executor/nodeBitmapHeapscan.c | 118 ++++++----------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 7 ++
src/include/nodes/execnodes.h | 8 +-
6 files changed, 94 insertions(+), 92 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 707460a5364..b93f243c282 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -955,6 +955,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1043,6 +1045,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1062,6 +1070,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d15a02b2be7..7661acac3a8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, and the bitmap entries don't need rechecking, and all tuples on
+ * the page are visible to our transaction.
+ */
+ if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2236,6 +2255,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a9ba2bdfb88..2e4f87ea3a3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,16 +108,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -211,6 +201,17 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags |= SO_TEMP_SNAPSHOT;
}
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
+ extra_flags |= SO_CAN_SKIP_FETCH;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
snapshot,
@@ -224,8 +225,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -245,32 +244,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!table_scan_bitmap_next_block(scan, tbmres))
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -318,52 +292,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -535,7 +490,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -586,7 +542,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -656,8 +612,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -667,7 +621,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -712,8 +665,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -757,8 +708,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -770,7 +719,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..3dfb19ec7d5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5375dd7150f..c193ea5db43 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed.
+ */
+ SO_CAN_SKIP_FETCH = 1 << 10,
} ScanOptions;
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 00c75fb10e2..6fb4ec07c5f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1711,10 +1711,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1736,9 +1733,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.37.2
[text/x-diff] v4-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (2.2K, ../../20240227015028.knohvy3spaqwk7lf@liskov/5-v4-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From 17fc9d4c35e42b6e870b7e7f7c3495114e393e8a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v4 04/14] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2e4f87ea3a3..35ef26221ba 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -484,15 +484,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
-
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -543,7 +537,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.37.2
[text/x-diff] v4-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../20240227015028.knohvy3spaqwk7lf@liskov/6-v4-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From 67a9fb1848718cabfcfd5c98368ab2aa79a6b213 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v4 05/14] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 35ef26221ba..3439c02e989 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -55,7 +55,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -242,7 +242,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
if (!table_scan_bitmap_next_block(scan, tbmres))
{
@@ -351,7 +351,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -370,7 +370,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.37.2
[text/x-diff] v4-0006-EXPLAIN-Bitmap-table-scan-also-count-no-visible-t.patch (3.0K, ../../20240227015028.knohvy3spaqwk7lf@liskov/7-v4-0006-EXPLAIN-Bitmap-table-scan-also-count-no-visible-t.patch)
download | inline diff:
From 4ad9d2798dff02537d0b5e7b807a5e80c7f0551d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 18:35:28 -0500
Subject: [PATCH v4 06/14] EXPLAIN Bitmap table scan also count no visible
tuple pages
Previously, bitmap heap scans only counted lossy and exact pages for
explain when there was at least one visible tuple on the page.
heapam_scan_bitmap_next_block() returned true only if there was a
"valid" page with tuples to be processed. However, the lossy and exact
page counters in EXPLAIN should count the number of pages represented in
a lossy or non-lossy way in the constructured bitmap, so it doesn't make
sense to omit pages without visible tuples.
---
src/backend/executor/nodeBitmapHeapscan.c | 15 ++++++++++-----
src/test/regress/expected/partition_prune.out | 4 +++-
2 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3439c02e989..75e896074bf 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -225,6 +225,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
+ bool valid;
+
CHECK_FOR_INTERRUPTS();
/*
@@ -244,17 +246,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- if (!table_scan_bitmap_next_block(scan, tbmres))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
+ valid = table_scan_bitmap_next_block(scan, tbmres);
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
+ if (!valid)
+ {
+ /* AM doesn't think this block is valid, skip */
+ continue;
+ }
+
+
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 9a4c48c0556..d9ec6492f96 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2709,6 +2709,7 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a = ab_a1.a;
Index Cond: (a = 1)
-> Bitmap Heap Scan on ab_a1_b3 ab_a1_3 (actual rows=0 loops=1)
Recheck Cond: (a = 1)
+ Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1 loops=1)
Index Cond: (a = 1)
-> Materialize (actual rows=1 loops=1)
@@ -2724,9 +2725,10 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a = ab_a1.a;
Index Cond: (a = 1)
-> Bitmap Heap Scan on ab_a1_b3 ab_3 (actual rows=0 loops=1)
Recheck Cond: (a = 1)
+ Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1 loops=1)
Index Cond: (a = 1)
-(34 rows)
+(36 rows)
table ab;
a | b
--
2.37.2
[text/x-diff] v4-0007-table_scan_bitmap_next_block-returns-lossy-or-exa.patch (4.4K, ../../20240227015028.knohvy3spaqwk7lf@liskov/8-v4-0007-table_scan_bitmap_next_block-returns-lossy-or-exa.patch)
download | inline diff:
From 1d179a330c870ac6cf78cc4be56fb6e48298d093 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v4 07/14] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 7661acac3a8..a6e52671d9b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2242,6 +2243,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 75e896074bf..054f745eeba 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -225,7 +225,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -246,12 +246,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index c193ea5db43..8280035e39f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -796,6 +796,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -811,7 +814,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1952,14 +1956,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1970,7 +1976,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.37.2
[text/x-diff] v4-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../20240227015028.knohvy3spaqwk7lf@liskov/9-v4-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From b5c9f5aef18124f93886c25fceb56706dcdb813a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v4 08/14] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 054f745eeba..a639d6e7415 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -74,8 +74,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -88,10 +86,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -108,6 +102,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -116,7 +113,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -169,8 +166,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -220,6 +216,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -235,9 +233,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.37.2
[text/x-diff] v4-0009-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (4.1K, ../../20240227015028.knohvy3spaqwk7lf@liskov/10-v4-0009-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From 46b086ad26c7c6a832892d91d3da2fd75f0a2039 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v4 09/14] Remove table_scan_bitmap_next_tuple parameter tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index a6e52671d9b..5dc9c51ca95 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2250,7 +2250,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a639d6e7415..87991266931 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -301,7 +301,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8280035e39f..8d7c800d157 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -787,10 +787,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -821,15 +818,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -1989,7 +1981,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2001,7 +1992,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.37.2
[text/x-diff] v4-0010-Make-table_scan_bitmap_next_block-async-friendly.patch (20.1K, ../../20240227015028.knohvy3spaqwk7lf@liskov/11-v4-0010-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From f43d9b0913815b98b2a6216440a2b5e87ad95936 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:57:07 -0500
Subject: [PATCH v4 10/14] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 167 +++++++++-------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 ++++++---
src/include/nodes/execnodes.h | 9 +-
5 files changed, 168 insertions(+), 142 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 5dc9c51ca95..a439ddc87bf 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,18 +2114,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, and the bitmap entries don't need rechecking, and all tuples on
@@ -2144,16 +2177,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2245,7 +2269,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 87991266931..3be433ea6e1 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -73,8 +73,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -86,7 +86,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -114,7 +113,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -167,7 +165,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -216,56 +213,29 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
- node->initialized = true;
- }
-
- for (;;)
- {
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
+ node->initialized = true;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
+ /* Get the first block. if none, end of scan */
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ return ExecClearTuple(slot);
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ BitmapAdjustPrefetchTarget(node);
+ }
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ for (;;)
+ {
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -287,45 +257,48 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
-
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
- }
+ /*
+ * We prefetch before fetching the current pages. We expect that a
+ * future streaming read API will do this, so do it this way now
+ * for consistency. Also, this should happen only when we have
+ * determined there is still something to do on the current page,
+ * else we may uselessly prefetch the same page we are just about
+ * to request for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
{
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
}
+
+ /* OK to return this tuple */
+ return slot;
}
- /* OK to return this tuple */
- return slot;
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -599,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -612,13 +581,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -649,28 +617,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -703,8 +667,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -713,10 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..92b829cebc7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8d7c800d157..2adead958cb 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -780,19 +781,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -811,8 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -950,9 +946,13 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key,
uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1012,6 +1012,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1022,6 +1037,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1945,19 +1975,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1967,8 +1995,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6fb4ec07c5f..a59df51dd69 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1709,8 +1709,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1720,10 +1718,10 @@ typedef struct ParallelBitmapHeapState
* prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
+ * recheck do current page's tuples need recheck
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1731,8 +1729,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1742,10 +1738,11 @@ typedef struct BitmapHeapScanState
int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
+ bool recheck;
+ BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-diff] v4-0011-Hard-code-TBMIterateResult-offsets-array-size.patch (5.3K, ../../20240227015028.knohvy3spaqwk7lf@liskov/12-v4-0011-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From 9c4b0c681205cdb4f48f544832fd0d4cd965f3c5 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v4 11/14] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 29 +++++++----------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 0f4850065fb..689a959b467 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,21 +40,12 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "nodes/bitmapset.h"
#include "nodes/tidbitmap.h"
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -66,7 +57,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -78,7 +69,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -180,7 +171,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -221,7 +212,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -389,7 +380,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -691,12 +682,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1470,8 +1456,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
* Create the TBMSharedIterator struct, with enough trailing space to
* serve the needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.37.2
[text/x-diff] v4-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.4K, ../../20240227015028.knohvy3spaqwk7lf@liskov/13-v4-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From e2bf17c5a936c3d536d9f25150b81d00969963b1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v4 12/14] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 33 +++++-----
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 107 insertions(+), 87 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3aa457a29e1 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index a439ddc87bf..daa5902e24d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2121,7 +2121,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2134,11 +2134,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2153,11 +2153,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2165,19 +2165,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* the page are visible to our transaction.
*/
if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2206,7 +2206,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2215,9 +2215,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2267,7 +2267,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ *lossy = tbmres.ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3be433ea6e1..74b92d4cbf4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -344,9 +344,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
+ tbm_iterate(prefetch_iterator, &tbmpre);
- if (tbmpre == NULL || tbmpre->blockno != blockno)
+ if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
@@ -364,6 +365,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
}
else
{
+ TBMIterateResult tbmpre;
+
/* Release the mutex before iterating */
SpinLockRelease(&pstate->mutex);
@@ -376,7 +379,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
}
}
#endif /* USE_PREFETCH */
@@ -443,10 +446,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -462,13 +467,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -483,7 +488,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -502,8 +507,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -513,13 +518,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 689a959b467..b4dcb1cbb88 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -171,7 +171,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -212,7 +211,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -943,20 +941,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -984,6 +983,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -994,11 +994,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1014,16 +1014,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1033,10 +1034,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1087,13 +1087,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1103,21 +1103,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 51d0c74a6b0..e423d92b41c 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -352,7 +352,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.37.2
[text/x-diff] v4-0013-Streaming-Read-API.patch (56.0K, ../../20240227015028.knohvy3spaqwk7lf@liskov/14-v4-0013-Streaming-Read-API.patch)
download | inline diff:
From 0a6454968309ddaa85653ff9efacd54072f7fc33 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Sat, 22 Jul 2023 17:31:54 +1200
Subject: [PATCH v4 13/14] Streaming Read API
---
contrib/pg_prewarm/pg_prewarm.c | 40 +-
src/backend/access/transam/xlogutils.c | 2 +-
src/backend/postmaster/bgwriter.c | 8 +-
src/backend/postmaster/checkpointer.c | 15 +-
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 435 ++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 560 +++++++++++++++--------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/backend/storage/smgr/smgr.c | 49 +-
src/include/storage/bufmgr.h | 22 +
src/include/storage/smgr.h | 4 +-
src/include/storage/streaming_read.h | 45 ++
src/include/utils/rel.h | 6 -
src/tools/pgindent/typedefs.list | 2 +
17 files changed, 986 insertions(+), 238 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index 8541e4d6e46..9617bf130bd 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -20,6 +20,7 @@
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
+#include "storage/streaming_read.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
@@ -38,6 +39,25 @@ typedef enum
static PGIOAlignedBlock blockbuffer;
+struct pg_prewarm_streaming_read_private
+{
+ BlockNumber blocknum;
+ int64 last_block;
+};
+
+static BlockNumber
+pg_prewarm_streaming_read_next(PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_data)
+{
+ struct pg_prewarm_streaming_read_private *p = pgsr_private;
+
+ if (p->blocknum <= p->last_block)
+ return p->blocknum++;
+
+ return InvalidBlockNumber;
+}
+
/*
* pg_prewarm(regclass, mode text, fork text,
* first_block int8, last_block int8)
@@ -183,18 +203,36 @@ pg_prewarm(PG_FUNCTION_ARGS)
}
else if (ptype == PREWARM_BUFFER)
{
+ struct pg_prewarm_streaming_read_private p;
+ PgStreamingRead *pgsr;
+
/*
* In buffer mode, we actually pull the data into shared_buffers.
*/
+
+ /* Set up the private state for our streaming buffer read callback. */
+ p.blocknum = first_block;
+ p.last_block = last_block;
+
+ pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ &p,
+ 0,
+ NULL,
+ BMR_REL(rel),
+ forkNumber,
+ pg_prewarm_streaming_read_next);
+
for (block = first_block; block <= last_block; ++block)
{
Buffer buf;
CHECK_FOR_INTERRUPTS();
- buf = ReadBufferExtended(rel, forkNumber, block, RBM_NORMAL, NULL);
+ buf = pg_streaming_read_buffer_get_next(pgsr, NULL);
ReleaseBuffer(buf);
++blocks_done;
}
+ Assert(pg_streaming_read_buffer_get_next(pgsr, NULL) == InvalidBuffer);
+ pg_streaming_read_free(pgsr);
}
/* Close relation, release lock. */
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index aa8667abd10..8775b5789be 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -657,7 +657,7 @@ XLogDropDatabase(Oid dbid)
* This is unnecessarily heavy-handed, as it will close SMgrRelation
* objects for other databases as well. DROP DATABASE occurs seldom enough
* that it's not worth introducing a variant of smgrclose for just this
- * purpose. XXX: Or should we rather leave the smgr entries dangling?
+ * purpose.
*/
smgrcloseall();
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index d7d6cc0cd7b..13e5376619e 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -246,10 +246,12 @@ BackgroundWriterMain(void)
if (FirstCallSinceLastCheckpoint())
{
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the bgwriter does
+ * not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
}
/*
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index 5e949fc885b..5d843b61426 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -469,10 +469,12 @@ CheckpointerMain(void)
ckpt_performed = CreateRestartPoint(flags);
/*
- * After any checkpoint, close all smgr files. This is so we
- * won't hang onto smgr references to deleted files indefinitely.
+ * After any checkpoint, free all smgr objects. Otherwise we
+ * would never do so for dropped relations, as the checkpointer
+ * does not process shared invalidation messages or call
+ * AtEOXact_SMgr().
*/
- smgrcloseall();
+ smgrdestroyall();
/*
* Indicate checkpoint completion to any waiting backends.
@@ -958,11 +960,8 @@ RequestCheckpoint(int flags)
*/
CreateCheckPoint(flags | CHECKPOINT_IMMEDIATE);
- /*
- * After any checkpoint, close all smgr files. This is so we won't
- * hang onto smgr references to deleted files indefinitely.
- */
- smgrcloseall();
+ /* Free all smgr objects, as CheckpointerMain() normally would. */
+ smgrdestroyall();
return;
}
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..bcab44c802f
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..39aef2a84a2
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 00000000000..19605090fea
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,435 @@
+#include "postgres.h"
+
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ *
+ * For hits, need_to_complete is false and there is just one block per
+ * range, already pinned and ready for use.
+ *
+ * For misses, need_to_complete is true and buffers[] holds a range of
+ * blocks that are contiguous in storage (though the buffers may not be
+ * contiguous in memory), so we can complete them with a single call to
+ * CompleteReadBuffers().
+ */
+typedef struct PgStreamingReadRange
+{
+ bool advice_issued;
+ bool need_complete;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index[MAX_BUFFERS_PER_TRANSFER];
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+} PgStreamingReadRange;
+
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int ios_in_progress_trigger;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int pinned_buffers_trigger;
+ int next_tail_buffer;
+ bool finished;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ bool advice_enabled;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+ int per_buffer_data_next;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+static PgStreamingRead *
+pg_streaming_read_buffer_alloc_internal(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = maintenance_io_concurrency;
+ else
+ max_ios = effective_io_concurrency;
+
+ /*
+ * The desired level of I/O concurrency controls how far ahead we are
+ * willing to look ahead. We also clamp it to at least
+ * MAX_BUFFER_PER_TRANFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /*
+ * The *_io_concurrency GUCs, we might have 0. We want to allow at least
+ * one, to keep our gating logic simple.
+ */
+ max_ios = Max(max_ios, 1);
+
+ /*
+ * Don't allow this backend to pin too many buffers. For now we'll apply
+ * the limit for the shared buffer pool and the local buffer pool, without
+ * worrying which it is.
+ */
+ LimitAdditionalPins(&max_pinned_buffers);
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * We want to avoid creating ranges that are smaller than they could be
+ * just because we hit max_pinned_buffers. We only look ahead when the
+ * number of pinned buffers falls below this trigger number, or put
+ * another way, we stop looking ahead when we wouldn't be able to build a
+ * "full sized" range.
+ */
+ pgsr->pinned_buffers_trigger =
+ Max(1, (int) max_pinned_buffers - MAX_BUFFERS_PER_TRANSFER);
+
+ /* Space the callback to store extra data along with each block. */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * max_pinned_buffers);
+
+ return pgsr;
+}
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *result;
+
+ result = pg_streaming_read_buffer_alloc_internal(flags,
+ pgsr_private,
+ per_buffer_data_size,
+ strategy);
+ result->callback = next_block_cb;
+ result->bmr = bmr;
+ result->forknum = forknum;
+
+ return result;
+}
+
+/*
+ * Start building a new range. This is called after the previous one
+ * reached maximum size, or the callback's next block can't be merged with it.
+ *
+ * Since the previous head range has now reached its full potential size, this
+ * is also a good time to issue 'prefetch' advice, because we know that'll
+ * soon be reading. In future, we could start an actual I/O here.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_new_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If a call to CompleteReadBuffers() will be needed, and we can issue
+ * advice to the kernel to get the read started. We suppress it if the
+ * access pattern appears to be completely sequential, though, because on
+ * some systems that interfers with the kernel's own sequential read ahead
+ * heurstics and hurts performance.
+ */
+ if (pgsr->advice_enabled)
+ {
+ BlockNumber blocknum = head_range->blocknum;
+ int nblocks = head_range->nblocks;
+
+ if (head_range->need_complete && blocknum != pgsr->seq_blocknum)
+ {
+ SMgrRelation smgr =
+ pgsr->bmr.smgr ? pgsr->bmr.smgr :
+ RelationGetSmgr(pgsr->bmr.rel);
+
+ Assert(!head_range->advice_issued);
+
+ smgrprefetch(smgr, pgsr->forknum, blocknum, nblocks);
+
+ /*
+ * Count this as an I/O that is concurrently in progress, though
+ * we don't really know if the kernel generates a physical I/O.
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ }
+
+ /* Remember the block after this range, for sequence detection. */
+ pgsr->seq_blocknum = blocknum + nblocks;
+ }
+
+ /* Create a new head range. There must be space. */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ head_range = &pgsr->ranges[pgsr->head];
+ head_range->nblocks = 0;
+
+ return head_range;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ /*
+ * If we're finished or can't start more I/O, then don't look ahead.
+ */
+ if (pgsr->finished || pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * We'll also wait until the number of pinned buffers falls below our
+ * trigger level, so that we have the chance to create a full range.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger)
+ return;
+
+ do
+ {
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ Buffer buffer;
+ bool found;
+ bool need_complete;
+ PgStreamingReadRange *head_range;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks == lengthof(head_range->buffers))
+ {
+ Assert(head_range->need_complete);
+ head_range = pg_streaming_read_new_range(pgsr);
+
+ /*
+ * Give up now if I/O is saturated, or we wouldn't be able form
+ * another full range after this due to the pin limit.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger ||
+ pgsr->ios_in_progress == pgsr->max_ios)
+ break;
+ }
+
+ per_buffer_data = (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * pgsr->per_buffer_data_next;
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ pgsr->finished = true;
+ break;
+ }
+ bmr = pgsr->bmr;
+ forknum = pgsr->forknum;
+
+ Assert(pgsr->pinned_buffers < pgsr->max_pinned_buffers);
+
+ buffer = PrepareReadBuffer(bmr,
+ forknum,
+ blocknum,
+ pgsr->strategy,
+ &found);
+ pgsr->pinned_buffers++;
+
+ need_complete = !found;
+
+ /* Is there a head range that we can't extend? */
+ head_range = &pgsr->ranges[pgsr->head];
+ if (head_range->nblocks > 0 &&
+ (!need_complete ||
+ !head_range->need_complete ||
+ head_range->blocknum + head_range->nblocks != blocknum))
+ {
+ /* Yes, time to start building a new one. */
+ head_range = pg_streaming_read_new_range(pgsr);
+ Assert(head_range->nblocks == 0);
+ }
+
+ if (head_range->nblocks == 0)
+ {
+ /* Initialize a new range beginning at this block. */
+ head_range->blocknum = blocknum;
+ head_range->need_complete = need_complete;
+ head_range->advice_issued = false;
+ }
+ else
+ {
+ /* We can extend an existing range by one block. */
+ Assert(head_range->blocknum + head_range->nblocks == blocknum);
+ Assert(head_range->need_complete);
+ }
+
+ head_range->per_buffer_data_index[head_range->nblocks] = pgsr->per_buffer_data_next++;
+ head_range->buffers[head_range->nblocks] = buffer;
+ head_range->nblocks++;
+
+ if (pgsr->per_buffer_data_next == pgsr->max_pinned_buffers)
+ pgsr->per_buffer_data_next = 0;
+
+ } while (pgsr->pinned_buffers < pgsr->max_pinned_buffers &&
+ pgsr->ios_in_progress < pgsr->max_ios);
+
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ pg_streaming_read_new_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* See if we have one buffer to return. */
+ while (pgsr->tail != pgsr->head)
+ {
+ PgStreamingReadRange *tail_range;
+
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to perform an I/O before returning the buffers from this
+ * range?
+ */
+ if (tail_range->need_complete)
+ {
+ CompleteReadBuffers(pgsr->bmr,
+ tail_range->buffers,
+ pgsr->forknum,
+ tail_range->blocknum,
+ tail_range->nblocks,
+ false,
+ pgsr->strategy);
+ tail_range->need_complete = false;
+
+ /*
+ * We don't really know if the kernel generated an physical I/O
+ * when we issued advice, let alone when it finished, but it has
+ * certainly finished after a read call returns.
+ */
+ if (tail_range->advice_issued)
+ pgsr->ios_in_progress--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = (char *) pgsr->per_buffer_data +
+ tail_range->per_buffer_data_index[buffer_index] *
+ pgsr->per_buffer_data_size;
+
+ return buffer;
+ }
+
+ /* Advance tail to next range, if there is one. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead, and unpin anything that wasn't consumed. */
+ pgsr->finished = true;
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7d601bef6dd..2157a97b973 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -472,7 +472,7 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool *hit);
@@ -501,7 +501,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -795,15 +795,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
+ buf = ReadBuffer_common(BMR_REL(reln),
forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+
return buf;
}
@@ -827,8 +821,9 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
SMgrRelation smgr = smgropen(rlocator, InvalidBackendId);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
mode, strategy, &hit);
}
@@ -1002,7 +997,7 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
bool hit;
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
+ buffer = ReadBuffer_common(bmr,
fork, extend_to - 1, mode, strategy,
&hit);
}
@@ -1016,18 +1011,11 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool *hit)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ Buffer buffer;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1046,175 +1034,339 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ *hit = false;
+
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ buffer = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum,
+ strategy,
+ hit);
+
+ /* At this point we do NOT hold any locks. */
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ {
+ /* if we just want zeroes and a lock, we're done */
+ ZeroBuffer(buffer, mode);
+ }
+ else if (!*hit)
+ {
+ /* we might need to perform I/O */
+ CompleteReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ 1,
+ mode == RBM_ZERO_ON_ERROR,
+ strategy);
+ }
+
+ return buffer;
+}
+
+/*
+ * Prepare to read a block. The buffer is pinned. If this is a 'hit', then
+ * the returned buffer can be used immediately. Otherwise, a physical read
+ * should be completed with CompleteReadBuffers(), or the buffer should be
+ * zeroed with ZeroBuffer(). PrepareReadBuffer() followed by
+ * CompleteReadBuffers() or ZeroBuffer() is equivalent to ReadBuffer(), but
+ * the caller has the opportunity to combine reads of multiple neighboring
+ * blocks into one CompleteReadBuffers() call.
+ *
+ * *foundPtr is set to true for a hit, and false for a miss.
+ */
+Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * CompleteReadBuffers() (so, not for hits, and not for buffers that
+ * are zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+static inline bool
+CompleteReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
+ {
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+/*
+ * Complete a set reads prepared with PrepareReadBuffers(). The buffers must
+ * cover a cluster of neighboring block numbers.
+ *
+ * Typically this performs one physical vector read covering the block range,
+ * but if some of the buffers have already been read in the meantime by any
+ * backend, zero or multiple reads may be performed.
+ */
+void
+CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy)
+{
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(strategy);
+ io_object = IOOBJECT_RELATION;
+ }
/*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
*/
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
else
+ pgBufferUsage.shared_blks_read += nblocks;
+
+ for (int i = 0; i < nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+#ifdef USE_ASSERT_CHECKING
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ /*
+ * We could get all the information from buffer headers, but it can be
+ * expensive to access buffer header cache lines so we make the caller
+ * provide all the information we need, and assert that it is
+ * consistent.
+ */
+ {
+ RelFileLocator xlocator;
+ ForkNumber xforknum;
+ BlockNumber xblocknum;
+
+ BufferGetTag(buffers[i], &xlocator, &xforknum, &xblocknum);
+ Assert(RelFileLocatorEquals(bmr.smgr->smgr_rlocator.locator, xlocator));
+ Assert(xforknum == forknum);
+ Assert(xblocknum == blocknum + i);
+ }
+#endif
+
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!CompleteReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ CompleteReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
{
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
}
else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- }
- }
-
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
- {
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
- }
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
- if (isLocalBuf)
- {
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if (zero_on_error || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
- }
- else
- {
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
- }
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
- return BufferDescriptorGetBuffer(bufHdr);
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
@@ -1228,11 +1380,8 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false. A read should be
+ * performed with CompleteReadBuffers().
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1291,19 +1440,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1368,19 +1508,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called PrepareReadBuffer() but not yet CompleteReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1412,15 +1543,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1774,7 +1899,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2043,7 +2168,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2066,7 +2191,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2381,7 +2506,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if PrepareReadBuffer() was called and
+ * CompleteReadBuffers() hasn't been called yet. We'll check by
+ * loading the flags without locking. This is racy, but it's OK to
+ * return false spuriously: when CompleteReadBuffers() calls
+ * StartBufferIO(), it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2390,7 +2520,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3458,7 +3588,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -4845,6 +4975,46 @@ ConditionalLockBuffer(Buffer buffer)
LW_EXCLUSIVE);
}
+/*
+ * Zero a buffer, and lock it as RBM_ZERO_AND_LOCK or
+ * RBM_ZERO_AND_CLEANUP_LOCK would. The buffer must be already pinned. It
+ * does not have to be valid, but it is valid and locked on return.
+ */
+void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* Verify that this backend is pinning the buffer exactly once.
*
@@ -5197,9 +5367,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5212,6 +5388,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 1be4f4f8daf..717b8f58daf 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -109,10 +109,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -288,7 +287,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -298,9 +297,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 563a0be5c74..0d7272e796e 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -147,7 +147,9 @@ smgrshutdown(int code, Datum arg)
/*
* smgropen() -- Return an SMgrRelation object, creating it if need be.
*
- * This does not attempt to actually open the underlying file.
+ * This does not attempt to actually open the underlying files. The returned
+ * object remains valid at least until AtEOXact_SMgr() is called, or until
+ * smgrdestroy() is called in non-transaction backends.
*/
SMgrRelation
smgropen(RelFileLocator rlocator, BackendId backend)
@@ -259,10 +261,10 @@ smgrexists(SMgrRelation reln, ForkNumber forknum)
}
/*
- * smgrclose() -- Close and delete an SMgrRelation object.
+ * smgrdestroy() -- Delete an SMgrRelation object.
*/
void
-smgrclose(SMgrRelation reln)
+smgrdestroy(SMgrRelation reln)
{
SMgrRelation *owner;
ForkNumber forknum;
@@ -289,12 +291,14 @@ smgrclose(SMgrRelation reln)
}
/*
- * smgrrelease() -- Release all resources used by this object.
+ * smgrclose() -- Release all resources used by this object.
*
- * The object remains valid.
+ * The object remains valid, but is moved to the unknown list where it will
+ * be destroyed by AtEOXact_SMgr(). It may be re-owned if it is accessed by a
+ * relation before then.
*/
void
-smgrrelease(SMgrRelation reln)
+smgrclose(SMgrRelation reln)
{
for (ForkNumber forknum = 0; forknum <= MAX_FORKNUM; forknum++)
{
@@ -302,15 +306,20 @@ smgrrelease(SMgrRelation reln)
reln->smgr_cached_nblocks[forknum] = InvalidBlockNumber;
}
reln->smgr_targblock = InvalidBlockNumber;
+
+ if (reln->smgr_owner)
+ {
+ *reln->smgr_owner = NULL;
+ reln->smgr_owner = NULL;
+ dlist_push_tail(&unowned_relns, &reln->node);
+ }
}
/*
- * smgrreleaseall() -- Release resources used by all objects.
- *
- * This is called for PROCSIGNAL_BARRIER_SMGRRELEASE.
+ * smgrcloseall() -- Close all objects.
*/
void
-smgrreleaseall(void)
+smgrcloseall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -322,14 +331,17 @@ smgrreleaseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrrelease(reln);
+ smgrclose(reln);
}
/*
- * smgrcloseall() -- Close all existing SMgrRelation objects.
+ * smgrdestroyall() -- Destroy all SMgrRelation objects.
+ *
+ * It must be known that there are no pointers to SMgrRelations, other than
+ * those registered with smgrsetowner().
*/
void
-smgrcloseall(void)
+smgrdestroyall(void)
{
HASH_SEQ_STATUS status;
SMgrRelation reln;
@@ -341,7 +353,7 @@ smgrcloseall(void)
hash_seq_init(&status, SMgrRelationHash);
while ((reln = (SMgrRelation) hash_seq_search(&status)) != NULL)
- smgrclose(reln);
+ smgrdestroy(reln);
}
/*
@@ -733,7 +745,8 @@ smgrimmedsync(SMgrRelation reln, ForkNumber forknum)
* AtEOXact_SMgr
*
* This routine is called during transaction commit or abort (it doesn't
- * particularly care which). All transient SMgrRelation objects are closed.
+ * particularly care which). All transient SMgrRelation objects are
+ * destroyed.
*
* We do this as a compromise between wanting transient SMgrRelations to
* live awhile (to amortize the costs of blind writes of multiple blocks)
@@ -747,7 +760,7 @@ AtEOXact_SMgr(void)
dlist_mutable_iter iter;
/*
- * Zap all unowned SMgrRelations. We rely on smgrclose() to remove each
+ * Zap all unowned SMgrRelations. We rely on smgrdestroy() to remove each
* one from the list.
*/
dlist_foreach_modify(iter, &unowned_relns)
@@ -757,7 +770,7 @@ AtEOXact_SMgr(void)
Assert(rel->smgr_owner == NULL);
- smgrclose(rel);
+ smgrdestroy(rel);
}
}
@@ -768,6 +781,6 @@ AtEOXact_SMgr(void)
bool
ProcessBarrierSmgrRelease(void)
{
- smgrreleaseall();
+ smgrcloseall();
return true;
}
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..a38f1acb37a 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,18 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+extern Buffer PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr);
+extern void CompleteReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int nblocks,
+ bool zero_on_error,
+ BufferAccessStrategy strategy);
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -247,9 +265,13 @@ extern void LockBufferForCleanup(Buffer buffer);
extern bool ConditionalLockBufferForCleanup(Buffer buffer);
extern bool IsBufferCleanupOK(Buffer buffer);
extern bool HoldingBufferPinThatDelaysRecovery(void);
+extern void ZeroBuffer(Buffer buffer, ReadBufferMode mode);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index 527cd2a0568..d8ffe397faf 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -85,8 +85,8 @@ extern void smgrclearowner(SMgrRelation *owner, SMgrRelation reln);
extern void smgrclose(SMgrRelation reln);
extern void smgrcloseall(void);
extern void smgrcloserellocator(RelFileLocatorBackend rlocator);
-extern void smgrrelease(SMgrRelation reln);
-extern void smgrreleaseall(void);
+extern void smgrdestroy(SMgrRelation reln);
+extern void smgrdestroyall(void);
extern void smgrcreate(SMgrRelation reln, ForkNumber forknum, bool isRedo);
extern void smgrdosyncall(SMgrRelation *rels, int nrels);
extern void smgrdounlinkall(SMgrRelation *rels, int nrels, bool isRedo);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 00000000000..40c3408c541
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,45 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index a584b1ddff3..6636cc82c09 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -561,12 +561,6 @@ typedef struct ViewOptions
*
* Very little code is authorized to touch rel->rd_smgr directly. Instead
* use this function to fetch its value.
- *
- * Note: since a relcache flush can cause the file handle to be closed again,
- * it's unwise to hold onto the pointer returned by this function for any
- * long period. Recommended practice is to just re-execute RelationGetSmgr
- * each time you need to access the SMgrRelation. It's quite cheap in
- * comparison to whatever an smgr function is going to do.
*/
static inline SMgrRelation
RelationGetSmgr(Relation rel)
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 91433d439b7..8007f17320a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2094,6 +2094,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
--
2.37.2
[text/x-diff] v4-0014-BitmapHeapScan-uses-streaming-read-API.patch (28.5K, ../../20240227015028.knohvy3spaqwk7lf@liskov/15-v4-0014-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From 4c9c90df25b4e421c34913b5da3da071fd4b15e1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:04:18 -0500
Subject: [PATCH v4 14/14] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
---
src/backend/access/heap/heapam.c | 68 +++++
src/backend/access/heap/heapam_handler.c | 88 +++---
src/backend/executor/nodeBitmapHeapscan.c | 336 +---------------------
src/include/access/heapam.h | 4 +
src/include/access/tableam.h | 22 +-
src/include/nodes/execnodes.h | 19 --
6 files changed, 117 insertions(+), 420 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b93f243c282..c965048af60 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -115,6 +115,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -335,6 +337,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -955,6 +973,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
@@ -1093,6 +1112,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10250,3 +10272,49 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hdesc->rs_base.shared_tbmiterator)
+ tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
+ else
+ tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index daa5902e24d..cade7edd900 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2113,79 +2113,65 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, and the bitmap entries don't need rechecking, and all tuples on
- * the page are visible to our transaction.
- */
- if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2206,7 +2192,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2215,9 +2201,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2267,7 +2253,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres.ntuples < 0;
+ *lossy = tbmres->ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 74b92d4cbf4..c5a482cc175 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -54,11 +54,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -90,14 +85,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -113,15 +100,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
}
else
{
@@ -144,20 +122,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
- }
-#endif
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
@@ -165,14 +129,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -219,16 +175,13 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->initialized = true;
/* Get the first block. if none, end of scan */
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy))
return ExecClearTuple(slot);
if (lossy)
node->lossy_pages++;
else
node->exact_pages++;
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- BitmapAdjustPrefetchTarget(node);
}
for (;;)
@@ -237,37 +190,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We prefetch before fetching the current pages. We expect that a
- * future streaming read API will do this, so do it this way now
- * for consistency. Also, this should happen only when we have
- * determined there is still something to do on the current page,
- * else we may uselessly prefetch the same page we are just about
- * to request for real.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -288,17 +210,13 @@ BitmapHeapNext(BitmapHeapScanState *node)
return slot;
}
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy))
break;
if (lossy)
node->lossy_pages++;
else
node->exact_pages++;
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -322,215 +240,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult tbmpre;
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
- }
- return;
- }
-
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- TBMIterateResult tbmpre;
-
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -576,22 +285,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -630,16 +329,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -672,19 +365,13 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -724,13 +411,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -814,14 +494,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, node->pscan_len);
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -853,11 +529,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d5..1cad9c04f01 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2adead958cb..1a7b9db8b40 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -792,23 +792,11 @@ typedef struct TableAmRoutine
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool (*scan_bitmap_next_block) (TableScanDesc scan, bool *recheck,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1984,8 +1972,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* used after verifying the presence (at plan time or such).
*/
static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+table_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1995,8 +1982,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, lossy);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a59df51dd69..d41a3e134d8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1682,11 +1682,8 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
* mutex mutual exclusion for the prefetching variable
* and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
* phs_snapshot_data snapshot data shared to workers
@@ -1695,10 +1692,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
char phs_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
@@ -1709,16 +1703,10 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
* recheck do current page's tuples need recheck
@@ -1729,20 +1717,13 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
bool recheck;
- BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-27 14:22 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 2 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-02-27 14:22 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Mon, Feb 26, 2024 at 08:50:28PM -0500, Melanie Plageman wrote:
> On Fri, Feb 16, 2024 at 12:35:59PM -0500, Melanie Plageman wrote:
> > In the attached v3, I've reordered the commits, updated some errant
> > comments, and improved the commit messages.
> >
> > I've also made some updates to the TIDBitmap API that seem like a
> > clarity improvement to the API in general. These also reduce the diff
> > for GIN when separating the TBMIterateResult from the
> > TBM[Shared]Iterator. And these TIDBitmap API changes are now all in
> > their own commits (previously those were in the same commit as adding
> > the BitmapHeapScan streaming read user).
> >
> > The three outstanding issues I see in the patch set are:
> > 1) the lossy and exact page counters issue described in my previous
> > email
>
> I've resolved this. I added a new patch to the set which starts counting
> even pages with no visible tuples toward lossy and exact pages. After an
> off-list conversation with Andres, it seems that this omission in master
> may not have been intentional.
>
> Once we have only two types of pages to differentiate between (lossy and
> exact [no longer have to care about "has no visible tuples"]), it is
> easy enough to pass a "lossy" boolean paramater to
> table_scan_bitmap_next_block(). I've done this in the attached v4.
Thomas posted a new version of the Streaming Read API [1], so here is a
rebased v5. This should make it easier to review as it can be applied on
top of master.
- Melanie
[1] https://www.postgresql.org/message-id/CA%2BhUKGJtLyxcAEvLhVUhgD4fMQkOu3PDaj8Qb9SR_UsmzgsBpQ%40mail.g...
Attachments:
[text/x-diff] v5-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (6.3K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/2-v5-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 5f523e4839c935f3b126b0c388129eb919c82b81 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v5 01/14] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
ExecBitmapHeapInitializeWorker() overwrote the snapshot in the scan
descriptor with the correct one provided by the parallel leader. Since
ExecBitmapHeapInitializeWorker() is now called before the scan
descriptor has been created, save the worker's snapshot in the
BitmapHeapScanState and pass it to table_beginscan_bm().
---
src/backend/access/table/tableam.c | 11 ------
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++++++++++++-----
src/include/access/tableam.h | 10 ++---
src/include/nodes/execnodes.h | 2 +
4 files changed, 42 insertions(+), 28 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 6ed8cca05a1..e78d793f69c 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,17 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-void
-table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot)
-{
- Assert(IsMVCCSnapshot(snapshot));
-
- RegisterSnapshot(snapshot);
- scan->rs_snapshot = snapshot;
- scan->rs_flags |= SO_TEMP_SNAPSHOT;
-}
-
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c1e81ebed63..44bf38be3c9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -181,6 +181,34 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ Snapshot snapshot = node->ss.ps.state->es_snapshot;
+ uint32 extra_flags = 0;
+
+ /*
+ * Parallel workers must use the snapshot initialized by the
+ * parallel leader.
+ */
+ if (node->worker_snapshot)
+ {
+ snapshot = node->worker_snapshot;
+ extra_flags |= SO_TEMP_SNAPSHOT;
+ }
+
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ snapshot,
+ 0,
+ NULL,
+ extra_flags);
+ }
+
node->initialized = true;
}
@@ -604,7 +632,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -681,7 +710,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -739,6 +770,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
*/
scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
node->scan.plan.targetlist == NIL);
+ scanstate->worker_snapshot = NULL;
/*
* Miscellaneous initialization
@@ -787,11 +819,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
@@ -930,13 +957,13 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelBitmapHeapState *pstate;
- Snapshot snapshot;
Assert(node->ss.ps.state->es_query_dsa != NULL);
pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->pstate = pstate;
- snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
- table_scan_update_snapshot(node->ss.ss_currentScanDesc, snapshot);
+ node->worker_snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
+ Assert(IsMVCCSnapshot(node->worker_snapshot));
+ RegisterSnapshot(node->worker_snapshot);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5f8474871d2..5375dd7150f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -944,9 +944,10 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key,
+ uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
@@ -1038,11 +1039,6 @@ table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
allow_pagemode);
}
-/*
- * Update snapshot used by the scan.
- */
-extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
-
/*
* Return next tuple from `scan`, store in slot.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 444a5f0fd57..00c75fb10e2 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1726,6 +1726,7 @@ typedef struct ParallelBitmapHeapState
* shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * worker_snapshot snapshot for parallel worker
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1750,6 +1751,7 @@ typedef struct BitmapHeapScanState
TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ Snapshot worker_snapshot;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-diff] v5-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/3-v5-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From c76a0dc384143a23ac58d421df4d4956fee58961 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v5 02/14] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 44bf38be3c9..a9ba2bdfb88 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,6 +108,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -760,16 +770,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
--
2.37.2
[text/x-diff] v5-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (14.2K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/4-v5-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From b4148fc01e789700309cb144ca5e68bbcd9c2aa6 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v5 03/14] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 ++++++
src/backend/executor/nodeBitmapHeapscan.c | 118 ++++++----------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 7 ++
src/include/nodes/execnodes.h | 8 +-
6 files changed, 94 insertions(+), 92 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 707460a5364..b93f243c282 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -955,6 +955,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1043,6 +1045,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1062,6 +1070,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 680a50bf8b1..c9b9b4c00f1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2122,6 +2123,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, and the bitmap entries don't need rechecking, and all tuples on
+ * the page are visible to our transaction.
+ */
+ if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2234,6 +2253,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a9ba2bdfb88..2e4f87ea3a3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -108,16 +108,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -211,6 +201,17 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags |= SO_TEMP_SNAPSHOT;
}
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
+ extra_flags |= SO_CAN_SKIP_FETCH;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
snapshot,
@@ -224,8 +225,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -245,32 +244,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!table_scan_bitmap_next_block(scan, tbmres))
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -318,52 +292,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -535,7 +490,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -586,7 +542,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -656,8 +612,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -667,7 +621,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -712,8 +665,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -757,8 +708,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -770,7 +719,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
scanstate->worker_snapshot = NULL;
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..3dfb19ec7d5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5375dd7150f..c193ea5db43 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed.
+ */
+ SO_CAN_SKIP_FETCH = 1 << 10,
} ScanOptions;
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 00c75fb10e2..6fb4ec07c5f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1711,10 +1711,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1736,9 +1733,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.37.2
[text/x-diff] v5-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (2.2K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/5-v5-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From 3df428a0428b821ed1c19bb19a18e0d3b3d60a4a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v5 04/14] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2e4f87ea3a3..35ef26221ba 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -484,15 +484,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
-
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -543,7 +537,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.37.2
[text/x-diff] v5-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/6-v5-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From 43d0dd9b5661617bf61562f8c4bc189d66fc62a3 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v5 05/14] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 35ef26221ba..3439c02e989 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -55,7 +55,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -242,7 +242,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
if (!table_scan_bitmap_next_block(scan, tbmres))
{
@@ -351,7 +351,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -370,7 +370,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.37.2
[text/x-diff] v5-0006-EXPLAIN-Bitmap-table-scan-also-count-no-visible-t.patch (3.0K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/7-v5-0006-EXPLAIN-Bitmap-table-scan-also-count-no-visible-t.patch)
download | inline diff:
From c0f80d78a1d2a940a825beeb394f8cd025d260c0 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 18:35:28 -0500
Subject: [PATCH v5 06/14] EXPLAIN Bitmap table scan also count no visible
tuple pages
Previously, bitmap heap scans only counted lossy and exact pages for
explain when there was at least one visible tuple on the page.
heapam_scan_bitmap_next_block() returned true only if there was a
"valid" page with tuples to be processed. However, the lossy and exact
page counters in EXPLAIN should count the number of pages represented in
a lossy or non-lossy way in the constructured bitmap, so it doesn't make
sense to omit pages without visible tuples.
---
src/backend/executor/nodeBitmapHeapscan.c | 15 ++++++++++-----
src/test/regress/expected/partition_prune.out | 4 +++-
2 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3439c02e989..75e896074bf 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -225,6 +225,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
+ bool valid;
+
CHECK_FOR_INTERRUPTS();
/*
@@ -244,17 +246,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- if (!table_scan_bitmap_next_block(scan, tbmres))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
+ valid = table_scan_bitmap_next_block(scan, tbmres);
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
+ if (!valid)
+ {
+ /* AM doesn't think this block is valid, skip */
+ continue;
+ }
+
+
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index b41950d923b..7b1b1e97033 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2812,6 +2812,7 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a = ab_a1.a;
Index Cond: (a = 1)
-> Bitmap Heap Scan on ab_a1_b3 ab_a1_3 (actual rows=0 loops=1)
Recheck Cond: (a = 1)
+ Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1 loops=1)
Index Cond: (a = 1)
-> Materialize (actual rows=1 loops=1)
@@ -2827,9 +2828,10 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a = ab_a1.a;
Index Cond: (a = 1)
-> Bitmap Heap Scan on ab_a1_b3 ab_3 (actual rows=0 loops=1)
Recheck Cond: (a = 1)
+ Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1 loops=1)
Index Cond: (a = 1)
-(34 rows)
+(36 rows)
table ab;
a | b
--
2.37.2
[text/x-diff] v5-0007-table_scan_bitmap_next_block-returns-lossy-or-exa.patch (4.4K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/8-v5-0007-table_scan_bitmap_next_block-returns-lossy-or-exa.patch)
download | inline diff:
From 0f9b10773db6b406f2d9a481293a2fd1dfe0669d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v5 07/14] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c9b9b4c00f1..10c1c3b616b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2112,7 +2112,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2240,6 +2241,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 75e896074bf..054f745eeba 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -225,7 +225,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -246,12 +246,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index c193ea5db43..8280035e39f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -796,6 +796,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -811,7 +814,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1952,14 +1956,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1970,7 +1976,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.37.2
[text/x-diff] v5-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/9-v5-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From 9df8ee50ea111a49a04397db0ff2b77d89eab7a9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v5 08/14] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 054f745eeba..a639d6e7415 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -74,8 +74,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -88,10 +86,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -108,6 +102,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -116,7 +113,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -169,8 +166,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -220,6 +216,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -235,9 +233,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.37.2
[text/x-diff] v5-0009-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (4.1K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/10-v5-0009-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From 8a31b11113194b26526c3931d98e022f7f1d6603 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v5 09/14] Remove table_scan_bitmap_next_tuple parameter tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 10c1c3b616b..a1ec50ab7a8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2248,7 +2248,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a639d6e7415..87991266931 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -301,7 +301,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8280035e39f..8d7c800d157 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -787,10 +787,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -821,15 +818,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -1989,7 +1981,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2001,7 +1992,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.37.2
[text/x-diff] v5-0010-Make-table_scan_bitmap_next_block-async-friendly.patch (20.1K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/11-v5-0010-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From b89d1e2133e1959750c9081e27dfa21f4fa7e46b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:57:07 -0500
Subject: [PATCH v5 10/14] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 167 +++++++++-------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 ++++++---
src/include/nodes/execnodes.h | 9 +-
5 files changed, 168 insertions(+), 142 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index a1ec50ab7a8..e038e60cd8f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2112,18 +2112,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, and the bitmap entries don't need rechecking, and all tuples on
@@ -2142,16 +2175,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2243,7 +2267,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 87991266931..3be433ea6e1 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -73,8 +73,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -86,7 +86,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -114,7 +113,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -167,7 +165,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -216,56 +213,29 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
- node->initialized = true;
- }
-
- for (;;)
- {
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
+ node->initialized = true;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
+ /* Get the first block. if none, end of scan */
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ return ExecClearTuple(slot);
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ BitmapAdjustPrefetchTarget(node);
+ }
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ for (;;)
+ {
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -287,45 +257,48 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
-
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
- }
+ /*
+ * We prefetch before fetching the current pages. We expect that a
+ * future streaming read API will do this, so do it this way now
+ * for consistency. Also, this should happen only when we have
+ * determined there is still something to do on the current page,
+ * else we may uselessly prefetch the same page we are just about
+ * to request for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
{
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
}
+
+ /* OK to return this tuple */
+ return slot;
}
- /* OK to return this tuple */
- return slot;
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -599,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -612,13 +581,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -649,28 +617,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -703,8 +667,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -713,10 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..92b829cebc7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8d7c800d157..2adead958cb 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -780,19 +781,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -811,8 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -950,9 +946,13 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key,
uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1012,6 +1012,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1022,6 +1037,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1945,19 +1975,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1967,8 +1995,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6fb4ec07c5f..a59df51dd69 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1709,8 +1709,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1720,10 +1718,10 @@ typedef struct ParallelBitmapHeapState
* prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
+ * recheck do current page's tuples need recheck
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1731,8 +1729,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1742,10 +1738,11 @@ typedef struct BitmapHeapScanState
int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
+ bool recheck;
+ BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
[text/x-diff] v5-0011-Hard-code-TBMIterateResult-offsets-array-size.patch (5.3K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/12-v5-0011-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From c6518284a8c20aa4d9e3e2267ad2dfa0acb2aefa Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v5 11/14] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 29 +++++++----------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fcc..d2bf8f44d50 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1467,8 +1453,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
* Create the TBMSharedIterator struct, with enough trailing space to
* serve the needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.37.2
[text/x-diff] v5-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.4K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/13-v5-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From 602ad80e9045384c19387145bff41893945423ab Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v5 12/14] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 33 +++++-----
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 107 insertions(+), 87 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3aa457a29e1 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e038e60cd8f..022753e203a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2119,7 +2119,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2132,11 +2132,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2151,11 +2151,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2163,19 +2163,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* the page are visible to our transaction.
*/
if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2204,7 +2204,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2213,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2265,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ *lossy = tbmres.ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 3be433ea6e1..74b92d4cbf4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -344,9 +344,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
+ tbm_iterate(prefetch_iterator, &tbmpre);
- if (tbmpre == NULL || tbmpre->blockno != blockno)
+ if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
@@ -364,6 +365,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
}
else
{
+ TBMIterateResult tbmpre;
+
/* Release the mutex before iterating */
SpinLockRelease(&pstate->mutex);
@@ -376,7 +379,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
}
}
#endif /* USE_PREFETCH */
@@ -443,10 +446,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -462,13 +467,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -483,7 +488,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -502,8 +507,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -513,13 +518,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index d2bf8f44d50..7d038c2018d 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae1..3b432263bb0 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.37.2
[text/x-diff] v5-0013-Streaming-Read-API.patch (56.9K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/14-v5-0013-Streaming-Read-API.patch)
download | inline diff:
From 1b50526e266f2413e04572f8ea5007805e2f20c2 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Mon, 26 Feb 2024 23:48:31 +1300
Subject: [PATCH v5 13/14] Streaming Read API
---
contrib/pg_prewarm/pg_prewarm.c | 40 +-
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 612 ++++++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 641 ++++++++++++++++-------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/include/storage/bufmgr.h | 45 ++
src/include/storage/streaming_read.h | 52 ++
src/tools/pgindent/typedefs.list | 3 +
11 files changed, 1218 insertions(+), 211 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/contrib/pg_prewarm/pg_prewarm.c b/contrib/pg_prewarm/pg_prewarm.c
index 8541e4d6e46..1cc84bcb0c2 100644
--- a/contrib/pg_prewarm/pg_prewarm.c
+++ b/contrib/pg_prewarm/pg_prewarm.c
@@ -20,6 +20,7 @@
#include "miscadmin.h"
#include "storage/bufmgr.h"
#include "storage/smgr.h"
+#include "storage/streaming_read.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
@@ -38,6 +39,25 @@ typedef enum
static PGIOAlignedBlock blockbuffer;
+struct pg_prewarm_streaming_read_private
+{
+ BlockNumber blocknum;
+ int64 last_block;
+};
+
+static BlockNumber
+pg_prewarm_streaming_read_next(PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_data)
+{
+ struct pg_prewarm_streaming_read_private *p = pgsr_private;
+
+ if (p->blocknum <= p->last_block)
+ return p->blocknum++;
+
+ return InvalidBlockNumber;
+}
+
/*
* pg_prewarm(regclass, mode text, fork text,
* first_block int8, last_block int8)
@@ -183,18 +203,36 @@ pg_prewarm(PG_FUNCTION_ARGS)
}
else if (ptype == PREWARM_BUFFER)
{
+ struct pg_prewarm_streaming_read_private p;
+ PgStreamingRead *pgsr;
+
/*
* In buffer mode, we actually pull the data into shared_buffers.
*/
+
+ /* Set up the private state for our streaming buffer read callback. */
+ p.blocknum = first_block;
+ p.last_block = last_block;
+
+ pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_FULL,
+ &p,
+ 0,
+ NULL,
+ BMR_REL(rel),
+ forkNumber,
+ pg_prewarm_streaming_read_next);
+
for (block = first_block; block <= last_block; ++block)
{
Buffer buf;
CHECK_FOR_INTERRUPTS();
- buf = ReadBufferExtended(rel, forkNumber, block, RBM_NORMAL, NULL);
+ buf = pg_streaming_read_buffer_get_next(pgsr, NULL);
ReleaseBuffer(buf);
++blocks_done;
}
+ Assert(pg_streaming_read_buffer_get_next(pgsr, NULL) == InvalidBuffer);
+ pg_streaming_read_free(pgsr);
}
/* Close relation, release lock. */
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..bcab44c802f
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..39aef2a84a2
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 00000000000..71f2c4a70b6
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,612 @@
+#include "postgres.h"
+
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ */
+typedef struct PgStreamingReadRange
+{
+ bool need_wait;
+ bool advice_issued;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index;
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+ ReadBuffersOperation operation;
+} PgStreamingReadRange;
+
+/*
+ * Streaming read object.
+ */
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int pinned_buffers_trigger;
+ int next_tail_buffer;
+ int ramp_up_pin_limit;
+ int ramp_up_pin_stall;
+ bool finished;
+ bool advice_enabled;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ /* Sometimes we need to buffer one block for flow control. */
+ BlockNumber unget_blocknum;
+ void *unget_per_buffer_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+static PgStreamingRead *
+pg_streaming_read_buffer_alloc_internal(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = maintenance_io_concurrency;
+ else
+ max_ios = effective_io_concurrency;
+
+ /*
+ * The desired level of I/O concurrency controls how far ahead we are
+ * willing to look ahead. We also clamp it to at least
+ * MAX_BUFFER_PER_TRANFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /*
+ * The *_io_concurrency GUCs might be set to 0, but we want to allow at
+ * least one, to keep our gating logic simple.
+ */
+ max_ios = Max(max_ios, 1);
+
+ /*
+ * Don't allow this backend to pin too many buffers. For now we'll apply
+ * the limit for the shared buffer pool and the local buffer pool, without
+ * worrying which it is.
+ */
+ LimitAdditionalPins(&max_pinned_buffers);
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * We start off building small ranges, but double that quickly, for the
+ * benefit of users that don't know how far ahead they'll read. This can
+ * be disabled by users that already know they'll read all the way.
+ */
+ if (flags & PGSR_FLAG_FULL)
+ pgsr->ramp_up_pin_limit = INT_MAX;
+ else
+ pgsr->ramp_up_pin_limit = 1;
+
+ /*
+ * We want to avoid creating ranges that are smaller than they could be
+ * just because we hit max_pinned_buffers. We only look ahead when the
+ * number of pinned buffers falls below this trigger number, or put
+ * another way, we stop looking ahead when we wouldn't be able to build a
+ * "full sized" range.
+ */
+ pgsr->pinned_buffers_trigger =
+ Max(1, (int) max_pinned_buffers - MAX_BUFFERS_PER_TRANSFER);
+
+ /* Space for the callback to store extra data along with each block. */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * max_pinned_buffers);
+
+ return pgsr;
+}
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *result;
+
+ result = pg_streaming_read_buffer_alloc_internal(flags,
+ pgsr_private,
+ per_buffer_data_size,
+ strategy);
+ result->callback = next_block_cb;
+ result->bmr = bmr;
+ result->forknum = forknum;
+
+ return result;
+}
+
+/*
+ * Find the per-buffer data index for the Nth block of a range.
+ */
+static int
+get_per_buffer_data_index(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ int result;
+
+ /*
+ * Find slot in the circular buffer of per-buffer data, without using the
+ * expensive % operator.
+ */
+ result = range->per_buffer_data_index + n;
+ if (result >= pgsr->max_pinned_buffers)
+ result -= pgsr->max_pinned_buffers;
+ Assert(result == (range->per_buffer_data_index + n) % pgsr->max_pinned_buffers);
+
+ return result;
+}
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static void *
+get_per_buffer_data_by_index(PgStreamingRead *pgsr, int per_buffer_data_index)
+{
+ return (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * per_buffer_data_index;
+}
+
+/*
+ * Return a pointer to the per-buffer data for the Nth block of a range.
+ */
+static void *
+get_per_buffer_data(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ return get_per_buffer_data_by_index(pgsr,
+ get_per_buffer_data_index(pgsr,
+ range,
+ n));
+}
+
+/*
+ * Start reading the head range, and create a new head range. The new head
+ * range is returned. It may not be empty, if StartReadBuffers() couldn't
+ * start the entire range; in that case the returned range contains the
+ * remaining portion of the range.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_start_head_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+ PgStreamingReadRange *new_head_range;
+ int nblocks_pinned;
+ int flags;
+
+ /* Caller should make sure we never exceed max_ios. */
+ Assert(pgsr->ios_in_progress < pgsr->max_ios);
+
+ /* Should only call if the head range has some blocks to read. */
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If advice hasn't been suppressed, and this system supports it, this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (pgsr->advice_enabled && head_range->blocknum != pgsr->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+
+ /* Start reading as many blocks as we can from the head range. */
+ nblocks_pinned = head_range->nblocks;
+ head_range->need_wait =
+ StartReadBuffers(pgsr->bmr,
+ head_range->buffers,
+ pgsr->forknum,
+ head_range->blocknum,
+ &nblocks_pinned,
+ pgsr->strategy,
+ flags,
+ &head_range->operation);
+
+ /* Did that start an I/O? */
+ if (head_range->need_wait && (flags & READ_BUFFERS_ISSUE_ADVICE))
+ {
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ Assert(pgsr->ios_in_progress <= pgsr->max_ios);
+ }
+
+ /*
+ * StartReadBuffers() might have pinned fewer blocks than we asked it to,
+ * but always at least one.
+ */
+ Assert(nblocks_pinned <= head_range->nblocks);
+ Assert(nblocks_pinned >= 1);
+ pgsr->pinned_buffers += nblocks_pinned;
+
+ /*
+ * Remember where the next block would be after that, so we can detect
+ * sequential access next time.
+ */
+ pgsr->seq_blocknum = head_range->blocknum + nblocks_pinned;
+
+ /*
+ * Create a new head range. There must be space, because we have enough
+ * elements for every range to hold just one block, up to the pin limit.
+ */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ new_head_range = &pgsr->ranges[pgsr->head];
+ new_head_range->nblocks = 0;
+ new_head_range->advice_issued = false;
+
+ /*
+ * If we didn't manage to start the whole read above, we split the range,
+ * moving the remainder into the new head range.
+ */
+ if (nblocks_pinned < head_range->nblocks)
+ {
+ int nblocks_remaining = head_range->nblocks - nblocks_pinned;
+
+ head_range->nblocks = nblocks_pinned;
+
+ new_head_range->blocknum = head_range->blocknum + nblocks_pinned;
+ new_head_range->nblocks = nblocks_remaining;
+ }
+
+ /* The new range has per-buffer data starting after the previous range. */
+ new_head_range->per_buffer_data_index =
+ get_per_buffer_data_index(pgsr, head_range, nblocks_pinned);
+
+ return new_head_range;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow pg_streaming_unget_block() to work.
+ */
+static BlockNumber
+pg_streaming_get_block(PgStreamingRead *pgsr, void *per_buffer_data)
+{
+ BlockNumber result;
+
+ if (unlikely(pgsr->unget_blocknum != InvalidBlockNumber))
+ {
+ /*
+ * If we had to unget a block, now it is time to return that one
+ * again.
+ */
+ result = pgsr->unget_blocknum;
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+ /*
+ * The same per_buffer_data element must have been used, and still
+ * contains whatever data the callback wrote into it. So we just
+ * sanity-check that we were called with the value that
+ * pg_streaming_unget_block() pushed back.
+ */
+ Assert(per_buffer_data == pgsr->unget_per_buffer_data);
+ }
+ else
+ {
+ /* Use the installed callback directly. */
+ result = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ }
+
+ return result;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later. This *must* be called with the
+ * last value returned by pg_streaming_get_block().
+ */
+static void
+pg_streaming_unget_block(PgStreamingRead *pgsr, BlockNumber blocknum, void *per_buffer_data)
+{
+ Assert(pgsr->unget_blocknum == InvalidBlockNumber);
+ pgsr->unget_blocknum = blocknum;
+ pgsr->unget_per_buffer_data = per_buffer_data;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *range;
+
+ /*
+ * If we're still ramping up, we may have to stall to wait for buffers to
+ * be consumed first before we do any more prefetching.
+ */
+ if (pgsr->ramp_up_pin_stall > 0)
+ {
+ Assert(pgsr->pinned_buffers > 0);
+ return;
+ }
+
+ /*
+ * If we're finished or can't start more I/O, then don't look ahead.
+ */
+ if (pgsr->finished || pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * We'll also wait until the number of pinned buffers falls below our
+ * trigger level, so that we have the chance to create a full range.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger)
+ return;
+
+ do
+ {
+ BlockNumber blocknum;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ range = &pgsr->ranges[pgsr->head];
+ if (range->nblocks == lengthof(range->buffers))
+ {
+ /* Start as much of it as we can. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /* If we're now at the I/O limit, stop here. */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * If we couldn't form a full range, then stop here to avoid
+ * creating small I/O.
+ */
+ if (pgsr->pinned_buffers >= pgsr->pinned_buffers_trigger)
+ return;
+
+ /*
+ * That might have only been partially started, but always
+ * processes at least one so that'll do for now.
+ */
+ Assert(range->nblocks < lengthof(range->buffers));
+ }
+
+ /* Find per-buffer data slot for the next block. */
+ per_buffer_data = get_per_buffer_data(pgsr, range, range->nblocks);
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pg_streaming_get_block(pgsr, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ pgsr->finished = true;
+ break;
+ }
+
+ /*
+ * Is there a head range that we cannot extend, because the requested
+ * block is not consecutive?
+ */
+ if (range->nblocks > 0 &&
+ range->blocknum + range->nblocks != blocknum)
+ {
+ /* Yes. Start it, so we can begin building a new one. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * It's possible that it was only partially started, and we have a
+ * new range with the remainder. Keep starting I/Os until we get
+ * it all out of the way, or we hit the I/O limit.
+ */
+ while (range->nblocks > 0 && pgsr->ios_in_progress < pgsr->max_ios)
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * We have to 'unget' the block returned by the callback if we
+ * don't have enough I/O capacity left to start something.
+ */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ {
+ pg_streaming_unget_block(pgsr, blocknum, per_buffer_data);
+ return;
+ }
+ }
+
+ /* If we have a new, empty range, initialize the start block. */
+ if (range->nblocks == 0)
+ {
+ range->blocknum = blocknum;
+ }
+
+ /* This block extends the range by one. */
+ Assert(range->blocknum + range->nblocks == blocknum);
+ range->nblocks++;
+
+ } while (pgsr->pinned_buffers + range->nblocks < pgsr->max_pinned_buffers &&
+ pgsr->pinned_buffers + range->nblocks < pgsr->ramp_up_pin_limit);
+
+ /* If we've hit the ramp-up limit, insert a stall. */
+ if (pgsr->pinned_buffers + range->nblocks >= pgsr->ramp_up_pin_limit)
+ {
+ /* Can't get here if an earlier stall hasn't finished. */
+ Assert(pgsr->ramp_up_pin_stall == 0);
+ /* Don't do any more prefetching until these buffers are consumed. */
+ pgsr->ramp_up_pin_stall = pgsr->ramp_up_pin_limit;
+ /* Double it. It will soon be out of the way. */
+ pgsr->ramp_up_pin_limit *= 2;
+ }
+
+ /* Start as much as we can. */
+ while (range->nblocks > 0)
+ {
+ range = pg_streaming_read_start_head_range(pgsr);
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ break;
+ }
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* See if we have one buffer to return. */
+ while (pgsr->tail != pgsr->head)
+ {
+ PgStreamingReadRange *tail_range;
+
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to perform an I/O before returning the buffers from this
+ * range?
+ */
+ if (tail_range->need_wait)
+ {
+ WaitReadBuffers(&tail_range->operation);
+ tail_range->need_wait = false;
+
+ /*
+ * We don't really know if the kernel generated a physical I/O
+ * when we issued advice, let alone when it finished, but it has
+ * certainly finished now because we've performed the read.
+ */
+ if (tail_range->advice_issued)
+ {
+ Assert(pgsr->ios_in_progress > 0);
+ pgsr->ios_in_progress--;
+ }
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (pgsr->ramp_up_pin_stall > 0)
+ pgsr->ramp_up_pin_stall--;
+
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(pgsr, tail_range, buffer_index);
+
+ return buffer;
+ }
+
+ /* Advance tail to next range, if there is one. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+
+ /*
+ * If tail crashed into head, and head is not empty, then it is time
+ * to start that range.
+ */
+ if (pgsr->tail == pgsr->head &&
+ pgsr->ranges[pgsr->head].nblocks > 0)
+ pg_streaming_read_start_head_range(pgsr);
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ pgsr->finished = true;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(pgsr->pinned_buffers == 0);
+ Assert(pgsr->ios_in_progress == 0);
+
+ /* Release memory. */
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index bdf89bbc4dc..3b1b0ad99df 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -472,10 +477,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -501,7 +505,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -782,7 +786,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -795,15 +798,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -823,13 +820,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, InvalidBackendId);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -995,35 +991,68 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
+/*
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
+ */
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* ReadBuffer_common -- common logic for all ReadBuffer variants
*
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+ BufferAccessStrategy strategy)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int nblocks;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1042,181 +1071,404 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ nblocks = 1;
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
+ else
+ flags = 0;
+ if (StartReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ &nblocks,
+ strategy,
+ flags,
+ &operation))
+ WaitReadBuffers(&operation);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ ZeroBuffer(buffer, mode);
+
+ return buffer;
+}
+
+static Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+ Assert(bmr.smgr);
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested.
+ *
+ * If false is returned, no I/O is necessary and WaitReadBuffers() is not
+ * necessary. If true is returned, one I/O has been started, and
+ * WaitReadBuffers() must be called with the same operation object before the
+ * buffers are accessed. Along with the operation object, the caller-supplied
+ * array of buffers must remain valid until WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+bool
+StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation)
+{
+ int actual_nblocks = *nblocks;
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ operation->bmr = bmr;
+ operation->forknum = forkNum;
+ operation->blocknum = blockNum;
+ operation->buffers = buffers;
+ operation->nblocks = actual_nblocks;
+ operation->strategy = strategy;
+ operation->flags = flags;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ operation->io_buffers_len = 0;
- /*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
- */
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
- else
+ for (int i = 0; i < actual_nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ bool found;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ buffers[i] = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum + i,
+ strategy,
+ &found);
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = operation->nblocks = *nblocks = i + 1;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ operation->io_buffers_len++;
+ }
+ }
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (operation->io_buffers_len > 0)
+ {
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
- {
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
+ /*
+ * In theory we should only do this if PrepareReadBuffers() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(bmr.smgr, forkNum, blockNum, operation->io_buffers_len);
}
+
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
}
+ else
+ {
+ return false;
+ }
+}
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
{
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ /*
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
+ */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+ bmr = operation->bmr;
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
-
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
}
else
{
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
}
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ /*
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
+ */
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
+ else
+ pgBufferUsage.shared_blks_read += nblocks;
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ for (int i = 0; i < nblocks; ++i)
+ {
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- return BufferDescriptorGetBuffer(bufHdr);
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
+
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
+ {
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
+ {
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
+ }
+ else
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
+
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
+
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
+
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
+
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for StartReadBuffers. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1224,11 +1476,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1287,19 +1535,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1364,19 +1603,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1408,15 +1638,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1770,7 +1994,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2035,7 +2259,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2058,7 +2282,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2373,7 +2597,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2382,7 +2611,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3450,7 +3679,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5185,9 +5414,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5200,6 +5435,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 1f02fed250e..6956d4e5b49 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -109,10 +109,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -288,7 +287,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -298,9 +297,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..b57f71f97e3 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,42 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+/*
+ * Private state used by StartReadBuffers() and WaitReadBuffers(). Declared
+ * in public header only to allow inclusion in other structs, but contents
+ * should not be accessed.
+ */
+struct ReadBuffersOperation
+{
+ /* Parameters passed in to StartReadBuffers(). */
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ int nblocks;
+ BufferAccessStrategy strategy;
+ int flags;
+
+ /* Range of buffers, if we need to perform a read. */
+ int io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +292,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 00000000000..c4d3892bb26
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,52 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define PGSR_FLAG_FULL 0x04
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fc8b15d0cf2..cfb58cf4836 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2097,6 +2097,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
@@ -2267,6 +2269,7 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
--
2.37.2
[text/x-diff] v5-0014-BitmapHeapScan-uses-streaming-read-API.patch (28.5K, ../../20240227142230.nu3ytvcjwouvczlt@liskov/15-v5-0014-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From e2267faf0fed9006fb0b437406737f1644d582c2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:04:18 -0500
Subject: [PATCH v5 14/14] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
---
src/backend/access/heap/heapam.c | 68 +++++
src/backend/access/heap/heapam_handler.c | 88 +++---
src/backend/executor/nodeBitmapHeapscan.c | 336 +---------------------
src/include/access/heapam.h | 4 +
src/include/access/tableam.h | 22 +-
src/include/nodes/execnodes.h | 19 --
6 files changed, 117 insertions(+), 420 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b93f243c282..c965048af60 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -115,6 +115,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -335,6 +337,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -955,6 +973,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
@@ -1093,6 +1112,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10250,3 +10272,49 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hdesc->rs_base.shared_tbmiterator)
+ tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
+ else
+ tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 022753e203a..9727613e87f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2111,79 +2111,65 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, and the bitmap entries don't need rechecking, and all tuples on
- * the page are visible to our transaction.
- */
- if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2204,7 +2190,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2199,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2251,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres.ntuples < 0;
+ *lossy = tbmres->ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 74b92d4cbf4..c5a482cc175 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -54,11 +54,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -90,14 +85,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -113,15 +100,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
}
else
{
@@ -144,20 +122,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
- }
-#endif
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
@@ -165,14 +129,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -219,16 +175,13 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->initialized = true;
/* Get the first block. if none, end of scan */
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy))
return ExecClearTuple(slot);
if (lossy)
node->lossy_pages++;
else
node->exact_pages++;
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- BitmapAdjustPrefetchTarget(node);
}
for (;;)
@@ -237,37 +190,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We prefetch before fetching the current pages. We expect that a
- * future streaming read API will do this, so do it this way now
- * for consistency. Also, this should happen only when we have
- * determined there is still something to do on the current page,
- * else we may uselessly prefetch the same page we are just about
- * to request for real.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -288,17 +210,13 @@ BitmapHeapNext(BitmapHeapScanState *node)
return slot;
}
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy))
break;
if (lossy)
node->lossy_pages++;
else
node->exact_pages++;
-
- BitmapAdjustPrefetchIterator(node, node->blockno);
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -322,215 +240,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult tbmpre;
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno) || tbmpre.blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
- }
- return;
- }
-
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- TBMIterateResult tbmpre;
-
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -576,22 +285,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -630,16 +329,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -672,19 +365,13 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->pscan_len = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->worker_snapshot = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -724,13 +411,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -814,14 +494,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, node->pscan_len);
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -853,11 +529,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d5..1cad9c04f01 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2adead958cb..1a7b9db8b40 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -792,23 +792,11 @@ typedef struct TableAmRoutine
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool (*scan_bitmap_next_block) (TableScanDesc scan, bool *recheck,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1984,8 +1972,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* used after verifying the presence (at plan time or such).
*/
static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+table_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1995,8 +1982,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, lossy);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index a59df51dd69..d41a3e134d8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1682,11 +1682,8 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
* mutex mutual exclusion for the prefetching variable
* and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
* phs_snapshot_data snapshot data shared to workers
@@ -1695,10 +1692,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
char phs_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
@@ -1709,16 +1703,10 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* worker_snapshot snapshot for parallel worker
* recheck do current page's tuples need recheck
@@ -1729,20 +1717,13 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
Size pscan_len;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
Snapshot worker_snapshot;
bool recheck;
- BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.37.2
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-28 13:22 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-02-28 13:22 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
Hi,
I haven't looked at the code very closely yet, but I decided to do some
basic benchmarks to see if/how this refactoring affects behavior.
Attached is a simple .sh script that
1) creates a table with one of a couple basic data distributions
(uniform, linear, ...), with an index on top
2) runs a simple query with a where condition matching a known fraction
of the table (0 - 100%), and measures duration
3) the query is forced to use bitmapscan by disabling other options
4) there's a couple parameters the script varies (work_mem, parallel
workers, ...), the script drops caches etc.
5) I only have results for table with 1M rows, which is ~320MB, so not
huge. I'm running this for larger data set, but that will take time.
I did this on my two "usual" machines - i5 and xeon. Both have flash
storage, although i5 is SATA and xeon has NVMe. I won't share the raw
results, because the CSV is like 5MB - ping me off-list if you need the
file, ofc.
Attached is PDF summarizing the results as a pivot table, with results
for "master" and "patched" builds. The interesting bit is the last
column, which shows whether the patch makes it faster (green) or slower
(red).
The results seem pretty mixed, on both machines. If you focus on the
uncached results (pages 4 and 8-9), there's both runs that are much
faster (by a factor of 2-5x) and slower (similar factor).
Of course, these results are with forced bitmap scans, so the question
is if those regressions even matter - maybe we'd use a different scan
type, making these changes less severe. So I logged "optimal plan" for
each run, tracking the scan type the optimizer would really pick without
all the enable_* GUCs. And the -optimal.pdf shows only results for the
runs where the optimal plan uses the bitmap scan. And yes, while the
impact of the changes (in either direction) is reduced, it's still very
much there.
What's a bit surprising to me is that these regressions affect runs with
effective_io_concurrency=0 in particular, which traditionally meant to
not do any prefetching / async stuff. I've perceived the patch mostly as
refactoring, so have not really expected such massive impact on these cases.
So I wonder if the refactoring means that we're actually doing some sort
amount of prefetching even with e_i_c=0. I'm not sure that'd be great, I
assume people have valid reasons to disable prefetching ...
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] bitmapscan-results.pdf (483.7K, ../../40c213cc-6a15-4e2c-9e56-dae344d52a73@enterprisedb.com/2-bitmapscan-results.pdf)
download
[application/pdf] bitmapscan-results-optimal.pdf (177.6K, ../../40c213cc-6a15-4e2c-9e56-dae344d52a73@enterprisedb.com/3-bitmapscan-results-optimal.pdf)
download
[application/x-shellscript] bitmapscan-test.sh (7.2K, ../../40c213cc-6a15-4e2c-9e56-dae344d52a73@enterprisedb.com/4-bitmapscan-test.sh)
download
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-02-28 14:38 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-28 14:38 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Wed, Feb 28, 2024 at 8:22 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> Hi,
>
> I haven't looked at the code very closely yet, but I decided to do some
> basic benchmarks to see if/how this refactoring affects behavior.
>
> Attached is a simple .sh script that
>
> 1) creates a table with one of a couple basic data distributions
> (uniform, linear, ...), with an index on top
>
> 2) runs a simple query with a where condition matching a known fraction
> of the table (0 - 100%), and measures duration
>
> 3) the query is forced to use bitmapscan by disabling other options
>
> 4) there's a couple parameters the script varies (work_mem, parallel
> workers, ...), the script drops caches etc.
>
> 5) I only have results for table with 1M rows, which is ~320MB, so not
> huge. I'm running this for larger data set, but that will take time.
>
>
> I did this on my two "usual" machines - i5 and xeon. Both have flash
> storage, although i5 is SATA and xeon has NVMe. I won't share the raw
> results, because the CSV is like 5MB - ping me off-list if you need the
> file, ofc.
I haven't looked at your results in detail yet. I plan to dig into
this more later today. But, I was wondering if it was easy for you to
run the shorter tests on just the commits before the last
https://github.com/melanieplageman/postgres/tree/bhs_pgsr
i.e. patches 0001-0013. Patch 0014 implements the streaming read user
and removes all of the existing prefetch code. I would be interested
to know if the behavior with just the preliminary refactoring differs
at all.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-28 14:56 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-02-28 14:56 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 2/28/24 15:38, Melanie Plageman wrote:
> On Wed, Feb 28, 2024 at 8:22 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> Hi,
>>
>> I haven't looked at the code very closely yet, but I decided to do some
>> basic benchmarks to see if/how this refactoring affects behavior.
>>
>> Attached is a simple .sh script that
>>
>> 1) creates a table with one of a couple basic data distributions
>> (uniform, linear, ...), with an index on top
>>
>> 2) runs a simple query with a where condition matching a known fraction
>> of the table (0 - 100%), and measures duration
>>
>> 3) the query is forced to use bitmapscan by disabling other options
>>
>> 4) there's a couple parameters the script varies (work_mem, parallel
>> workers, ...), the script drops caches etc.
>>
>> 5) I only have results for table with 1M rows, which is ~320MB, so not
>> huge. I'm running this for larger data set, but that will take time.
>>
>>
>> I did this on my two "usual" machines - i5 and xeon. Both have flash
>> storage, although i5 is SATA and xeon has NVMe. I won't share the raw
>> results, because the CSV is like 5MB - ping me off-list if you need the
>> file, ofc.
>
> I haven't looked at your results in detail yet. I plan to dig into
> this more later today. But, I was wondering if it was easy for you to
> run the shorter tests on just the commits before the last
> https://github.com/melanieplageman/postgres/tree/bhs_pgsr
> i.e. patches 0001-0013. Patch 0014 implements the streaming read user
> and removes all of the existing prefetch code. I would be interested
> to know if the behavior with just the preliminary refactoring differs
> at all.
>
Sure, I can do that. It'll take a couple hours to get the results, I'll
share them when I have them.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-02-28 19:23 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-02-28 19:23 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 2/28/24 15:56, Tomas Vondra wrote:
>> ...
>
> Sure, I can do that. It'll take a couple hours to get the results, I'll
> share them when I have them.
>
Here are the results with only patches 0001 - 0012 applied (i.e. without
the patch introducing the streaming read API, and the patch switching
the bitmap heap scan to use it).
The changes in performance don't disappear entirely, but the scale is
certainly much smaller - both in the complete results for all runs, and
for the "optimal" runs that would actually pick bitmapscan.
FWIW I'm not implying the patch must 100% maintain the current behavior,
or anything like that. At this point I'm more motivated to understand if
this change in behavior is expected and/or what this means for users.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] bitmapscan-refactoring-results-optimal.pdf (174.1K, ../../d0a5199d-f195-4378-8ac1-fa2534192041@enterprisedb.com/2-bitmapscan-refactoring-results-optimal.pdf)
download
[application/pdf] bitmapscan-refactoring-results.pdf (479.9K, ../../d0a5199d-f195-4378-8ac1-fa2534192041@enterprisedb.com/3-bitmapscan-refactoring-results.pdf)
download
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-02-28 20:06 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-28 20:06 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 2/28/24 15:56, Tomas Vondra wrote:
> >> ...
> >
> > Sure, I can do that. It'll take a couple hours to get the results, I'll
> > share them when I have them.
> >
>
> Here are the results with only patches 0001 - 0012 applied (i.e. without
> the patch introducing the streaming read API, and the patch switching
> the bitmap heap scan to use it).
>
> The changes in performance don't disappear entirely, but the scale is
> certainly much smaller - both in the complete results for all runs, and
> for the "optimal" runs that would actually pick bitmapscan.
Hmm. I'm trying to think how my refactor could have had this impact.
It seems like all the most notable regressions are with 4 parallel
workers. What do the numeric column labels mean across the top
(2,4,8,16...) -- are they related to "matches"? And if so, what does
that mean?
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-28 23:17 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-02-28 23:17 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 2/28/24 21:06, Melanie Plageman wrote:
> On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 2/28/24 15:56, Tomas Vondra wrote:
>>>> ...
>>>
>>> Sure, I can do that. It'll take a couple hours to get the results, I'll
>>> share them when I have them.
>>>
>>
>> Here are the results with only patches 0001 - 0012 applied (i.e. without
>> the patch introducing the streaming read API, and the patch switching
>> the bitmap heap scan to use it).
>>
>> The changes in performance don't disappear entirely, but the scale is
>> certainly much smaller - both in the complete results for all runs, and
>> for the "optimal" runs that would actually pick bitmapscan.
>
> Hmm. I'm trying to think how my refactor could have had this impact.
> It seems like all the most notable regressions are with 4 parallel
> workers. What do the numeric column labels mean across the top
> (2,4,8,16...) -- are they related to "matches"? And if so, what does
> that mean?
>
That's the number of distinct values matched by the query, which should
be an approximation of the number of matching rows. The number of
distinct values in the data set differs by data set, but for 1M rows
it's roughly like this:
uniform: 10k
linear: 10k
cyclic: 100
So for example matches=128 means ~1% of rows for uniform/linear, and
100% for cyclic data sets.
As for the possible cause, I think it's clear most of the difference
comes from the last patch that actually switches bitmap heap scan to the
streaming read API. That's mostly expected/understandable, although we
probably need to look into the regressions or cases with e_i_c=0.
To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
individual patches. I can try doing that tomorrow. It'll have to be a
limited set of tests, to reduce the time, but might tell us whether it's
due to a single patch or multiple patches.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-02-28 23:40 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-28 23:40 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Wed, Feb 28, 2024 at 6:17 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
>
>
> On 2/28/24 21:06, Melanie Plageman wrote:
> > On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >> On 2/28/24 15:56, Tomas Vondra wrote:
> >>>> ...
> >>>
> >>> Sure, I can do that. It'll take a couple hours to get the results, I'll
> >>> share them when I have them.
> >>>
> >>
> >> Here are the results with only patches 0001 - 0012 applied (i.e. without
> >> the patch introducing the streaming read API, and the patch switching
> >> the bitmap heap scan to use it).
> >>
> >> The changes in performance don't disappear entirely, but the scale is
> >> certainly much smaller - both in the complete results for all runs, and
> >> for the "optimal" runs that would actually pick bitmapscan.
> >
> > Hmm. I'm trying to think how my refactor could have had this impact.
> > It seems like all the most notable regressions are with 4 parallel
> > workers. What do the numeric column labels mean across the top
> > (2,4,8,16...) -- are they related to "matches"? And if so, what does
> > that mean?
> >
>
> That's the number of distinct values matched by the query, which should
> be an approximation of the number of matching rows. The number of
> distinct values in the data set differs by data set, but for 1M rows
> it's roughly like this:
>
> uniform: 10k
> linear: 10k
> cyclic: 100
>
> So for example matches=128 means ~1% of rows for uniform/linear, and
> 100% for cyclic data sets.
Ah, thank you for the explanation. I also looked at your script after
having sent this email and saw that it is clear in your script what
"matches" is.
> As for the possible cause, I think it's clear most of the difference
> comes from the last patch that actually switches bitmap heap scan to the
> streaming read API. That's mostly expected/understandable, although we
> probably need to look into the regressions or cases with e_i_c=0.
Right, I'm mostly surprised about the regressions for patches 0001-0012.
Re eic 0: Thomas Munro and I chatted off-list, and you bring up a
great point about eic 0. In old bitmapheapscan code eic 0 basically
disabled prefetching but with the streaming read API, it will still
issue fadvises when eic is 0. That is an easy one line fix. Thomas
prefers to fix it by always avoiding an fadvise for the last buffer in
a range before issuing a read (since we are about to read it anyway,
best not fadvise it too). This will fix eic 0 and also cut one system
call from each invocation of the streaming read machinery.
> To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
> individual patches. I can try doing that tomorrow. It'll have to be a
> limited set of tests, to reduce the time, but might tell us whether it's
> due to a single patch or multiple patches.
Yes, tomorrow I planned to start trying to repro some of the "red"
cases myself. Any one of the commits could cause a slight regression
but a 3.5x regression is quite surprising, so I might focus on trying
to repro that locally and then narrow down which patch causes it.
For the non-cached regressions, perhaps the commit to use the correct
recheck flag (0004) when prefetching could be the culprit. And for the
cached regressions, my money is on the commit which changes the whole
control flow of BitmapHeapNext() and the next_block() and next_tuple()
functions (0010).
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-29 12:54 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-02-29 12:54 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 2/29/24 00:40, Melanie Plageman wrote:
> On Wed, Feb 28, 2024 at 6:17 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>>
>>
>> On 2/28/24 21:06, Melanie Plageman wrote:
>>> On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
>>> <tomas.vondra@enterprisedb.com> wrote:
>>>>
>>>> On 2/28/24 15:56, Tomas Vondra wrote:
>>>>>> ...
>>>>>
>>>>> Sure, I can do that. It'll take a couple hours to get the results, I'll
>>>>> share them when I have them.
>>>>>
>>>>
>>>> Here are the results with only patches 0001 - 0012 applied (i.e. without
>>>> the patch introducing the streaming read API, and the patch switching
>>>> the bitmap heap scan to use it).
>>>>
>>>> The changes in performance don't disappear entirely, but the scale is
>>>> certainly much smaller - both in the complete results for all runs, and
>>>> for the "optimal" runs that would actually pick bitmapscan.
>>>
>>> Hmm. I'm trying to think how my refactor could have had this impact.
>>> It seems like all the most notable regressions are with 4 parallel
>>> workers. What do the numeric column labels mean across the top
>>> (2,4,8,16...) -- are they related to "matches"? And if so, what does
>>> that mean?
>>>
>>
>> That's the number of distinct values matched by the query, which should
>> be an approximation of the number of matching rows. The number of
>> distinct values in the data set differs by data set, but for 1M rows
>> it's roughly like this:
>>
>> uniform: 10k
>> linear: 10k
>> cyclic: 100
>>
>> So for example matches=128 means ~1% of rows for uniform/linear, and
>> 100% for cyclic data sets.
>
> Ah, thank you for the explanation. I also looked at your script after
> having sent this email and saw that it is clear in your script what
> "matches" is.
>
>> As for the possible cause, I think it's clear most of the difference
>> comes from the last patch that actually switches bitmap heap scan to the
>> streaming read API. That's mostly expected/understandable, although we
>> probably need to look into the regressions or cases with e_i_c=0.
>
> Right, I'm mostly surprised about the regressions for patches 0001-0012.
>
> Re eic 0: Thomas Munro and I chatted off-list, and you bring up a
> great point about eic 0. In old bitmapheapscan code eic 0 basically
> disabled prefetching but with the streaming read API, it will still
> issue fadvises when eic is 0. That is an easy one line fix. Thomas
> prefers to fix it by always avoiding an fadvise for the last buffer in
> a range before issuing a read (since we are about to read it anyway,
> best not fadvise it too). This will fix eic 0 and also cut one system
> call from each invocation of the streaming read machinery.
>
>> To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
>> individual patches. I can try doing that tomorrow. It'll have to be a
>> limited set of tests, to reduce the time, but might tell us whether it's
>> due to a single patch or multiple patches.
>
> Yes, tomorrow I planned to start trying to repro some of the "red"
> cases myself. Any one of the commits could cause a slight regression
> but a 3.5x regression is quite surprising, so I might focus on trying
> to repro that locally and then narrow down which patch causes it.
>
> For the non-cached regressions, perhaps the commit to use the correct
> recheck flag (0004) when prefetching could be the culprit. And for the
> cached regressions, my money is on the commit which changes the whole
> control flow of BitmapHeapNext() and the next_block() and next_tuple()
> functions (0010).
>
I do have some partial results, comparing the patches. I only ran one of
the more affected workloads (cyclic) on the xeon, attached is a PDF
comparing master and the 0001-0014 patches. The percentages are timing
vs. the preceding patch (green - faster, red - slower).
This suggests only patches 0010 and 0014 affect performance, the rest is
just noise. I'll see if I can do more runs and get data from the other
machine (seems it's more significant on old SATA SSDs).
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] patch-comparison.pdf (155.3K, ../../91090d58-7d3f-4447-9425-f24ba66e292a@enterprisedb.com/2-patch-comparison.pdf)
download
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-02-29 21:19 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-02-29 21:19 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Thu, Feb 29, 2024 at 7:54 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
>
>
> On 2/29/24 00:40, Melanie Plageman wrote:
> > On Wed, Feb 28, 2024 at 6:17 PM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >>
> >>
> >> On 2/28/24 21:06, Melanie Plageman wrote:
> >>> On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
> >>> <tomas.vondra@enterprisedb.com> wrote:
> >>>>
> >>>> On 2/28/24 15:56, Tomas Vondra wrote:
> >>>>>> ...
> >>>>>
> >>>>> Sure, I can do that. It'll take a couple hours to get the results, I'll
> >>>>> share them when I have them.
> >>>>>
> >>>>
> >>>> Here are the results with only patches 0001 - 0012 applied (i.e. without
> >>>> the patch introducing the streaming read API, and the patch switching
> >>>> the bitmap heap scan to use it).
> >>>>
> >>>> The changes in performance don't disappear entirely, but the scale is
> >>>> certainly much smaller - both in the complete results for all runs, and
> >>>> for the "optimal" runs that would actually pick bitmapscan.
> >>>
> >>> Hmm. I'm trying to think how my refactor could have had this impact.
> >>> It seems like all the most notable regressions are with 4 parallel
> >>> workers. What do the numeric column labels mean across the top
> >>> (2,4,8,16...) -- are they related to "matches"? And if so, what does
> >>> that mean?
> >>>
> >>
> >> That's the number of distinct values matched by the query, which should
> >> be an approximation of the number of matching rows. The number of
> >> distinct values in the data set differs by data set, but for 1M rows
> >> it's roughly like this:
> >>
> >> uniform: 10k
> >> linear: 10k
> >> cyclic: 100
> >>
> >> So for example matches=128 means ~1% of rows for uniform/linear, and
> >> 100% for cyclic data sets.
> >
> > Ah, thank you for the explanation. I also looked at your script after
> > having sent this email and saw that it is clear in your script what
> > "matches" is.
> >
> >> As for the possible cause, I think it's clear most of the difference
> >> comes from the last patch that actually switches bitmap heap scan to the
> >> streaming read API. That's mostly expected/understandable, although we
> >> probably need to look into the regressions or cases with e_i_c=0.
> >
> > Right, I'm mostly surprised about the regressions for patches 0001-0012.
> >
> > Re eic 0: Thomas Munro and I chatted off-list, and you bring up a
> > great point about eic 0. In old bitmapheapscan code eic 0 basically
> > disabled prefetching but with the streaming read API, it will still
> > issue fadvises when eic is 0. That is an easy one line fix. Thomas
> > prefers to fix it by always avoiding an fadvise for the last buffer in
> > a range before issuing a read (since we are about to read it anyway,
> > best not fadvise it too). This will fix eic 0 and also cut one system
> > call from each invocation of the streaming read machinery.
> >
> >> To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
> >> individual patches. I can try doing that tomorrow. It'll have to be a
> >> limited set of tests, to reduce the time, but might tell us whether it's
> >> due to a single patch or multiple patches.
> >
> > Yes, tomorrow I planned to start trying to repro some of the "red"
> > cases myself. Any one of the commits could cause a slight regression
> > but a 3.5x regression is quite surprising, so I might focus on trying
> > to repro that locally and then narrow down which patch causes it.
> >
> > For the non-cached regressions, perhaps the commit to use the correct
> > recheck flag (0004) when prefetching could be the culprit. And for the
> > cached regressions, my money is on the commit which changes the whole
> > control flow of BitmapHeapNext() and the next_block() and next_tuple()
> > functions (0010).
> >
>
> I do have some partial results, comparing the patches. I only ran one of
> the more affected workloads (cyclic) on the xeon, attached is a PDF
> comparing master and the 0001-0014 patches. The percentages are timing
> vs. the preceding patch (green - faster, red - slower).
Just confirming: the results are for uncached?
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-02-29 22:44 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 00:29 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 2 replies; 248+ messages in thread
From: Tomas Vondra @ 2024-02-29 22:44 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 2/29/24 22:19, Melanie Plageman wrote:
> On Thu, Feb 29, 2024 at 7:54 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>>
>>
>> On 2/29/24 00:40, Melanie Plageman wrote:
>>> On Wed, Feb 28, 2024 at 6:17 PM Tomas Vondra
>>> <tomas.vondra@enterprisedb.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2/28/24 21:06, Melanie Plageman wrote:
>>>>> On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
>>>>> <tomas.vondra@enterprisedb.com> wrote:
>>>>>>
>>>>>> On 2/28/24 15:56, Tomas Vondra wrote:
>>>>>>>> ...
>>>>>>>
>>>>>>> Sure, I can do that. It'll take a couple hours to get the results, I'll
>>>>>>> share them when I have them.
>>>>>>>
>>>>>>
>>>>>> Here are the results with only patches 0001 - 0012 applied (i.e. without
>>>>>> the patch introducing the streaming read API, and the patch switching
>>>>>> the bitmap heap scan to use it).
>>>>>>
>>>>>> The changes in performance don't disappear entirely, but the scale is
>>>>>> certainly much smaller - both in the complete results for all runs, and
>>>>>> for the "optimal" runs that would actually pick bitmapscan.
>>>>>
>>>>> Hmm. I'm trying to think how my refactor could have had this impact.
>>>>> It seems like all the most notable regressions are with 4 parallel
>>>>> workers. What do the numeric column labels mean across the top
>>>>> (2,4,8,16...) -- are they related to "matches"? And if so, what does
>>>>> that mean?
>>>>>
>>>>
>>>> That's the number of distinct values matched by the query, which should
>>>> be an approximation of the number of matching rows. The number of
>>>> distinct values in the data set differs by data set, but for 1M rows
>>>> it's roughly like this:
>>>>
>>>> uniform: 10k
>>>> linear: 10k
>>>> cyclic: 100
>>>>
>>>> So for example matches=128 means ~1% of rows for uniform/linear, and
>>>> 100% for cyclic data sets.
>>>
>>> Ah, thank you for the explanation. I also looked at your script after
>>> having sent this email and saw that it is clear in your script what
>>> "matches" is.
>>>
>>>> As for the possible cause, I think it's clear most of the difference
>>>> comes from the last patch that actually switches bitmap heap scan to the
>>>> streaming read API. That's mostly expected/understandable, although we
>>>> probably need to look into the regressions or cases with e_i_c=0.
>>>
>>> Right, I'm mostly surprised about the regressions for patches 0001-0012.
>>>
>>> Re eic 0: Thomas Munro and I chatted off-list, and you bring up a
>>> great point about eic 0. In old bitmapheapscan code eic 0 basically
>>> disabled prefetching but with the streaming read API, it will still
>>> issue fadvises when eic is 0. That is an easy one line fix. Thomas
>>> prefers to fix it by always avoiding an fadvise for the last buffer in
>>> a range before issuing a read (since we are about to read it anyway,
>>> best not fadvise it too). This will fix eic 0 and also cut one system
>>> call from each invocation of the streaming read machinery.
>>>
>>>> To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
>>>> individual patches. I can try doing that tomorrow. It'll have to be a
>>>> limited set of tests, to reduce the time, but might tell us whether it's
>>>> due to a single patch or multiple patches.
>>>
>>> Yes, tomorrow I planned to start trying to repro some of the "red"
>>> cases myself. Any one of the commits could cause a slight regression
>>> but a 3.5x regression is quite surprising, so I might focus on trying
>>> to repro that locally and then narrow down which patch causes it.
>>>
>>> For the non-cached regressions, perhaps the commit to use the correct
>>> recheck flag (0004) when prefetching could be the culprit. And for the
>>> cached regressions, my money is on the commit which changes the whole
>>> control flow of BitmapHeapNext() and the next_block() and next_tuple()
>>> functions (0010).
>>>
>>
>> I do have some partial results, comparing the patches. I only ran one of
>> the more affected workloads (cyclic) on the xeon, attached is a PDF
>> comparing master and the 0001-0014 patches. The percentages are timing
>> vs. the preceding patch (green - faster, red - slower).
>
> Just confirming: the results are for uncached?
>
Yes, cyclic data set, uncached case. I picked this because it seemed
like one of the most affected cases. Do you want me to test some other
cases too?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-02-29 23:44 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-02-29 23:44 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 2/29/24 23:44, Tomas Vondra wrote:
>
> ...
>
>>>
>>> I do have some partial results, comparing the patches. I only ran one of
>>> the more affected workloads (cyclic) on the xeon, attached is a PDF
>>> comparing master and the 0001-0014 patches. The percentages are timing
>>> vs. the preceding patch (green - faster, red - slower).
>>
>> Just confirming: the results are for uncached?
>>
>
> Yes, cyclic data set, uncached case. I picked this because it seemed
> like one of the most affected cases. Do you want me to test some other
> cases too?
>
BTW I decided to look at the data from a slightly different angle and
compare the behavior with increasing effective_io_concurrency. Attached
are charts for three "uncached" cases:
* uniform, work_mem=4MB, workers_per_gather=0
* linear-fuzz, work_mem=4MB, workers_per_gather=0
* uniform, work_mem=4MB, workers_per_gather=4
Each page has charts for master and patched build (with all patches). I
think there's a pretty obvious difference in how increasing e_i_c
affects the two builds:
1) On master there's clear difference between eic=0 and eic=1 cases, but
on the patched build there's literally no difference - for example the
"uniform" distribution is clearly not great for prefetching, but eic=0
regresses to eic=1 poor behavior).
Note: This is where the the "red bands" in the charts come from.
2) For some reason, the prefetching with eic>1 perform much better with
the patches, except for with very low selectivity values (close to 0%).
Not sure why this is happening - either the overhead is much lower
(which would matter on these "adversarial" data distribution, but how
could that be when fadvise is not free), or it ends up not doing any
prefetching (but then what about (1)?).
3) I'm not sure about the linear-fuzz case, the only explanation I have
we're able to skip almost all of the prefetches (and read-ahead likely
works pretty well here).
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] prefetch-charts.pdf (389.3K, ../../35bdb7db-7412-4d18-9b06-fea5fcea37bc@enterprisedb.com/2-prefetch-charts.pdf)
download
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-01 01:18 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-01 01:18 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Thu, Feb 29, 2024 at 6:44 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 2/29/24 23:44, Tomas Vondra wrote:
> >
> > ...
> >
> >>>
> >>> I do have some partial results, comparing the patches. I only ran one of
> >>> the more affected workloads (cyclic) on the xeon, attached is a PDF
> >>> comparing master and the 0001-0014 patches. The percentages are timing
> >>> vs. the preceding patch (green - faster, red - slower).
> >>
> >> Just confirming: the results are for uncached?
> >>
> >
> > Yes, cyclic data set, uncached case. I picked this because it seemed
> > like one of the most affected cases. Do you want me to test some other
> > cases too?
> >
>
> BTW I decided to look at the data from a slightly different angle and
> compare the behavior with increasing effective_io_concurrency. Attached
> are charts for three "uncached" cases:
>
> * uniform, work_mem=4MB, workers_per_gather=0
> * linear-fuzz, work_mem=4MB, workers_per_gather=0
> * uniform, work_mem=4MB, workers_per_gather=4
>
> Each page has charts for master and patched build (with all patches). I
> think there's a pretty obvious difference in how increasing e_i_c
> affects the two builds:
Wow! These visualizations make it exceptionally clear. I want to go to
the Vondra school of data visualizations for performance results!
> 1) On master there's clear difference between eic=0 and eic=1 cases, but
> on the patched build there's literally no difference - for example the
> "uniform" distribution is clearly not great for prefetching, but eic=0
> regresses to eic=1 poor behavior).
Yes, so eic=0 and eic=1 are identical with the streaming read API.
That is, eic 0 does not disable prefetching. Thomas is going to update
the streaming read API to avoid issuing an fadvise for the last block
in a range before issuing a read -- which would mean no prefetching
with eic 0 and eic 1. Not doing prefetching with eic 1 actually seems
like the right behavior -- which would be different than what master
is doing, right?
Hopefully this fixes the clear difference between master and the
patched version at eic 0.
> 2) For some reason, the prefetching with eic>1 perform much better with
> the patches, except for with very low selectivity values (close to 0%).
> Not sure why this is happening - either the overhead is much lower
> (which would matter on these "adversarial" data distribution, but how
> could that be when fadvise is not free), or it ends up not doing any
> prefetching (but then what about (1)?).
For the uniform with four parallel workers, eic == 0 being worse than
master makes sense for the above reason. But I'm not totally sure why
eic == 1 would be worse with the patch than with master. Both are
doing a (somewhat useless) prefetch.
With very low selectivity, you are less likely to get readahead
(right?) and similarly less likely to be able to build up > 8kB IOs --
which is one of the main value propositions of the streaming read
code. I imagine that this larger read benefit is part of why the
performance is better at higher selectivities with the patch. This
might be a silly experiment, but we could try decreasing
MAX_BUFFERS_PER_TRANSFER on the patched version and see if the
performance gains go away.
> 3) I'm not sure about the linear-fuzz case, the only explanation I have
> we're able to skip almost all of the prefetches (and read-ahead likely
> works pretty well here).
I started looking at the data generated by linear-fuzz to understand
exactly what effect the fuzz was having but haven't had time to really
understand the characteristics of this dataset. In the original
results, I thought uncached linear-fuzz and linear had similar results
(performance improvement from master). What do you expect with linear
vs linear-fuzz?
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-01 14:05 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-01 14:05 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/1/24 02:18, Melanie Plageman wrote:
> On Thu, Feb 29, 2024 at 6:44 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 2/29/24 23:44, Tomas Vondra wrote:
>>>
>>> ...
>>>
>>>>>
>>>>> I do have some partial results, comparing the patches. I only ran one of
>>>>> the more affected workloads (cyclic) on the xeon, attached is a PDF
>>>>> comparing master and the 0001-0014 patches. The percentages are timing
>>>>> vs. the preceding patch (green - faster, red - slower).
>>>>
>>>> Just confirming: the results are for uncached?
>>>>
>>>
>>> Yes, cyclic data set, uncached case. I picked this because it seemed
>>> like one of the most affected cases. Do you want me to test some other
>>> cases too?
>>>
>>
>> BTW I decided to look at the data from a slightly different angle and
>> compare the behavior with increasing effective_io_concurrency. Attached
>> are charts for three "uncached" cases:
>>
>> * uniform, work_mem=4MB, workers_per_gather=0
>> * linear-fuzz, work_mem=4MB, workers_per_gather=0
>> * uniform, work_mem=4MB, workers_per_gather=4
>>
>> Each page has charts for master and patched build (with all patches). I
>> think there's a pretty obvious difference in how increasing e_i_c
>> affects the two builds:
>
> Wow! These visualizations make it exceptionally clear. I want to go to
> the Vondra school of data visualizations for performance results!
>
Welcome to my lecture on how to visualize data. The process has about
four simple steps:
1) collect data for a lot of potentially interesting cases
2) load them into excel / google sheets / ...
3) slice and dice them into charts that you understand / can explain
4) every now and then there's something you can't understand / explain
Thank you for attending my lecture ;-) No homework today.
>> 1) On master there's clear difference between eic=0 and eic=1 cases, but
>> on the patched build there's literally no difference - for example the
>> "uniform" distribution is clearly not great for prefetching, but eic=0
>> regresses to eic=1 poor behavior).
>
> Yes, so eic=0 and eic=1 are identical with the streaming read API.
> That is, eic 0 does not disable prefetching. Thomas is going to update
> the streaming read API to avoid issuing an fadvise for the last block
> in a range before issuing a read -- which would mean no prefetching
> with eic 0 and eic 1. Not doing prefetching with eic 1 actually seems
> like the right behavior -- which would be different than what master
> is doing, right?
>
I don't think we should stop doing prefetching for eic=1, or at least
not based just on these charts. I suspect these "uniform" charts are not
a great example for the prefetching, because it's about distribution of
individual rows, and even a small fraction of rows may match most of the
pages. It's great for finding strange behaviors / corner cases, but
probably not a sufficient reason to change the default.
I think it makes sense to issue a prefetch one page ahead, before
reading/processing the preceding one, and it's fairly conservative
setting, and I assume the default was chosen for a reason / after
discussion.
My suggestion would be to keep the master behavior unless not practical,
and then maybe discuss changing the details later. The patch is already
complicated enough, better to leave that discussion for later.
> Hopefully this fixes the clear difference between master and the
> patched version at eic 0.
>
>> 2) For some reason, the prefetching with eic>1 perform much better with
>> the patches, except for with very low selectivity values (close to 0%).
>> Not sure why this is happening - either the overhead is much lower
>> (which would matter on these "adversarial" data distribution, but how
>> could that be when fadvise is not free), or it ends up not doing any
>> prefetching (but then what about (1)?).
>
> For the uniform with four parallel workers, eic == 0 being worse than
> master makes sense for the above reason. But I'm not totally sure why
> eic == 1 would be worse with the patch than with master. Both are
> doing a (somewhat useless) prefetch.
>
Right.
> With very low selectivity, you are less likely to get readahead
> (right?) and similarly less likely to be able to build up > 8kB IOs --
> which is one of the main value propositions of the streaming read
> code. I imagine that this larger read benefit is part of why the
> performance is better at higher selectivities with the patch. This
> might be a silly experiment, but we could try decreasing
> MAX_BUFFERS_PER_TRANSFER on the patched version and see if the
> performance gains go away.
>
Sure, I can do that. Do you have any particular suggestion what value to
use for MAX_BUFFERS_PER_TRANSFER?
I'll also try to add a better version of uniform, where the selectivity
matches more closely to pages, not rows.
>> 3) I'm not sure about the linear-fuzz case, the only explanation I have
>> we're able to skip almost all of the prefetches (and read-ahead likely
>> works pretty well here).
>
> I started looking at the data generated by linear-fuzz to understand
> exactly what effect the fuzz was having but haven't had time to really
> understand the characteristics of this dataset. In the original
> results, I thought uncached linear-fuzz and linear had similar results
> (performance improvement from master). What do you expect with linear
> vs linear-fuzz?
>
I don't know, TBH. My intent was to have a data set with correlated
data, either perfectly (linear) or with some noise (linear-fuzz). But
it's not like I spent too much thinking about it. It's more a case of
throwing stuff at the wall, seeing what sticks.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-01 16:51 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 23:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 2 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-03-01 16:51 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Fri, Mar 1, 2024 at 9:05 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 3/1/24 02:18, Melanie Plageman wrote:
> > On Thu, Feb 29, 2024 at 6:44 PM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >> On 2/29/24 23:44, Tomas Vondra wrote:
> >> 1) On master there's clear difference between eic=0 and eic=1 cases, but
> >> on the patched build there's literally no difference - for example the
> >> "uniform" distribution is clearly not great for prefetching, but eic=0
> >> regresses to eic=1 poor behavior).
> >
> > Yes, so eic=0 and eic=1 are identical with the streaming read API.
> > That is, eic 0 does not disable prefetching. Thomas is going to update
> > the streaming read API to avoid issuing an fadvise for the last block
> > in a range before issuing a read -- which would mean no prefetching
> > with eic 0 and eic 1. Not doing prefetching with eic 1 actually seems
> > like the right behavior -- which would be different than what master
> > is doing, right?
>
> I don't think we should stop doing prefetching for eic=1, or at least
> not based just on these charts. I suspect these "uniform" charts are not
> a great example for the prefetching, because it's about distribution of
> individual rows, and even a small fraction of rows may match most of the
> pages. It's great for finding strange behaviors / corner cases, but
> probably not a sufficient reason to change the default.
Yes, I would like to see results from a data set where selectivity is
more correlated to pages/heap fetches. But, I'm not sure I see how
that is related to prefetching when eic = 1.
> I think it makes sense to issue a prefetch one page ahead, before
> reading/processing the preceding one, and it's fairly conservative
> setting, and I assume the default was chosen for a reason / after
> discussion.
Yes, I suppose the overhead of an fadvise does not compare to the IO
latency of synchronously reading that block. Actually, I bet the
regression I saw by accidentally moving BitmapAdjustPrefetchIterator()
after table_scan_bitmap_next_block() would be similar to the
regression introduced by making eic = 1 not prefetch.
When you think about IO concurrency = 1, it doesn't imply prefetching
to me. But, I think we want to do the right thing and have parity with
master.
> My suggestion would be to keep the master behavior unless not practical,
> and then maybe discuss changing the details later. The patch is already
> complicated enough, better to leave that discussion for later.
Agreed. Speaking of which, we need to add back use of tablespace IO
concurrency for the streaming read API (which is used by
BitmapHeapScan in master).
> > With very low selectivity, you are less likely to get readahead
> > (right?) and similarly less likely to be able to build up > 8kB IOs --
> > which is one of the main value propositions of the streaming read
> > code. I imagine that this larger read benefit is part of why the
> > performance is better at higher selectivities with the patch. This
> > might be a silly experiment, but we could try decreasing
> > MAX_BUFFERS_PER_TRANSFER on the patched version and see if the
> > performance gains go away.
>
> Sure, I can do that. Do you have any particular suggestion what value to
> use for MAX_BUFFERS_PER_TRANSFER?
I think setting it to 1 would be the same as always master -- doing
only 8kB reads. The only thing about that is that I imagine the other
streaming read code has some overhead which might end up being a
regression on balance even with the prefetching if we aren't actually
using the ranges/vectored capabilities of the streaming read
interface. Maybe if you just run it for one of the very obvious
performance improvement cases? I can also try this locally.
> I'll also try to add a better version of uniform, where the selectivity
> matches more closely to pages, not rows.
This would be great.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-01 17:08 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
1 sibling, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-01 17:08 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/1/24 17:51, Melanie Plageman wrote:
> On Fri, Mar 1, 2024 at 9:05 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 3/1/24 02:18, Melanie Plageman wrote:
>>> On Thu, Feb 29, 2024 at 6:44 PM Tomas Vondra
>>> <tomas.vondra@enterprisedb.com> wrote:
>>>>
>>>> On 2/29/24 23:44, Tomas Vondra wrote:
>>>> 1) On master there's clear difference between eic=0 and eic=1 cases, but
>>>> on the patched build there's literally no difference - for example the
>>>> "uniform" distribution is clearly not great for prefetching, but eic=0
>>>> regresses to eic=1 poor behavior).
>>>
>>> Yes, so eic=0 and eic=1 are identical with the streaming read API.
>>> That is, eic 0 does not disable prefetching. Thomas is going to update
>>> the streaming read API to avoid issuing an fadvise for the last block
>>> in a range before issuing a read -- which would mean no prefetching
>>> with eic 0 and eic 1. Not doing prefetching with eic 1 actually seems
>>> like the right behavior -- which would be different than what master
>>> is doing, right?
>>
>> I don't think we should stop doing prefetching for eic=1, or at least
>> not based just on these charts. I suspect these "uniform" charts are not
>> a great example for the prefetching, because it's about distribution of
>> individual rows, and even a small fraction of rows may match most of the
>> pages. It's great for finding strange behaviors / corner cases, but
>> probably not a sufficient reason to change the default.
>
> Yes, I would like to see results from a data set where selectivity is
> more correlated to pages/heap fetches. But, I'm not sure I see how
> that is related to prefetching when eic = 1.
>
OK, I'll make that happen.
>> I think it makes sense to issue a prefetch one page ahead, before
>> reading/processing the preceding one, and it's fairly conservative
>> setting, and I assume the default was chosen for a reason / after
>> discussion.
>
> Yes, I suppose the overhead of an fadvise does not compare to the IO
> latency of synchronously reading that block. Actually, I bet the
> regression I saw by accidentally moving BitmapAdjustPrefetchIterator()
> after table_scan_bitmap_next_block() would be similar to the
> regression introduced by making eic = 1 not prefetch.
>
> When you think about IO concurrency = 1, it doesn't imply prefetching
> to me. But, I think we want to do the right thing and have parity with
> master.
>
Just to be sure we're on the same page regarding what eic=1 means,
consider a simple sequence of pages: A, B, C, D, E, ...
With the current "master" code, eic=1 means we'll issue a prefetch for B
and then read+process A. And then issue prefetch for C and read+process
B, and so on. It's always one page ahead.
Yes, if the page is already in memory, the fadvise is just overhead. It
may happen for various reasons (say, read-ahead). But it's just this one
case, I'd bet in other cases eic=1 would be a win.
>> My suggestion would be to keep the master behavior unless not practical,
>> and then maybe discuss changing the details later. The patch is already
>> complicated enough, better to leave that discussion for later.
>
> Agreed. Speaking of which, we need to add back use of tablespace IO
> concurrency for the streaming read API (which is used by
> BitmapHeapScan in master).
>
+1
>>> With very low selectivity, you are less likely to get readahead
>>> (right?) and similarly less likely to be able to build up > 8kB IOs --
>>> which is one of the main value propositions of the streaming read
>>> code. I imagine that this larger read benefit is part of why the
>>> performance is better at higher selectivities with the patch. This
>>> might be a silly experiment, but we could try decreasing
>>> MAX_BUFFERS_PER_TRANSFER on the patched version and see if the
>>> performance gains go away.
>>
>> Sure, I can do that. Do you have any particular suggestion what value to
>> use for MAX_BUFFERS_PER_TRANSFER?
>
> I think setting it to 1 would be the same as always master -- doing
> only 8kB reads. The only thing about that is that I imagine the other
> streaming read code has some overhead which might end up being a
> regression on balance even with the prefetching if we aren't actually
> using the ranges/vectored capabilities of the streaming read
> interface. Maybe if you just run it for one of the very obvious
> performance improvement cases? I can also try this locally.
>
OK, I'll try with 1, and then we can adjust.
>> I'll also try to add a better version of uniform, where the selectivity
>> matches more closely to pages, not rows.
>
> This would be great.
>
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-02 15:05 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:28 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-02 15:05 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/1/24 18:08, Tomas Vondra wrote:
>
> On 3/1/24 17:51, Melanie Plageman wrote:
>> On Fri, Mar 1, 2024 at 9:05 AM Tomas Vondra
>> <tomas.vondra@enterprisedb.com> wrote:
>>>
>>> On 3/1/24 02:18, Melanie Plageman wrote:
>>>> On Thu, Feb 29, 2024 at 6:44 PM Tomas Vondra
>>>> <tomas.vondra@enterprisedb.com> wrote:
>>>>>
>>>>> On 2/29/24 23:44, Tomas Vondra wrote:
>>>>> 1) On master there's clear difference between eic=0 and eic=1 cases, but
>>>>> on the patched build there's literally no difference - for example the
>>>>> "uniform" distribution is clearly not great for prefetching, but eic=0
>>>>> regresses to eic=1 poor behavior).
>>>>
>>>> Yes, so eic=0 and eic=1 are identical with the streaming read API.
>>>> That is, eic 0 does not disable prefetching. Thomas is going to update
>>>> the streaming read API to avoid issuing an fadvise for the last block
>>>> in a range before issuing a read -- which would mean no prefetching
>>>> with eic 0 and eic 1. Not doing prefetching with eic 1 actually seems
>>>> like the right behavior -- which would be different than what master
>>>> is doing, right?
>>>
>>> I don't think we should stop doing prefetching for eic=1, or at least
>>> not based just on these charts. I suspect these "uniform" charts are not
>>> a great example for the prefetching, because it's about distribution of
>>> individual rows, and even a small fraction of rows may match most of the
>>> pages. It's great for finding strange behaviors / corner cases, but
>>> probably not a sufficient reason to change the default.
>>
>> Yes, I would like to see results from a data set where selectivity is
>> more correlated to pages/heap fetches. But, I'm not sure I see how
>> that is related to prefetching when eic = 1.
>>
>
> OK, I'll make that happen.
>
Here's a PDF with charts for a dataset where the row selectivity is more
correlated to selectivity of pages. I'm attaching the updated script,
with the SQL generating the data set. But the short story is all rows on
a single page have the same random value, so the selectivity of rows and
pages should be the same.
The first page has results for the original "uniform", the second page
is the new "uniform-pages" data set. There are 4 charts, for
master/patched and 0/4 parallel workers. Overall the behavior is the
same, but for the "uniform-pages" it's much more gradual (with respect
to row selectivity). I think that's expected.
As for how this is related to eic=1 - I think my point was that these
are "adversary" data sets, most likely to show regressions. This applies
especially to the "uniform" data set, because as the row selectivity
grows, it's more and more likely it's right after to the current one,
and so a read-ahead would likely do the trick.
Also, this is forcing a bitmap scan plan - it's possible many of these
cases would use some other scan type, making the regression somewhat
irrelevant. Not entirely, because we make planning mistakes and for
robustness reasons it's good to keep the regression small.
But that's just how I think about it now. I don't think I have some
grand theory that'd dictate we have to do prefetching for eic=1.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] uniform-pages.pdf (493.7K, ../../186bcba1-a0e1-4871-8ed2-0d301901d0ba@enterprisedb.com/2-uniform-pages.pdf)
download
[application/x-shellscript] run.sh (7.8K, ../../186bcba1-a0e1-4871-8ed2-0d301901d0ba@enterprisedb.com/3-run.sh)
download
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-02 22:28 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:41 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-02 22:28 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sat, Mar 2, 2024 at 10:05 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> Here's a PDF with charts for a dataset where the row selectivity is more
> correlated to selectivity of pages. I'm attaching the updated script,
> with the SQL generating the data set. But the short story is all rows on
> a single page have the same random value, so the selectivity of rows and
> pages should be the same.
>
> The first page has results for the original "uniform", the second page
> is the new "uniform-pages" data set. There are 4 charts, for
> master/patched and 0/4 parallel workers. Overall the behavior is the
> same, but for the "uniform-pages" it's much more gradual (with respect
> to row selectivity). I think that's expected.
Cool! Thanks for doing this. I have convinced myself that Thomas'
forthcoming patch which will eliminate prefetching with eic = 0 will
fix the eic 0 blue line regressions. The eic = 1 with four parallel
workers is more confusing. And it seems more noticeably bad with your
randomized-pages dataset.
Regarding your earlier question:
> Just to be sure we're on the same page regarding what eic=1 means,
> consider a simple sequence of pages: A, B, C, D, E, ...
>
> With the current "master" code, eic=1 means we'll issue a prefetch for B
> and then read+process A. And then issue prefetch for C and read+process
> B, and so on. It's always one page ahead.
Yes, that is what I mean for eic = 1
> As for how this is related to eic=1 - I think my point was that these
> are "adversary" data sets, most likely to show regressions. This applies
> especially to the "uniform" data set, because as the row selectivity
> grows, it's more and more likely it's right after to the current one,
> and so a read-ahead would likely do the trick.
No, I think you're right that eic=1 should prefetch. As you say, with
high selectivity, a bitmap plan is likely not the best one anyway, so
not prefetching in order to preserve the performance of those cases
seems silly.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:28 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-02 22:41 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:52 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 22:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
0 siblings, 2 replies; 248+ messages in thread
From: Tomas Vondra @ 2024-03-02 22:41 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/2/24 23:28, Melanie Plageman wrote:
> On Sat, Mar 2, 2024 at 10:05 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> Here's a PDF with charts for a dataset where the row selectivity is more
>> correlated to selectivity of pages. I'm attaching the updated script,
>> with the SQL generating the data set. But the short story is all rows on
>> a single page have the same random value, so the selectivity of rows and
>> pages should be the same.
>>
>> The first page has results for the original "uniform", the second page
>> is the new "uniform-pages" data set. There are 4 charts, for
>> master/patched and 0/4 parallel workers. Overall the behavior is the
>> same, but for the "uniform-pages" it's much more gradual (with respect
>> to row selectivity). I think that's expected.
>
> Cool! Thanks for doing this. I have convinced myself that Thomas'
> forthcoming patch which will eliminate prefetching with eic = 0 will
> fix the eic 0 blue line regressions. The eic = 1 with four parallel
> workers is more confusing. And it seems more noticeably bad with your
> randomized-pages dataset.
>
> Regarding your earlier question:
>
>> Just to be sure we're on the same page regarding what eic=1 means,
>> consider a simple sequence of pages: A, B, C, D, E, ...
>>
>> With the current "master" code, eic=1 means we'll issue a prefetch for B
>> and then read+process A. And then issue prefetch for C and read+process
>> B, and so on. It's always one page ahead.
>
> Yes, that is what I mean for eic = 1
>
>> As for how this is related to eic=1 - I think my point was that these
>> are "adversary" data sets, most likely to show regressions. This applies
>> especially to the "uniform" data set, because as the row selectivity
>> grows, it's more and more likely it's right after to the current one,
>> and so a read-ahead would likely do the trick.
>
> No, I think you're right that eic=1 should prefetch. As you say, with
> high selectivity, a bitmap plan is likely not the best one anyway, so
> not prefetching in order to preserve the performance of those cases
> seems silly.
>
I was just trying to respond do this from an earlier message:
> Yes, I would like to see results from a data set where selectivity is
> more correlated to pages/heap fetches. But, I'm not sure I see how
> that is related to prefetching when eic = 1.
And in that same message you also said "Not doing prefetching with eic 1
actually seems like the right behavior". Hence my argument we should not
stop prefetching for eic=1.
But maybe I'm confused - it seems agree eic=1 should prefetch, and that
uniform data set may not be a good argument against that.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:28 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:41 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-02 22:52 ` Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 0 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-03-02 22:52 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sat, Mar 2, 2024 at 5:41 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
>
>
> On 3/2/24 23:28, Melanie Plageman wrote:
> > On Sat, Mar 2, 2024 at 10:05 AM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >> Here's a PDF with charts for a dataset where the row selectivity is more
> >> correlated to selectivity of pages. I'm attaching the updated script,
> >> with the SQL generating the data set. But the short story is all rows on
> >> a single page have the same random value, so the selectivity of rows and
> >> pages should be the same.
> >>
> >> The first page has results for the original "uniform", the second page
> >> is the new "uniform-pages" data set. There are 4 charts, for
> >> master/patched and 0/4 parallel workers. Overall the behavior is the
> >> same, but for the "uniform-pages" it's much more gradual (with respect
> >> to row selectivity). I think that's expected.
> >
> > Cool! Thanks for doing this. I have convinced myself that Thomas'
> > forthcoming patch which will eliminate prefetching with eic = 0 will
> > fix the eic 0 blue line regressions. The eic = 1 with four parallel
> > workers is more confusing. And it seems more noticeably bad with your
> > randomized-pages dataset.
> >
> > Regarding your earlier question:
> >
> >> Just to be sure we're on the same page regarding what eic=1 means,
> >> consider a simple sequence of pages: A, B, C, D, E, ...
> >>
> >> With the current "master" code, eic=1 means we'll issue a prefetch for B
> >> and then read+process A. And then issue prefetch for C and read+process
> >> B, and so on. It's always one page ahead.
> >
> > Yes, that is what I mean for eic = 1
> >
> >> As for how this is related to eic=1 - I think my point was that these
> >> are "adversary" data sets, most likely to show regressions. This applies
> >> especially to the "uniform" data set, because as the row selectivity
> >> grows, it's more and more likely it's right after to the current one,
> >> and so a read-ahead would likely do the trick.
> >
> > No, I think you're right that eic=1 should prefetch. As you say, with
> > high selectivity, a bitmap plan is likely not the best one anyway, so
> > not prefetching in order to preserve the performance of those cases
> > seems silly.
> >
>
> I was just trying to respond do this from an earlier message:
>
> > Yes, I would like to see results from a data set where selectivity is
> > more correlated to pages/heap fetches. But, I'm not sure I see how
> > that is related to prefetching when eic = 1.
>
> And in that same message you also said "Not doing prefetching with eic 1
> actually seems like the right behavior". Hence my argument we should not
> stop prefetching for eic=1.
>
> But maybe I'm confused - it seems agree eic=1 should prefetch, and that
> uniform data set may not be a good argument against that.
Yep, we agree. I was being confusing and wrong :) I just wanted to
make sure the thread had a clear consensus that, yes, it is the right
thing to do to prefetch blocks for bitmap heap scans when
effective_io_concurrency = 1.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:28 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:41 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-13 22:38 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-14 14:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
1 sibling, 1 reply; 248+ messages in thread
From: Thomas Munro @ 2024-03-13 22:38 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sun, Mar 3, 2024 at 11:41 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
> On 3/2/24 23:28, Melanie Plageman wrote:
> > On Sat, Mar 2, 2024 at 10:05 AM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >> With the current "master" code, eic=1 means we'll issue a prefetch for B
> >> and then read+process A. And then issue prefetch for C and read+process
> >> B, and so on. It's always one page ahead.
> >
> > Yes, that is what I mean for eic = 1
I spent quite a few days thinking about the meaning of eic=0 and eic=1
for streaming_read.c v7[1], to make it agree with the above and with
master. Here's why I was confused:
Both eic=0 and eic=1 are expected to generate at most 1 physical I/O
at a time, or I/O queue depth 1 if you want to put it that way. But
this isn't just about concurrency of I/O, it's also about computation.
Duh.
eic=0 means that the I/O is not concurrent with executor computation.
So, to annotate an excerpt from [1]'s random.txt, we have:
effective_io_concurrency = 0, range size = 1
unpatched patched
==============================================================================
pread(43,...,8192,0x58000) = 8192 pread(82,...,8192,0x58000) = 8192
*** executor now has page at 0x58000 to work on ***
pread(43,...,8192,0xb0000) = 8192 pread(82,...,8192,0xb0000) = 8192
*** executor now has page at 0xb0000 to work on ***
eic=1 means that a single I/O is started and then control is returned
to the executor code to do useful work concurrently with the
background read that we assume is happening:
effective_io_concurrency = 1, range size = 1
unpatched patched
==============================================================================
pread(43,...,8192,0x58000) = 8192 pread(82,...,8192,0x58000) = 8192
posix_fadvise(43,0xb0000,0x2000,...) posix_fadvise(82,0xb0000,0x2000,...)
*** executor now has page at 0x58000 to work on ***
pread(43,...,8192,0xb0000) = 8192 pread(82,...,8192,0xb0000) = 8192
posix_fadvise(43,0x108000,0x2000,...) posix_fadvise(82,0x108000,0x2000,...)
*** executor now has page at 0xb0000 to work on ***
pread(43,...,8192,0x108000) = 8192 pread(82,...,8192,0x108000) = 8192
posix_fadvise(43,0x160000,0x2000,...) posix_fadvise(82,0x160000,0x2000,...)
In other words, 'concurrency' doesn't mean 'number of I/Os running
concurrently with each other', it means 'number of I/Os running
concurrently with computation', and when you put it that way, 0 and 1
are different.
Note that the first read is a bit special: by the time the consumer is
ready to pull a buffer out of the stream when we don't have a buffer
ready yet, it is too late to issue useful advice, so we don't bother.
FWIW I think even in the AIO future we would have a synchronous read
in that specific place, at least when using io_method=worker, because
it would be stupid to ask another process to read a block for us that
we want right now and then wait for it wake us up when it's done.
Note that even when we aren't issuing any advice because eic=0 or
because we detected sequential access and we believe the kernel can do
a better job than us, we still 'look ahead' (= call the callback to
see which block numbers are coming down the pipe), but only as far as
we need to coalesce neighbouring blocks. (I deliberately avoid using
the word "prefetch" except in very general discussions because it
means different things to different layers of the code, hence talk of
"look ahead" and "advice".) That's how we get this change:
effective_io_concurrency = 0, range size = 4
unpatched patched
==============================================================================
pread(43,...,8192,0x58000) = 8192 pread(82,...,8192,0x58000) = 8192
pread(43,...,8192,0x5a000) = 8192 preadv(82,...,2,0x5a000) = 16384
pread(43,...,8192,0x5c000) = 8192 pread(82,...,8192,0x5e000) = 8192
pread(43,...,8192,0x5e000) = 8192 preadv(82,...,4,0xb0000) = 32768
pread(43,...,8192,0xb0000) = 8192 preadv(82,...,4,0x108000) = 32768
pread(43,...,8192,0xb2000) = 8192 preadv(82,...,4,0x160000) = 32768
And then once we introduce eic > 0 to the picture with neighbouring
blocks that can be coalesced, "patched" starts to diverge even more
from "unpatched" because it tracks the number of wide I/Os in
progress, not the number of single blocks.
[1] https://www.postgresql.org/message-id/CA+hUKGLJi+c5jB3j6UvkgMYHky-qu+LPCsiNahUGSa5Z4DvyVA@mail.gmail...
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:28 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:41 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-13 22:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
@ 2024-03-14 14:17 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 20:58 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-14 14:17 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/13/24 23:38, Thomas Munro wrote:
> On Sun, Mar 3, 2024 at 11:41 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>> On 3/2/24 23:28, Melanie Plageman wrote:
>>> On Sat, Mar 2, 2024 at 10:05 AM Tomas Vondra
>>> <tomas.vondra@enterprisedb.com> wrote:
>>>> With the current "master" code, eic=1 means we'll issue a prefetch for B
>>>> and then read+process A. And then issue prefetch for C and read+process
>>>> B, and so on. It's always one page ahead.
>>>
>>> Yes, that is what I mean for eic = 1
>
> I spent quite a few days thinking about the meaning of eic=0 and eic=1
> for streaming_read.c v7[1], to make it agree with the above and with
> master. Here's why I was confused:
>
> Both eic=0 and eic=1 are expected to generate at most 1 physical I/O
> at a time, or I/O queue depth 1 if you want to put it that way. But
> this isn't just about concurrency of I/O, it's also about computation.
> Duh.
>
> eic=0 means that the I/O is not concurrent with executor computation.
> So, to annotate an excerpt from [1]'s random.txt, we have:
>
> effective_io_concurrency = 0, range size = 1
> unpatched patched
> ==============================================================================
> pread(43,...,8192,0x58000) = 8192 pread(82,...,8192,0x58000) = 8192
> *** executor now has page at 0x58000 to work on ***
> pread(43,...,8192,0xb0000) = 8192 pread(82,...,8192,0xb0000) = 8192
> *** executor now has page at 0xb0000 to work on ***
>
> eic=1 means that a single I/O is started and then control is returned
> to the executor code to do useful work concurrently with the
> background read that we assume is happening:
>
> effective_io_concurrency = 1, range size = 1
> unpatched patched
> ==============================================================================
> pread(43,...,8192,0x58000) = 8192 pread(82,...,8192,0x58000) = 8192
> posix_fadvise(43,0xb0000,0x2000,...) posix_fadvise(82,0xb0000,0x2000,...)
> *** executor now has page at 0x58000 to work on ***
> pread(43,...,8192,0xb0000) = 8192 pread(82,...,8192,0xb0000) = 8192
> posix_fadvise(43,0x108000,0x2000,...) posix_fadvise(82,0x108000,0x2000,...)
> *** executor now has page at 0xb0000 to work on ***
> pread(43,...,8192,0x108000) = 8192 pread(82,...,8192,0x108000) = 8192
> posix_fadvise(43,0x160000,0x2000,...) posix_fadvise(82,0x160000,0x2000,...)
>
> In other words, 'concurrency' doesn't mean 'number of I/Os running
> concurrently with each other', it means 'number of I/Os running
> concurrently with computation', and when you put it that way, 0 and 1
> are different.
>
Interesting. For some reason I thought with eic=1 we'd issue the fadvise
for page #2 before pread of page #1, so that there'd be 2 IO requests in
flight at the same time for a bit of time ... it'd give the fadvise more
time to actually get the data into page cache.
> Note that the first read is a bit special: by the time the consumer is
> ready to pull a buffer out of the stream when we don't have a buffer
> ready yet, it is too late to issue useful advice, so we don't bother.
> FWIW I think even in the AIO future we would have a synchronous read
> in that specific place, at least when using io_method=worker, because
> it would be stupid to ask another process to read a block for us that
> we want right now and then wait for it wake us up when it's done.
>
> Note that even when we aren't issuing any advice because eic=0 or
> because we detected sequential access and we believe the kernel can do
> a better job than us, we still 'look ahead' (= call the callback to
> see which block numbers are coming down the pipe), but only as far as
> we need to coalesce neighbouring blocks. (I deliberately avoid using
> the word "prefetch" except in very general discussions because it
> means different things to different layers of the code, hence talk of
> "look ahead" and "advice".) That's how we get this change:
>
> effective_io_concurrency = 0, range size = 4
> unpatched patched
> ==============================================================================
> pread(43,...,8192,0x58000) = 8192 pread(82,...,8192,0x58000) = 8192
> pread(43,...,8192,0x5a000) = 8192 preadv(82,...,2,0x5a000) = 16384
> pread(43,...,8192,0x5c000) = 8192 pread(82,...,8192,0x5e000) = 8192
> pread(43,...,8192,0x5e000) = 8192 preadv(82,...,4,0xb0000) = 32768
> pread(43,...,8192,0xb0000) = 8192 preadv(82,...,4,0x108000) = 32768
> pread(43,...,8192,0xb2000) = 8192 preadv(82,...,4,0x160000) = 32768
>
> And then once we introduce eic > 0 to the picture with neighbouring
> blocks that can be coalesced, "patched" starts to diverge even more
> from "unpatched" because it tracks the number of wide I/Os in
> progress, not the number of single blocks.
>
So, IIUC this means (1) the patched code is more aggressive wrt
prefetching (because we prefetch more data overall, because master would
prefetch N pages and patched prefetches N ranges, each of which may be
multiple pages. And (2) it's not easy to quantify how much more
aggressive it is, because it depends on how we happen to coalesce the
pages into ranges.
Do I understand this correctly?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:28 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:41 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-13 22:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
2024-03-14 14:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-14 20:58 ` Thomas Munro <thomas.munro@gmail.com>
0 siblings, 0 replies; 248+ messages in thread
From: Thomas Munro @ 2024-03-14 20:58 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Fri, Mar 15, 2024 at 3:18 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
> So, IIUC this means (1) the patched code is more aggressive wrt
> prefetching (because we prefetch more data overall, because master would
> prefetch N pages and patched prefetches N ranges, each of which may be
> multiple pages. And (2) it's not easy to quantify how much more
> aggressive it is, because it depends on how we happen to coalesce the
> pages into ranges.
>
> Do I understand this correctly?
Yes.
Parallelism must prevent coalescing here though. Any parallel aware
executor node that allocates block numbers to workers without trying
to preserve ranges will. That not only hides the opportunity to
coalesce reads, it also makes (globally) sequential scans look random
(ie locally they are more random), so that our logic to avoid issuing
advice for sequential scan won't work, and we'll inject extra useless
or harmful (?) fadvise calls. I don't know what to do about that yet,
but it seems like a subject for future research. Should we recognise
sequential scans with a window (like Linux does), instead of strictly
next-block detection (like some other OSes do)? Maybe a shared
streaming read that all workers pull blocks from, so it can see what's
going on? I think the latter would be strictly more like what the ad
hoc BHS prefetching code in master is doing, but I don't know if it'd
be over-engineering, or hard to do for some reason.
Another aspect of per-backend streaming reads in one parallel query
that don't know about each other is that they will all have their own
effective_io_concurrency limit. That is a version of a problem that
comes up again and again in parallel query, to be solved by the grand
unified resource control system of the future.
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-02 23:59 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-03 00:15 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-02 23:59 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/1/24 17:51, Melanie Plageman wrote:
> On Fri, Mar 1, 2024 at 9:05 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 3/1/24 02:18, Melanie Plageman wrote:
>>> On Thu, Feb 29, 2024 at 6:44 PM Tomas Vondra
>>> <tomas.vondra@enterprisedb.com> wrote:
>>>>
>>>> On 2/29/24 23:44, Tomas Vondra wrote:
>>>> 1) On master there's clear difference between eic=0 and eic=1 cases, but
>>>> on the patched build there's literally no difference - for example the
>>>> "uniform" distribution is clearly not great for prefetching, but eic=0
>>>> regresses to eic=1 poor behavior).
>>>
>>> Yes, so eic=0 and eic=1 are identical with the streaming read API.
>>> That is, eic 0 does not disable prefetching. Thomas is going to update
>>> the streaming read API to avoid issuing an fadvise for the last block
>>> in a range before issuing a read -- which would mean no prefetching
>>> with eic 0 and eic 1. Not doing prefetching with eic 1 actually seems
>>> like the right behavior -- which would be different than what master
>>> is doing, right?
>>
>> I don't think we should stop doing prefetching for eic=1, or at least
>> not based just on these charts. I suspect these "uniform" charts are not
>> a great example for the prefetching, because it's about distribution of
>> individual rows, and even a small fraction of rows may match most of the
>> pages. It's great for finding strange behaviors / corner cases, but
>> probably not a sufficient reason to change the default.
>
> Yes, I would like to see results from a data set where selectivity is
> more correlated to pages/heap fetches. But, I'm not sure I see how
> that is related to prefetching when eic = 1.
>
>> I think it makes sense to issue a prefetch one page ahead, before
>> reading/processing the preceding one, and it's fairly conservative
>> setting, and I assume the default was chosen for a reason / after
>> discussion.
>
> Yes, I suppose the overhead of an fadvise does not compare to the IO
> latency of synchronously reading that block. Actually, I bet the
> regression I saw by accidentally moving BitmapAdjustPrefetchIterator()
> after table_scan_bitmap_next_block() would be similar to the
> regression introduced by making eic = 1 not prefetch.
>
> When you think about IO concurrency = 1, it doesn't imply prefetching
> to me. But, I think we want to do the right thing and have parity with
> master.
>
>> My suggestion would be to keep the master behavior unless not practical,
>> and then maybe discuss changing the details later. The patch is already
>> complicated enough, better to leave that discussion for later.
>
> Agreed. Speaking of which, we need to add back use of tablespace IO
> concurrency for the streaming read API (which is used by
> BitmapHeapScan in master).
>
>>> With very low selectivity, you are less likely to get readahead
>>> (right?) and similarly less likely to be able to build up > 8kB IOs --
>>> which is one of the main value propositions of the streaming read
>>> code. I imagine that this larger read benefit is part of why the
>>> performance is better at higher selectivities with the patch. This
>>> might be a silly experiment, but we could try decreasing
>>> MAX_BUFFERS_PER_TRANSFER on the patched version and see if the
>>> performance gains go away.
>>
>> Sure, I can do that. Do you have any particular suggestion what value to
>> use for MAX_BUFFERS_PER_TRANSFER?
>
> I think setting it to 1 would be the same as always master -- doing
> only 8kB reads. The only thing about that is that I imagine the other
> streaming read code has some overhead which might end up being a
> regression on balance even with the prefetching if we aren't actually
> using the ranges/vectored capabilities of the streaming read
> interface. Maybe if you just run it for one of the very obvious
> performance improvement cases? I can also try this locally.
>
Here's some results from a build with
#define MAX_BUFFERS_PER_TRANSFER 1
There are three columns:
- master
- patched (original patches, with MAX_BUFFERS_PER_TRANSFER=128kB)
- patched-single (MAX_BUFFERS_PER_TRANSFER=8kB)
The color scales are always branch compared to master.
I think the expectation was that setting the transfer to 1 would make it
closer to master, reducing some of the regressions. But in practice the
effect is the opposite.
- In "cached" runs, this eliminates the small improvements (light
green), but leaves the regressions behind.
- In "uncached" runs, this exacerbates the regressions, particularly for
low selectivities (small values of matches).
I don't have a good intuition on why this would be happening :-(
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] cached-transfer-reduced.pdf (192.7K, ../../ed18f73d-216d-42f4-b08a-04a50a802d62@enterprisedb.com/2-cached-transfer-reduced.pdf)
download
[application/pdf] uncached-transfer-reduced.pdf (211.1K, ../../ed18f73d-216d-42f4-b08a-04a50a802d62@enterprisedb.com/3-uncached-transfer-reduced.pdf)
download
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 23:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-03 00:15 ` Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 0 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-03-03 00:15 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sat, Mar 2, 2024 at 6:59 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
>
>
> On 3/1/24 17:51, Melanie Plageman wrote:
> > On Fri, Mar 1, 2024 at 9:05 AM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >> On 3/1/24 02:18, Melanie Plageman wrote:
> >>> On Thu, Feb 29, 2024 at 6:44 PM Tomas Vondra
> >>> <tomas.vondra@enterprisedb.com> wrote:
> >>>>
> >>>> On 2/29/24 23:44, Tomas Vondra wrote:
> >>>> 1) On master there's clear difference between eic=0 and eic=1 cases, but
> >>>> on the patched build there's literally no difference - for example the
> >>>> "uniform" distribution is clearly not great for prefetching, but eic=0
> >>>> regresses to eic=1 poor behavior).
> >>>
> >>> Yes, so eic=0 and eic=1 are identical with the streaming read API.
> >>> That is, eic 0 does not disable prefetching. Thomas is going to update
> >>> the streaming read API to avoid issuing an fadvise for the last block
> >>> in a range before issuing a read -- which would mean no prefetching
> >>> with eic 0 and eic 1. Not doing prefetching with eic 1 actually seems
> >>> like the right behavior -- which would be different than what master
> >>> is doing, right?
> >>
> >> I don't think we should stop doing prefetching for eic=1, or at least
> >> not based just on these charts. I suspect these "uniform" charts are not
> >> a great example for the prefetching, because it's about distribution of
> >> individual rows, and even a small fraction of rows may match most of the
> >> pages. It's great for finding strange behaviors / corner cases, but
> >> probably not a sufficient reason to change the default.
> >
> > Yes, I would like to see results from a data set where selectivity is
> > more correlated to pages/heap fetches. But, I'm not sure I see how
> > that is related to prefetching when eic = 1.
> >
> >> I think it makes sense to issue a prefetch one page ahead, before
> >> reading/processing the preceding one, and it's fairly conservative
> >> setting, and I assume the default was chosen for a reason / after
> >> discussion.
> >
> > Yes, I suppose the overhead of an fadvise does not compare to the IO
> > latency of synchronously reading that block. Actually, I bet the
> > regression I saw by accidentally moving BitmapAdjustPrefetchIterator()
> > after table_scan_bitmap_next_block() would be similar to the
> > regression introduced by making eic = 1 not prefetch.
> >
> > When you think about IO concurrency = 1, it doesn't imply prefetching
> > to me. But, I think we want to do the right thing and have parity with
> > master.
> >
> >> My suggestion would be to keep the master behavior unless not practical,
> >> and then maybe discuss changing the details later. The patch is already
> >> complicated enough, better to leave that discussion for later.
> >
> > Agreed. Speaking of which, we need to add back use of tablespace IO
> > concurrency for the streaming read API (which is used by
> > BitmapHeapScan in master).
> >
> >>> With very low selectivity, you are less likely to get readahead
> >>> (right?) and similarly less likely to be able to build up > 8kB IOs --
> >>> which is one of the main value propositions of the streaming read
> >>> code. I imagine that this larger read benefit is part of why the
> >>> performance is better at higher selectivities with the patch. This
> >>> might be a silly experiment, but we could try decreasing
> >>> MAX_BUFFERS_PER_TRANSFER on the patched version and see if the
> >>> performance gains go away.
> >>
> >> Sure, I can do that. Do you have any particular suggestion what value to
> >> use for MAX_BUFFERS_PER_TRANSFER?
> >
> > I think setting it to 1 would be the same as always master -- doing
> > only 8kB reads. The only thing about that is that I imagine the other
> > streaming read code has some overhead which might end up being a
> > regression on balance even with the prefetching if we aren't actually
> > using the ranges/vectored capabilities of the streaming read
> > interface. Maybe if you just run it for one of the very obvious
> > performance improvement cases? I can also try this locally.
> >
>
> Here's some results from a build with
>
> #define MAX_BUFFERS_PER_TRANSFER 1
>
> There are three columns:
>
> - master
> - patched (original patches, with MAX_BUFFERS_PER_TRANSFER=128kB)
> - patched-single (MAX_BUFFERS_PER_TRANSFER=8kB)
>
> The color scales are always branch compared to master.
>
> I think the expectation was that setting the transfer to 1 would make it
> closer to master, reducing some of the regressions. But in practice the
> effect is the opposite.
>
> - In "cached" runs, this eliminates the small improvements (light
> green), but leaves the regressions behind.
For cached runs, I actually would expect that MAX_BUFFERS_PER_TRANSFER
would eliminate the regressions. Pinning more buffers will only hurt
us for cached workloads. This is evidence that we may need to control
the number of pinned buffers differently when there has been a run of
fully cached blocks.
> - In "uncached" runs, this exacerbates the regressions, particularly for
> low selectivities (small values of matches).
For the uncached runs, I actually expected it to eliminate the
performance gains that we saw with the patches applied. With
MAX_BUFFERS_PER_TRANSFER=1, we don't get the benefit of larger IOs and
fewer system calls but we still have the overhead of the streaming
read machinery. I was hoping to prove that the performance
improvements we saw with all the patches applied were due to
MAX_BUFFERS_PER_TRANSFER being > 1 causing fewer, bigger reads.
It did eliminate some performance gains, however, primarily for
cyclic-fuzz at lower selectivities. I am a little confused by this
part because with lower selectivities there are likely fewer
consecutive blocks that can be combined into one IO.
And, on average, we still see a lot of performance improvements that
were not eliminated by MAX_BUFFERS_PER_TRANSFER = 1. Hmm.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-01 00:29 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 19:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-01 00:29 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Thu, Feb 29, 2024 at 5:44 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
>
>
> On 2/29/24 22:19, Melanie Plageman wrote:
> > On Thu, Feb 29, 2024 at 7:54 AM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >>
> >>
> >> On 2/29/24 00:40, Melanie Plageman wrote:
> >>> On Wed, Feb 28, 2024 at 6:17 PM Tomas Vondra
> >>> <tomas.vondra@enterprisedb.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 2/28/24 21:06, Melanie Plageman wrote:
> >>>>> On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
> >>>>> <tomas.vondra@enterprisedb.com> wrote:
> >>>>>>
> >>>>>> On 2/28/24 15:56, Tomas Vondra wrote:
> >>>>>>>> ...
> >>>>>>>
> >>>>>>> Sure, I can do that. It'll take a couple hours to get the results, I'll
> >>>>>>> share them when I have them.
> >>>>>>>
> >>>>>>
> >>>>>> Here are the results with only patches 0001 - 0012 applied (i.e. without
> >>>>>> the patch introducing the streaming read API, and the patch switching
> >>>>>> the bitmap heap scan to use it).
> >>>>>>
> >>>>>> The changes in performance don't disappear entirely, but the scale is
> >>>>>> certainly much smaller - both in the complete results for all runs, and
> >>>>>> for the "optimal" runs that would actually pick bitmapscan.
> >>>>>
> >>>>> Hmm. I'm trying to think how my refactor could have had this impact.
> >>>>> It seems like all the most notable regressions are with 4 parallel
> >>>>> workers. What do the numeric column labels mean across the top
> >>>>> (2,4,8,16...) -- are they related to "matches"? And if so, what does
> >>>>> that mean?
> >>>>>
> >>>>
> >>>> That's the number of distinct values matched by the query, which should
> >>>> be an approximation of the number of matching rows. The number of
> >>>> distinct values in the data set differs by data set, but for 1M rows
> >>>> it's roughly like this:
> >>>>
> >>>> uniform: 10k
> >>>> linear: 10k
> >>>> cyclic: 100
> >>>>
> >>>> So for example matches=128 means ~1% of rows for uniform/linear, and
> >>>> 100% for cyclic data sets.
> >>>
> >>> Ah, thank you for the explanation. I also looked at your script after
> >>> having sent this email and saw that it is clear in your script what
> >>> "matches" is.
> >>>
> >>>> As for the possible cause, I think it's clear most of the difference
> >>>> comes from the last patch that actually switches bitmap heap scan to the
> >>>> streaming read API. That's mostly expected/understandable, although we
> >>>> probably need to look into the regressions or cases with e_i_c=0.
> >>>
> >>> Right, I'm mostly surprised about the regressions for patches 0001-0012.
> >>>
> >>> Re eic 0: Thomas Munro and I chatted off-list, and you bring up a
> >>> great point about eic 0. In old bitmapheapscan code eic 0 basically
> >>> disabled prefetching but with the streaming read API, it will still
> >>> issue fadvises when eic is 0. That is an easy one line fix. Thomas
> >>> prefers to fix it by always avoiding an fadvise for the last buffer in
> >>> a range before issuing a read (since we are about to read it anyway,
> >>> best not fadvise it too). This will fix eic 0 and also cut one system
> >>> call from each invocation of the streaming read machinery.
> >>>
> >>>> To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
> >>>> individual patches. I can try doing that tomorrow. It'll have to be a
> >>>> limited set of tests, to reduce the time, but might tell us whether it's
> >>>> due to a single patch or multiple patches.
> >>>
> >>> Yes, tomorrow I planned to start trying to repro some of the "red"
> >>> cases myself. Any one of the commits could cause a slight regression
> >>> but a 3.5x regression is quite surprising, so I might focus on trying
> >>> to repro that locally and then narrow down which patch causes it.
> >>>
> >>> For the non-cached regressions, perhaps the commit to use the correct
> >>> recheck flag (0004) when prefetching could be the culprit. And for the
> >>> cached regressions, my money is on the commit which changes the whole
> >>> control flow of BitmapHeapNext() and the next_block() and next_tuple()
> >>> functions (0010).
> >>>
> >>
> >> I do have some partial results, comparing the patches. I only ran one of
> >> the more affected workloads (cyclic) on the xeon, attached is a PDF
> >> comparing master and the 0001-0014 patches. The percentages are timing
> >> vs. the preceding patch (green - faster, red - slower).
> >
> > Just confirming: the results are for uncached?
> >
>
> Yes, cyclic data set, uncached case. I picked this because it seemed
> like one of the most affected cases. Do you want me to test some other
> cases too?
So, I actually may have found the source of at least part of the
regression with 0010. I was able to reproduce the regression with
patch 0010 applied for the unached case with 4 workers and eic 8 and
100000000 rows for the cyclic dataset. I see it for all number of
matches. The regression went away (for this specific example) when I
moved the BitmapAdjustPrefetchIterator call back up to before the call
to table_scan_bitmap_next_block() like this:
diff --git a/src/backend/executor/nodeBitmapHeapscan.c
b/src/backend/executor/nodeBitmapHeapscan.c
index f7ecc060317..268996bdeea 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -279,6 +279,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
new_page:
+ BitmapAdjustPrefetchIterator(node, node->blockno);
+
if (!table_scan_bitmap_next_block(scan, &node->recheck,
&lossy, &node->blockno))
break;
@@ -287,7 +289,6 @@ new_page:
else
node->exact_pages++;
- BitmapAdjustPrefetchIterator(node, node->blockno);
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
It makes sense this would fix it. I haven't tried all the combinations
you tried. Do you mind running your tests with the new code? I've
pushed it into this branch.
https://github.com/melanieplageman/postgres/commits/bhs_pgsr/
Note that this will fix none of the issues with 0014 because that has
removed all of the old prefetching code anyway.
Thank you sooo much for running these to begin with and then helping
me figure out what is going on!
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 00:29 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-01 19:31 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:11 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-01 19:31 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Thu, Feb 29, 2024 at 7:29 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> On Thu, Feb 29, 2024 at 5:44 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
> >
> >
> >
> > On 2/29/24 22:19, Melanie Plageman wrote:
> > > On Thu, Feb 29, 2024 at 7:54 AM Tomas Vondra
> > > <tomas.vondra@enterprisedb.com> wrote:
> > >>
> > >>
> > >>
> > >> On 2/29/24 00:40, Melanie Plageman wrote:
> > >>> On Wed, Feb 28, 2024 at 6:17 PM Tomas Vondra
> > >>> <tomas.vondra@enterprisedb.com> wrote:
> > >>>>
> > >>>>
> > >>>>
> > >>>> On 2/28/24 21:06, Melanie Plageman wrote:
> > >>>>> On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
> > >>>>> <tomas.vondra@enterprisedb.com> wrote:
> > >>>>>>
> > >>>>>> On 2/28/24 15:56, Tomas Vondra wrote:
> > >>>>>>>> ...
> > >>>>>>>
> > >>>>>>> Sure, I can do that. It'll take a couple hours to get the results, I'll
> > >>>>>>> share them when I have them.
> > >>>>>>>
> > >>>>>>
> > >>>>>> Here are the results with only patches 0001 - 0012 applied (i.e. without
> > >>>>>> the patch introducing the streaming read API, and the patch switching
> > >>>>>> the bitmap heap scan to use it).
> > >>>>>>
> > >>>>>> The changes in performance don't disappear entirely, but the scale is
> > >>>>>> certainly much smaller - both in the complete results for all runs, and
> > >>>>>> for the "optimal" runs that would actually pick bitmapscan.
> > >>>>>
> > >>>>> Hmm. I'm trying to think how my refactor could have had this impact.
> > >>>>> It seems like all the most notable regressions are with 4 parallel
> > >>>>> workers. What do the numeric column labels mean across the top
> > >>>>> (2,4,8,16...) -- are they related to "matches"? And if so, what does
> > >>>>> that mean?
> > >>>>>
> > >>>>
> > >>>> That's the number of distinct values matched by the query, which should
> > >>>> be an approximation of the number of matching rows. The number of
> > >>>> distinct values in the data set differs by data set, but for 1M rows
> > >>>> it's roughly like this:
> > >>>>
> > >>>> uniform: 10k
> > >>>> linear: 10k
> > >>>> cyclic: 100
> > >>>>
> > >>>> So for example matches=128 means ~1% of rows for uniform/linear, and
> > >>>> 100% for cyclic data sets.
> > >>>
> > >>> Ah, thank you for the explanation. I also looked at your script after
> > >>> having sent this email and saw that it is clear in your script what
> > >>> "matches" is.
> > >>>
> > >>>> As for the possible cause, I think it's clear most of the difference
> > >>>> comes from the last patch that actually switches bitmap heap scan to the
> > >>>> streaming read API. That's mostly expected/understandable, although we
> > >>>> probably need to look into the regressions or cases with e_i_c=0.
> > >>>
> > >>> Right, I'm mostly surprised about the regressions for patches 0001-0012.
> > >>>
> > >>> Re eic 0: Thomas Munro and I chatted off-list, and you bring up a
> > >>> great point about eic 0. In old bitmapheapscan code eic 0 basically
> > >>> disabled prefetching but with the streaming read API, it will still
> > >>> issue fadvises when eic is 0. That is an easy one line fix. Thomas
> > >>> prefers to fix it by always avoiding an fadvise for the last buffer in
> > >>> a range before issuing a read (since we are about to read it anyway,
> > >>> best not fadvise it too). This will fix eic 0 and also cut one system
> > >>> call from each invocation of the streaming read machinery.
> > >>>
> > >>>> To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
> > >>>> individual patches. I can try doing that tomorrow. It'll have to be a
> > >>>> limited set of tests, to reduce the time, but might tell us whether it's
> > >>>> due to a single patch or multiple patches.
> > >>>
> > >>> Yes, tomorrow I planned to start trying to repro some of the "red"
> > >>> cases myself. Any one of the commits could cause a slight regression
> > >>> but a 3.5x regression is quite surprising, so I might focus on trying
> > >>> to repro that locally and then narrow down which patch causes it.
> > >>>
> > >>> For the non-cached regressions, perhaps the commit to use the correct
> > >>> recheck flag (0004) when prefetching could be the culprit. And for the
> > >>> cached regressions, my money is on the commit which changes the whole
> > >>> control flow of BitmapHeapNext() and the next_block() and next_tuple()
> > >>> functions (0010).
> > >>>
> > >>
> > >> I do have some partial results, comparing the patches. I only ran one of
> > >> the more affected workloads (cyclic) on the xeon, attached is a PDF
> > >> comparing master and the 0001-0014 patches. The percentages are timing
> > >> vs. the preceding patch (green - faster, red - slower).
> > >
> > > Just confirming: the results are for uncached?
> > >
> >
> > Yes, cyclic data set, uncached case. I picked this because it seemed
> > like one of the most affected cases. Do you want me to test some other
> > cases too?
>
> So, I actually may have found the source of at least part of the
> regression with 0010. I was able to reproduce the regression with
> patch 0010 applied for the unached case with 4 workers and eic 8 and
> 100000000 rows for the cyclic dataset. I see it for all number of
> matches. The regression went away (for this specific example) when I
> moved the BitmapAdjustPrefetchIterator call back up to before the call
> to table_scan_bitmap_next_block() like this:
>
> diff --git a/src/backend/executor/nodeBitmapHeapscan.c
> b/src/backend/executor/nodeBitmapHeapscan.c
> index f7ecc060317..268996bdeea 100644
> --- a/src/backend/executor/nodeBitmapHeapscan.c
> +++ b/src/backend/executor/nodeBitmapHeapscan.c
> @@ -279,6 +279,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
> }
>
> new_page:
> + BitmapAdjustPrefetchIterator(node, node->blockno);
> +
> if (!table_scan_bitmap_next_block(scan, &node->recheck,
> &lossy, &node->blockno))
> break;
>
> @@ -287,7 +289,6 @@ new_page:
> else
> node->exact_pages++;
>
> - BitmapAdjustPrefetchIterator(node, node->blockno);
> /* Adjust the prefetch target */
> BitmapAdjustPrefetchTarget(node);
> }
>
> It makes sense this would fix it. I haven't tried all the combinations
> you tried. Do you mind running your tests with the new code? I've
> pushed it into this branch.
> https://github.com/melanieplageman/postgres/commits/bhs_pgsr/
Hold the phone on this one. I realized why I moved
BitmapAdjustPrefetchIterator after table_scan_bitmap_next_block() in
the first place -- master calls BitmapAdjustPrefetchIterator after the
tbm_iterate() for the current block -- otherwise with eic = 1, it
considers the prefetch iterator behind the current block iterator. I'm
going to go through and figure out what order this must be done in and
fix it.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 00:29 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 19:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-02 22:11 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-02 22:11 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Fri, Mar 1, 2024 at 2:31 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
>
> On Thu, Feb 29, 2024 at 7:29 PM Melanie Plageman
> <melanieplageman@gmail.com> wrote:
> >
> > On Thu, Feb 29, 2024 at 5:44 PM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> > >
> > >
> > >
> > > On 2/29/24 22:19, Melanie Plageman wrote:
> > > > On Thu, Feb 29, 2024 at 7:54 AM Tomas Vondra
> > > > <tomas.vondra@enterprisedb.com> wrote:
> > > >>
> > > >>
> > > >>
> > > >> On 2/29/24 00:40, Melanie Plageman wrote:
> > > >>> On Wed, Feb 28, 2024 at 6:17 PM Tomas Vondra
> > > >>> <tomas.vondra@enterprisedb.com> wrote:
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On 2/28/24 21:06, Melanie Plageman wrote:
> > > >>>>> On Wed, Feb 28, 2024 at 2:23 PM Tomas Vondra
> > > >>>>> <tomas.vondra@enterprisedb.com> wrote:
> > > >>>>>>
> > > >>>>>> On 2/28/24 15:56, Tomas Vondra wrote:
> > > >>>>>>>> ...
> > > >>>>>>>
> > > >>>>>>> Sure, I can do that. It'll take a couple hours to get the results, I'll
> > > >>>>>>> share them when I have them.
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>> Here are the results with only patches 0001 - 0012 applied (i.e. without
> > > >>>>>> the patch introducing the streaming read API, and the patch switching
> > > >>>>>> the bitmap heap scan to use it).
> > > >>>>>>
> > > >>>>>> The changes in performance don't disappear entirely, but the scale is
> > > >>>>>> certainly much smaller - both in the complete results for all runs, and
> > > >>>>>> for the "optimal" runs that would actually pick bitmapscan.
> > > >>>>>
> > > >>>>> Hmm. I'm trying to think how my refactor could have had this impact.
> > > >>>>> It seems like all the most notable regressions are with 4 parallel
> > > >>>>> workers. What do the numeric column labels mean across the top
> > > >>>>> (2,4,8,16...) -- are they related to "matches"? And if so, what does
> > > >>>>> that mean?
> > > >>>>>
> > > >>>>
> > > >>>> That's the number of distinct values matched by the query, which should
> > > >>>> be an approximation of the number of matching rows. The number of
> > > >>>> distinct values in the data set differs by data set, but for 1M rows
> > > >>>> it's roughly like this:
> > > >>>>
> > > >>>> uniform: 10k
> > > >>>> linear: 10k
> > > >>>> cyclic: 100
> > > >>>>
> > > >>>> So for example matches=128 means ~1% of rows for uniform/linear, and
> > > >>>> 100% for cyclic data sets.
> > > >>>
> > > >>> Ah, thank you for the explanation. I also looked at your script after
> > > >>> having sent this email and saw that it is clear in your script what
> > > >>> "matches" is.
> > > >>>
> > > >>>> As for the possible cause, I think it's clear most of the difference
> > > >>>> comes from the last patch that actually switches bitmap heap scan to the
> > > >>>> streaming read API. That's mostly expected/understandable, although we
> > > >>>> probably need to look into the regressions or cases with e_i_c=0.
> > > >>>
> > > >>> Right, I'm mostly surprised about the regressions for patches 0001-0012.
> > > >>>
> > > >>> Re eic 0: Thomas Munro and I chatted off-list, and you bring up a
> > > >>> great point about eic 0. In old bitmapheapscan code eic 0 basically
> > > >>> disabled prefetching but with the streaming read API, it will still
> > > >>> issue fadvises when eic is 0. That is an easy one line fix. Thomas
> > > >>> prefers to fix it by always avoiding an fadvise for the last buffer in
> > > >>> a range before issuing a read (since we are about to read it anyway,
> > > >>> best not fadvise it too). This will fix eic 0 and also cut one system
> > > >>> call from each invocation of the streaming read machinery.
> > > >>>
> > > >>>> To analyze the 0001-0012 patches, maybe it'd be helpful to run tests for
> > > >>>> individual patches. I can try doing that tomorrow. It'll have to be a
> > > >>>> limited set of tests, to reduce the time, but might tell us whether it's
> > > >>>> due to a single patch or multiple patches.
> > > >>>
> > > >>> Yes, tomorrow I planned to start trying to repro some of the "red"
> > > >>> cases myself. Any one of the commits could cause a slight regression
> > > >>> but a 3.5x regression is quite surprising, so I might focus on trying
> > > >>> to repro that locally and then narrow down which patch causes it.
> > > >>>
> > > >>> For the non-cached regressions, perhaps the commit to use the correct
> > > >>> recheck flag (0004) when prefetching could be the culprit. And for the
> > > >>> cached regressions, my money is on the commit which changes the whole
> > > >>> control flow of BitmapHeapNext() and the next_block() and next_tuple()
> > > >>> functions (0010).
> > > >>>
> > > >>
> > > >> I do have some partial results, comparing the patches. I only ran one of
> > > >> the more affected workloads (cyclic) on the xeon, attached is a PDF
> > > >> comparing master and the 0001-0014 patches. The percentages are timing
> > > >> vs. the preceding patch (green - faster, red - slower).
> > > >
> > > > Just confirming: the results are for uncached?
> > > >
> > >
> > > Yes, cyclic data set, uncached case. I picked this because it seemed
> > > like one of the most affected cases. Do you want me to test some other
> > > cases too?
> >
> > So, I actually may have found the source of at least part of the
> > regression with 0010. I was able to reproduce the regression with
> > patch 0010 applied for the unached case with 4 workers and eic 8 and
> > 100000000 rows for the cyclic dataset. I see it for all number of
> > matches. The regression went away (for this specific example) when I
> > moved the BitmapAdjustPrefetchIterator call back up to before the call
> > to table_scan_bitmap_next_block() like this:
> >
> > diff --git a/src/backend/executor/nodeBitmapHeapscan.c
> > b/src/backend/executor/nodeBitmapHeapscan.c
> > index f7ecc060317..268996bdeea 100644
> > --- a/src/backend/executor/nodeBitmapHeapscan.c
> > +++ b/src/backend/executor/nodeBitmapHeapscan.c
> > @@ -279,6 +279,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
> > }
> >
> > new_page:
> > + BitmapAdjustPrefetchIterator(node, node->blockno);
> > +
> > if (!table_scan_bitmap_next_block(scan, &node->recheck,
> > &lossy, &node->blockno))
> > break;
> >
> > @@ -287,7 +289,6 @@ new_page:
> > else
> > node->exact_pages++;
> >
> > - BitmapAdjustPrefetchIterator(node, node->blockno);
> > /* Adjust the prefetch target */
> > BitmapAdjustPrefetchTarget(node);
> > }
> >
> > It makes sense this would fix it. I haven't tried all the combinations
> > you tried. Do you mind running your tests with the new code? I've
> > pushed it into this branch.
> > https://github.com/melanieplageman/postgres/commits/bhs_pgsr/
>
> Hold the phone on this one. I realized why I moved
> BitmapAdjustPrefetchIterator after table_scan_bitmap_next_block() in
> the first place -- master calls BitmapAdjustPrefetchIterator after the
> tbm_iterate() for the current block -- otherwise with eic = 1, it
> considers the prefetch iterator behind the current block iterator. I'm
> going to go through and figure out what order this must be done in and
> fix it.
So, I investigated this further, and, as far as I can tell, for
parallel bitmapheapscan the timing around when workers decrement
prefetch_pages causes the performance differences with patch 0010
applied. It makes very little sense to me, but some of the queries I
borrowed from your regression examples are up to 30% slower when this
code from BitmapAdjustPrefetchIterator() is after
table_scan_bitmap_next_block() instead of before it.
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
pstate->prefetch_pages--;
SpinLockRelease(&pstate->mutex);
I did some stracing and did see much more time spent in futex/wait
with this code after the call to table_scan_bitmap_next_block() vs
before it. (table_scan_bitmap_next_block()) calls ReadBuffer()).
In my branch, I've now moved only the parallel prefetch_pages-- code
to before table_scan_bitmap_next_block().
https://github.com/melanieplageman/postgres/tree/bhs_pgsr
I'd be interested to know if you see the regressions go away with 0010
applied (commit message "Make table_scan_bitmap_next_block() async
friendly" and sha bfdcbfee7be8e2c461).
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 00:29 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 19:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:11 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-02 22:51 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 23:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-02 22:51 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/2/24 23:11, Melanie Plageman wrote:
> On Fri, Mar 1, 2024 at 2:31 PM Melanie Plageman
> <melanieplageman@gmail.com> wrote:
>>
>> ...
>>
>> Hold the phone on this one. I realized why I moved
>> BitmapAdjustPrefetchIterator after table_scan_bitmap_next_block() in
>> the first place -- master calls BitmapAdjustPrefetchIterator after the
>> tbm_iterate() for the current block -- otherwise with eic = 1, it
>> considers the prefetch iterator behind the current block iterator. I'm
>> going to go through and figure out what order this must be done in and
>> fix it.
>
> So, I investigated this further, and, as far as I can tell, for
> parallel bitmapheapscan the timing around when workers decrement
> prefetch_pages causes the performance differences with patch 0010
> applied. It makes very little sense to me, but some of the queries I
> borrowed from your regression examples are up to 30% slower when this
> code from BitmapAdjustPrefetchIterator() is after
> table_scan_bitmap_next_block() instead of before it.
>
> SpinLockAcquire(&pstate->mutex);
> if (pstate->prefetch_pages > 0)
> pstate->prefetch_pages--;
> SpinLockRelease(&pstate->mutex);
>
> I did some stracing and did see much more time spent in futex/wait
> with this code after the call to table_scan_bitmap_next_block() vs
> before it. (table_scan_bitmap_next_block()) calls ReadBuffer()).
>
> In my branch, I've now moved only the parallel prefetch_pages-- code
> to before table_scan_bitmap_next_block().
> https://github.com/melanieplageman/postgres/tree/bhs_pgsr
> I'd be interested to know if you see the regressions go away with 0010
> applied (commit message "Make table_scan_bitmap_next_block() async
> friendly" and sha bfdcbfee7be8e2c461).
>
I'll give this a try once the runs with MAX_BUFFERS_PER_TRANSFER=1
complete. But it seems really bizarre that simply moving this code a
little bit would cause such a regression ...
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 00:29 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 19:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:11 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-02 23:39 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-03 14:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-02 23:39 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sat, Mar 2, 2024 at 5:51 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 3/2/24 23:11, Melanie Plageman wrote:
> > On Fri, Mar 1, 2024 at 2:31 PM Melanie Plageman
> > <melanieplageman@gmail.com> wrote:
> >>
> >> ...
> >>
> >> Hold the phone on this one. I realized why I moved
> >> BitmapAdjustPrefetchIterator after table_scan_bitmap_next_block() in
> >> the first place -- master calls BitmapAdjustPrefetchIterator after the
> >> tbm_iterate() for the current block -- otherwise with eic = 1, it
> >> considers the prefetch iterator behind the current block iterator. I'm
> >> going to go through and figure out what order this must be done in and
> >> fix it.
> >
> > So, I investigated this further, and, as far as I can tell, for
> > parallel bitmapheapscan the timing around when workers decrement
> > prefetch_pages causes the performance differences with patch 0010
> > applied. It makes very little sense to me, but some of the queries I
> > borrowed from your regression examples are up to 30% slower when this
> > code from BitmapAdjustPrefetchIterator() is after
> > table_scan_bitmap_next_block() instead of before it.
> >
> > SpinLockAcquire(&pstate->mutex);
> > if (pstate->prefetch_pages > 0)
> > pstate->prefetch_pages--;
> > SpinLockRelease(&pstate->mutex);
> >
> > I did some stracing and did see much more time spent in futex/wait
> > with this code after the call to table_scan_bitmap_next_block() vs
> > before it. (table_scan_bitmap_next_block()) calls ReadBuffer()).
> >
> > In my branch, I've now moved only the parallel prefetch_pages-- code
> > to before table_scan_bitmap_next_block().
> > https://github.com/melanieplageman/postgres/tree/bhs_pgsr
> > I'd be interested to know if you see the regressions go away with 0010
> > applied (commit message "Make table_scan_bitmap_next_block() async
> > friendly" and sha bfdcbfee7be8e2c461).
> >
>
> I'll give this a try once the runs with MAX_BUFFERS_PER_TRANSFER=1
> complete. But it seems really bizarre that simply moving this code a
> little bit would cause such a regression ...
Yes, it is bizarre. It also might not be a reproducible performance
difference on the cases besides the one I was testing (cyclic dataset,
uncached, eic=8, matches 16+, distinct=100, rows=100000000, 4 parallel
workers). But even if it only affects that one case, it still had a
major, reproducible performance impact to move those 5 lines before
and after table_scan_bitmap_next_block().
The same number of reads and fadvises are being issued overall.
However, I did notice that the pread calls are skewed when the those
lines of code are after table_scan_bitmap_next_block() -- fewer of
the workers are doing more of the reads. Perhaps this explains what is
taking longer. Why those workers would end up doing more of the reads,
I don't quite know.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 00:29 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 19:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:11 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 23:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-03 14:36 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 0 replies; 248+ messages in thread
From: Tomas Vondra @ 2024-03-03 14:36 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/3/24 00:39, Melanie Plageman wrote:
> On Sat, Mar 2, 2024 at 5:51 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 3/2/24 23:11, Melanie Plageman wrote:
>>> On Fri, Mar 1, 2024 at 2:31 PM Melanie Plageman
>>> <melanieplageman@gmail.com> wrote:
>>>>
>>>> ...
>>>>
>>>> Hold the phone on this one. I realized why I moved
>>>> BitmapAdjustPrefetchIterator after table_scan_bitmap_next_block() in
>>>> the first place -- master calls BitmapAdjustPrefetchIterator after the
>>>> tbm_iterate() for the current block -- otherwise with eic = 1, it
>>>> considers the prefetch iterator behind the current block iterator. I'm
>>>> going to go through and figure out what order this must be done in and
>>>> fix it.
>>>
>>> So, I investigated this further, and, as far as I can tell, for
>>> parallel bitmapheapscan the timing around when workers decrement
>>> prefetch_pages causes the performance differences with patch 0010
>>> applied. It makes very little sense to me, but some of the queries I
>>> borrowed from your regression examples are up to 30% slower when this
>>> code from BitmapAdjustPrefetchIterator() is after
>>> table_scan_bitmap_next_block() instead of before it.
>>>
>>> SpinLockAcquire(&pstate->mutex);
>>> if (pstate->prefetch_pages > 0)
>>> pstate->prefetch_pages--;
>>> SpinLockRelease(&pstate->mutex);
>>>
>>> I did some stracing and did see much more time spent in futex/wait
>>> with this code after the call to table_scan_bitmap_next_block() vs
>>> before it. (table_scan_bitmap_next_block()) calls ReadBuffer()).
>>>
>>> In my branch, I've now moved only the parallel prefetch_pages-- code
>>> to before table_scan_bitmap_next_block().
>>> https://github.com/melanieplageman/postgres/tree/bhs_pgsr
>>> I'd be interested to know if you see the regressions go away with 0010
>>> applied (commit message "Make table_scan_bitmap_next_block() async
>>> friendly" and sha bfdcbfee7be8e2c461).
>>>
>>
>> I'll give this a try once the runs with MAX_BUFFERS_PER_TRANSFER=1
>> complete. But it seems really bizarre that simply moving this code a
>> little bit would cause such a regression ...
>
> Yes, it is bizarre. It also might not be a reproducible performance
> difference on the cases besides the one I was testing (cyclic dataset,
> uncached, eic=8, matches 16+, distinct=100, rows=100000000, 4 parallel
> workers). But even if it only affects that one case, it still had a
> major, reproducible performance impact to move those 5 lines before
> and after table_scan_bitmap_next_block().
>
> The same number of reads and fadvises are being issued overall.
> However, I did notice that the pread calls are skewed when the those
> lines of code are after table_scan_bitmap_next_block() -- fewer of
> the workers are doing more of the reads. Perhaps this explains what is
> taking longer. Why those workers would end up doing more of the reads,
> I don't quite know.
>
> - Melanie
I do have some numbers with e44505ce179e442bd50664c85a31a1805e13514a,
and I don't see any such effect - it performs pretty much exactly like
the v6 patches.
I used a slightly different visualization, plotting the timings on a
scatter plot, so values on diagonal mean "same performance" while values
above/below mean speedup/slowdown.
This is a bit more compact than the tables with color scales, and it
makes it harder (impossible) to see patterns (e.g. changes depending on
eic). But for evaluating if there's a shift overall it still works, and
it also shows clusters. So more a complementary & simpler visualization.
There are three charts
1) master-patched.png - master vs. v6 patches
2) master-locks.png - master vs. e44505ce
3) patched-locks.png - v6 patches vs. e44505ce
There's virtually no difference between (1) and (2) - same pattern of
regressions and speedups, almost as a copy. That's confirmed by (3)
where pretty much all values are exactly on the diagonal, with only a
couple outliers.
I'm not sure why you see a 30% difference with the change. I wonder if
that might be due to some issue in the environment? Are you running in a
VM, or something like that?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[image/png] pathed-locks.png (61.3K, ../../86ddff18-175b-4ae7-a039-129eaf4abe8f@enterprisedb.com/2-pathed-locks.png)
download | view image
[image/png] master-locks.png (80.2K, ../../86ddff18-175b-4ae7-a039-129eaf4abe8f@enterprisedb.com/3-master-locks.png)
download | view image
[image/png] master-patched.png (79.1K, ../../86ddff18-175b-4ae7-a039-129eaf4abe8f@enterprisedb.com/4-master-patched.png)
download | view image
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-13 13:34 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 18:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
1 sibling, 2 replies; 248+ messages in thread
From: Heikki Linnakangas @ 2024-03-13 13:34 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; +Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
(Adding Dilip, the original author of the parallel bitmap heap scan
patch all those years ago, in case you remember anything about the
snapshot stuff below.)
On 27/02/2024 16:22, Melanie Plageman wrote:
> On Mon, Feb 26, 2024 at 08:50:28PM -0500, Melanie Plageman wrote:
>> On Fri, Feb 16, 2024 at 12:35:59PM -0500, Melanie Plageman wrote:
>>> In the attached v3, I've reordered the commits, updated some errant
>>> comments, and improved the commit messages.
>>>
>>> I've also made some updates to the TIDBitmap API that seem like a
>>> clarity improvement to the API in general. These also reduce the diff
>>> for GIN when separating the TBMIterateResult from the
>>> TBM[Shared]Iterator. And these TIDBitmap API changes are now all in
>>> their own commits (previously those were in the same commit as adding
>>> the BitmapHeapScan streaming read user).
>>>
>>> The three outstanding issues I see in the patch set are:
>>> 1) the lossy and exact page counters issue described in my previous
>>> email
>>
>> I've resolved this. I added a new patch to the set which starts counting
>> even pages with no visible tuples toward lossy and exact pages. After an
>> off-list conversation with Andres, it seems that this omission in master
>> may not have been intentional.
>>
>> Once we have only two types of pages to differentiate between (lossy and
>> exact [no longer have to care about "has no visible tuples"]), it is
>> easy enough to pass a "lossy" boolean paramater to
>> table_scan_bitmap_next_block(). I've done this in the attached v4.
>
> Thomas posted a new version of the Streaming Read API [1], so here is a
> rebased v5. This should make it easier to review as it can be applied on
> top of master.
Lots of discussion happening on the performance results but it seems
that there is no performance impact with the preliminary patches up to
v5-0013-Streaming-Read-API.patch. I'm focusing purely on those
preliminary patches now, because I think they're worthwhile cleanups
independent of the streaming read API.
Andres already commented on the snapshot stuff on an earlier patch
version, and that's much nicer with this version. However, I don't
understand why a parallel bitmap heap scan needs to do anything at all
with the snapshot, even before these patches. The parallel worker
infrastructure already passes the active snapshot from the leader to the
parallel worker. Why does bitmap heap scan code need to do that too?
I disabled that with:
> --- a/src/backend/executor/nodeBitmapHeapscan.c
> +++ b/src/backend/executor/nodeBitmapHeapscan.c
> @@ -874,7 +874,9 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
> pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
> node->pstate = pstate;
>
> +#if 0
> node->worker_snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
> Assert(IsMVCCSnapshot(node->worker_snapshot));
> RegisterSnapshot(node->worker_snapshot);
> +#endif
> }
and ran "make check-world". All the tests passed. To be even more sure,
I added some code there to assert that the serialized version of
node->ss.ps.state->es_snapshot is equal to pstate->phs_snapshot_data,
and all the tests passed with that too.
I propose that we just remove the code in BitmapHeapScan to serialize
the snapshot, per attached patch.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
[text/x-patch] 0001-Remove-redundant-snapshot-copying-from-parallel-lead.patch (5.7K, ../../5f3b9d59-0f43-419d-80ca-6d04c07cf61a@iki.fi/2-0001-Remove-redundant-snapshot-copying-from-parallel-lead.patch)
download | inline diff:
From 265d77efa2b56d5cfc3caf7843058b7336524860 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 13 Mar 2024 15:25:07 +0200
Subject: [PATCH 1/1] Remove redundant snapshot copying from parallel leader to
workers
The parallel query infrastructure copies the leader backend's active
snapshot to the worker processes. But BitmapHeapScan node also had
bespoken code to pass the snapshot from leader to the worker. That was
redundant, so remove it.
Discussion: XX
---
src/backend/access/table/tableam.c | 10 ----------
src/backend/executor/nodeBitmapHeapscan.c | 17 ++---------------
src/include/access/tableam.h | 5 -----
src/include/nodes/execnodes.h | 4 ----
4 files changed, 2 insertions(+), 34 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 6ed8cca05a1..e57a0b7ea31 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,16 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-void
-table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot)
-{
- Assert(IsMVCCSnapshot(snapshot));
-
- RegisterSnapshot(snapshot);
- scan->rs_snapshot = snapshot;
- scan->rs_flags |= SO_TEMP_SNAPSHOT;
-}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 345b67649ea..ca548e44eb4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -721,7 +721,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
- scanstate->pscan_len = 0;
scanstate->initialized = false;
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
@@ -841,13 +840,7 @@ void
ExecBitmapHeapEstimate(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
- EState *estate = node->ss.ps.state;
-
- node->pscan_len = add_size(offsetof(ParallelBitmapHeapState,
- phs_snapshot_data),
- EstimateSnapshotSpace(estate->es_snapshot));
-
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ shm_toc_estimate_chunk(&pcxt->estimator, sizeof(ParallelBitmapHeapState));
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -862,14 +855,13 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
ParallelBitmapHeapState *pstate;
- EState *estate = node->ss.ps.state;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/* If there's no DSA, there are no workers; initialize nothing. */
if (dsa == NULL)
return;
- pstate = shm_toc_allocate(pcxt->toc, node->pscan_len);
+ pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
pstate->tbmiterator = 0;
pstate->prefetch_iterator = 0;
@@ -881,7 +873,6 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
- SerializeSnapshot(estate->es_snapshot, pstate->phs_snapshot_data);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pstate);
node->pstate = pstate;
@@ -927,13 +918,9 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelBitmapHeapState *pstate;
- Snapshot snapshot;
Assert(node->ss.ps.state->es_query_dsa != NULL);
pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->pstate = pstate;
-
- snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
- table_scan_update_snapshot(node->ss.ss_currentScanDesc, snapshot);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5f8474871d2..8249b37bbf1 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -1038,11 +1038,6 @@ table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
allow_pagemode);
}
-/*
- * Update snapshot used by the scan.
- */
-extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
-
/*
* Return next tuple from `scan`, store in slot.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 444a5f0fd57..27614ab50fb 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1689,7 +1689,6 @@ typedef enum
* prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
- * phs_snapshot_data snapshot data shared to workers
* ----------------
*/
typedef struct ParallelBitmapHeapState
@@ -1701,7 +1700,6 @@ typedef struct ParallelBitmapHeapState
int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
- char phs_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
} ParallelBitmapHeapState;
/* ----------------
@@ -1721,7 +1719,6 @@ typedef struct ParallelBitmapHeapState
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
- * pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
* shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
@@ -1745,7 +1742,6 @@ typedef struct BitmapHeapScanState
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
- Size pscan_len;
bool initialized;
TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
--
2.39.2
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-13 15:39 ` Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Dilip Kumar @ 2024-03-13 15:39 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On Wed, Mar 13, 2024 at 7:04 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
> (Adding Dilip, the original author of the parallel bitmap heap scan
> patch all those years ago, in case you remember anything about the
> snapshot stuff below.)
>
> On 27/02/2024 16:22, Melanie Plageman wrote:
> Andres already commented on the snapshot stuff on an earlier patch
> version, and that's much nicer with this version. However, I don't
> understand why a parallel bitmap heap scan needs to do anything at all
> with the snapshot, even before these patches. The parallel worker
> infrastructure already passes the active snapshot from the leader to the
> parallel worker. Why does bitmap heap scan code need to do that too?
Yeah thinking on this now it seems you are right that the parallel
infrastructure is already passing the active snapshot so why do we
need it again. Then I checked other low scan nodes like indexscan and
seqscan and it seems we are doing the same things there as well.
Check for SerializeSnapshot() in table_parallelscan_initialize() and
index_parallelscan_initialize() which are being called from
ExecSeqScanInitializeDSM() and ExecIndexScanInitializeDSM()
respectively.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
@ 2024-03-13 15:55 ` Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Robert Haas @ 2024-03-13 15:55 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On Wed, Mar 13, 2024 at 11:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > Andres already commented on the snapshot stuff on an earlier patch
> > version, and that's much nicer with this version. However, I don't
> > understand why a parallel bitmap heap scan needs to do anything at all
> > with the snapshot, even before these patches. The parallel worker
> > infrastructure already passes the active snapshot from the leader to the
> > parallel worker. Why does bitmap heap scan code need to do that too?
>
> Yeah thinking on this now it seems you are right that the parallel
> infrastructure is already passing the active snapshot so why do we
> need it again. Then I checked other low scan nodes like indexscan and
> seqscan and it seems we are doing the same things there as well.
> Check for SerializeSnapshot() in table_parallelscan_initialize() and
> index_parallelscan_initialize() which are being called from
> ExecSeqScanInitializeDSM() and ExecIndexScanInitializeDSM()
> respectively.
I remember thinking about this when I was writing very early parallel
query code. It seemed to me that there must be some reason why the
EState has a snapshot, as opposed to just using the active snapshot,
and so I took care to propagate that snapshot, which is used for the
leader's scans, to the worker scans also. Now, if the EState doesn't
need to contain a snapshot, then all of that mechanism is unnecessary,
but I don't see how it can be right for the leader to do
table_beginscan() using estate->es_snapshot and the worker to use the
active snapshot.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
@ 2024-03-14 04:54 ` Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 248+ messages in thread
From: Dilip Kumar @ 2024-03-14 04:54 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On Wed, Mar 13, 2024 at 9:25 PM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Wed, Mar 13, 2024 at 11:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > > Andres already commented on the snapshot stuff on an earlier patch
> > > version, and that's much nicer with this version. However, I don't
> > > understand why a parallel bitmap heap scan needs to do anything at all
> > > with the snapshot, even before these patches. The parallel worker
> > > infrastructure already passes the active snapshot from the leader to the
> > > parallel worker. Why does bitmap heap scan code need to do that too?
> >
> > Yeah thinking on this now it seems you are right that the parallel
> > infrastructure is already passing the active snapshot so why do we
> > need it again. Then I checked other low scan nodes like indexscan and
> > seqscan and it seems we are doing the same things there as well.
> > Check for SerializeSnapshot() in table_parallelscan_initialize() and
> > index_parallelscan_initialize() which are being called from
> > ExecSeqScanInitializeDSM() and ExecIndexScanInitializeDSM()
> > respectively.
>
> I remember thinking about this when I was writing very early parallel
> query code. It seemed to me that there must be some reason why the
> EState has a snapshot, as opposed to just using the active snapshot,
> and so I took care to propagate that snapshot, which is used for the
> leader's scans, to the worker scans also. Now, if the EState doesn't
> need to contain a snapshot, then all of that mechanism is unnecessary,
> but I don't see how it can be right for the leader to do
> table_beginscan() using estate->es_snapshot and the worker to use the
> active snapshot.
Yeah, that's a very valid point. So I think now Heikki/Melanie might
have got an answer to their question, about the thought process behind
serializing the snapshot for each scan node. And the same thing is
followed for BitmapHeapNode as well.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
@ 2024-03-14 10:37 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 12:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
0 siblings, 2 replies; 248+ messages in thread
From: Heikki Linnakangas @ 2024-03-14 10:37 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On 14/03/2024 06:54, Dilip Kumar wrote:
> On Wed, Mar 13, 2024 at 9:25 PM Robert Haas <robertmhaas@gmail.com> wrote:
>>
>> On Wed, Mar 13, 2024 at 11:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>>>> Andres already commented on the snapshot stuff on an earlier patch
>>>> version, and that's much nicer with this version. However, I don't
>>>> understand why a parallel bitmap heap scan needs to do anything at all
>>>> with the snapshot, even before these patches. The parallel worker
>>>> infrastructure already passes the active snapshot from the leader to the
>>>> parallel worker. Why does bitmap heap scan code need to do that too?
>>>
>>> Yeah thinking on this now it seems you are right that the parallel
>>> infrastructure is already passing the active snapshot so why do we
>>> need it again. Then I checked other low scan nodes like indexscan and
>>> seqscan and it seems we are doing the same things there as well.
>>> Check for SerializeSnapshot() in table_parallelscan_initialize() and
>>> index_parallelscan_initialize() which are being called from
>>> ExecSeqScanInitializeDSM() and ExecIndexScanInitializeDSM()
>>> respectively.
>>
>> I remember thinking about this when I was writing very early parallel
>> query code. It seemed to me that there must be some reason why the
>> EState has a snapshot, as opposed to just using the active snapshot,
>> and so I took care to propagate that snapshot, which is used for the
>> leader's scans, to the worker scans also. Now, if the EState doesn't
>> need to contain a snapshot, then all of that mechanism is unnecessary,
>> but I don't see how it can be right for the leader to do
>> table_beginscan() using estate->es_snapshot and the worker to use the
>> active snapshot.
>
> Yeah, that's a very valid point. So I think now Heikki/Melanie might
> have got an answer to their question, about the thought process behind
> serializing the snapshot for each scan node. And the same thing is
> followed for BitmapHeapNode as well.
I see. Thanks, understanding the thought process helps.
So when a parallel table or index scan runs in the executor as part of a
query, we could just use the active snapshot. But there are some other
callers of parallel table scans that don't use the executor, namely
parallel index builds. For those it makes sense to pass the snapshot for
the scan independent of the active snapshot.
A parallel bitmap heap scan isn't really a parallel scan as far as the
table AM is concerned, though. It's more like an independent bitmap heap
scan in each worker process, nodeBitmapHeapscan.c does all the
coordination of which blocks to scan. So I think that
table_parallelscan_initialize() was the wrong role model, and we should
still remove the snapshot serialization code from nodeBitmapHeapscan.c.
Digging deeper into the question of whether es_snapshot ==
GetActiveSnapshot() is a valid assumption:
<deep dive>
es_snapshot is copied from the QueryDesc in standard_ExecutorStart().
Looking at the callers of ExecutorStart(), they all get the QueryDesc by
calling CreateQueryDesc() with GetActiveSnapshot(). And I don't see any
callers changing the active snapshot between the ExecutorStart() and
ExecutorRun() calls either. In pquery.c, we explicitly
PushActiveSnapshot(queryDesc->snapshot) before calling ExecutorRun(). So
no live bug here AFAICS, es_snapshot == GetActiveSnapshot() holds.
_SPI_execute_plan() has code to deal with the possibility that the
active snapshot is not set. That seems fishy; do we really support SPI
without any snapshot? I'm inclined to turn that into an error. I ran the
regression tests with an "Assert(ActiveSnapshotSet())" there, and
everything worked.
If es_snapshot was different from the active snapshot, things would get
weird, even without parallel query. The scans would use es_snapshot for
the visibility checks, but any functions you execute in quals would use
the active snapshot.
We could double down on that assumption, and remove es_snapshot
altogether and use GetActiveSnapshot() instead. And perhaps add
"PushActiveSnapshot(queryDesc->snapshot)" to ExecutorRun().
</deep dive>
In summary, this es_snapshot stuff is a bit confusing and could use some
cleanup. But for now, I'd like to just add some assertions and a
comments about this, and remove the snapshot serialization from bitmap
heap scan node, to make it consistent with other non-parallel scan nodes
(it's not really a parallel scan as far as the table AM is concerned).
See attached patch, which is the same as previous patch with some extra
assertions.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
[text/x-patch] v2-0001-Remove-redundant-snapshot-copying-from-parallel-l.patch (8.2K, ../../cd50927a-8b19-4f0f-884d-609212c71c91@iki.fi/2-v2-0001-Remove-redundant-snapshot-copying-from-parallel-l.patch)
download | inline diff:
From 44eb77162ccc6f18a0a5eaccf0c083d4fefd076f Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Thu, 14 Mar 2024 11:56:11 +0200
Subject: [PATCH v2 1/1] Remove redundant snapshot copying from parallel leader
to workers
The parallel query infrastructure copies the leader backend's active
snapshot to the worker processes. But BitmapHeapScan node also had
bespoken code to pass the snapshot from leader to the worker. That was
redundant, so remove it.
The removed code was analogous to the snapshot serialization in
table_parallelscan_initialize(), but that was the wrong role model. A
parallel bitmap heap scan is more like an independent non-parallel
bitmap heap scan in each parallel worker as far as the table AM is
concerned, because the coordination is done in nodeBitmapHeapscan.c,
and the table AM doesn't need to know anything about it.
This relies on the assumption that es_snapshot ==
GetActiveSnapshot(). That's not a new assumption, things would get
weird if you used the QueryDesc's snapshot for visibility checks in
the scans, but the active snapshot for evaluating quals, for
example. This could use some refactoring and cleanup, but for now,
just add some assertions.
Discussion: https://www.postgresql.org/message-id/5f3b9d59-0f43-419d-80ca-6d04c07cf61a@iki.fi
---
src/backend/access/table/tableam.c | 10 ----------
src/backend/executor/execMain.c | 6 ++++++
src/backend/executor/execParallel.c | 7 +++++++
src/backend/executor/nodeBitmapHeapscan.c | 17 ++---------------
src/include/access/tableam.h | 5 -----
src/include/nodes/execnodes.h | 4 ----
6 files changed, 15 insertions(+), 34 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 6ed8cca05a..e57a0b7ea3 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,16 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-void
-table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot)
-{
- Assert(IsMVCCSnapshot(snapshot));
-
- RegisterSnapshot(snapshot);
- scan->rs_snapshot = snapshot;
- scan->rs_flags |= SO_TEMP_SNAPSHOT;
-}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 940499cc61..7eb1f7d020 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -147,6 +147,9 @@ standard_ExecutorStart(QueryDesc *queryDesc, int eflags)
Assert(queryDesc != NULL);
Assert(queryDesc->estate == NULL);
+ /* caller must ensure the query's snapshot is active */
+ Assert(GetActiveSnapshot() == queryDesc->snapshot);
+
/*
* If the transaction is read-only, we need to check if any writes are
* planned to non-temporary tables. EXPLAIN is considered read-only.
@@ -319,6 +322,9 @@ standard_ExecutorRun(QueryDesc *queryDesc,
Assert(estate != NULL);
Assert(!(estate->es_top_eflags & EXEC_FLAG_EXPLAIN_ONLY));
+ /* caller must ensure the query's snapshot is active */
+ Assert(GetActiveSnapshot() == estate->es_snapshot);
+
/*
* Switch into per-query memory context
*/
diff --git a/src/backend/executor/execParallel.c b/src/backend/executor/execParallel.c
index 3f84c002dc..8c53d1834e 100644
--- a/src/backend/executor/execParallel.c
+++ b/src/backend/executor/execParallel.c
@@ -720,6 +720,13 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
shm_toc_estimate_chunk(&pcxt->estimator, dsa_minsize);
shm_toc_estimate_keys(&pcxt->estimator, 1);
+ /*
+ * InitializeParallelDSM() passes the active snapshot to the parallel
+ * worker, which uses it to set es_snapshot. Make sure we don't set
+ * es_snapshot differently in the child.
+ */
+ Assert(GetActiveSnapshot() == estate->es_snapshot);
+
/* Everyone's had a chance to ask for space, so now create the DSM. */
InitializeParallelDSM(pcxt);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 345b67649e..ca548e44eb 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -721,7 +721,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
- scanstate->pscan_len = 0;
scanstate->initialized = false;
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
@@ -841,13 +840,7 @@ void
ExecBitmapHeapEstimate(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
- EState *estate = node->ss.ps.state;
-
- node->pscan_len = add_size(offsetof(ParallelBitmapHeapState,
- phs_snapshot_data),
- EstimateSnapshotSpace(estate->es_snapshot));
-
- shm_toc_estimate_chunk(&pcxt->estimator, node->pscan_len);
+ shm_toc_estimate_chunk(&pcxt->estimator, sizeof(ParallelBitmapHeapState));
shm_toc_estimate_keys(&pcxt->estimator, 1);
}
@@ -862,14 +855,13 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt)
{
ParallelBitmapHeapState *pstate;
- EState *estate = node->ss.ps.state;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/* If there's no DSA, there are no workers; initialize nothing. */
if (dsa == NULL)
return;
- pstate = shm_toc_allocate(pcxt->toc, node->pscan_len);
+ pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
pstate->tbmiterator = 0;
pstate->prefetch_iterator = 0;
@@ -881,7 +873,6 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
- SerializeSnapshot(estate->es_snapshot, pstate->phs_snapshot_data);
shm_toc_insert(pcxt->toc, node->ss.ps.plan->plan_node_id, pstate);
node->pstate = pstate;
@@ -927,13 +918,9 @@ ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt)
{
ParallelBitmapHeapState *pstate;
- Snapshot snapshot;
Assert(node->ss.ps.state->es_query_dsa != NULL);
pstate = shm_toc_lookup(pwcxt->toc, node->ss.ps.plan->plan_node_id, false);
node->pstate = pstate;
-
- snapshot = RestoreSnapshot(pstate->phs_snapshot_data);
- table_scan_update_snapshot(node->ss.ss_currentScanDesc, snapshot);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5f8474871d..8249b37bbf 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -1038,11 +1038,6 @@ table_rescan_set_params(TableScanDesc scan, struct ScanKeyData *key,
allow_pagemode);
}
-/*
- * Update snapshot used by the scan.
- */
-extern void table_scan_update_snapshot(TableScanDesc scan, Snapshot snapshot);
-
/*
* Return next tuple from `scan`, store in slot.
*/
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 444a5f0fd5..27614ab50f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1689,7 +1689,6 @@ typedef enum
* prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
- * phs_snapshot_data snapshot data shared to workers
* ----------------
*/
typedef struct ParallelBitmapHeapState
@@ -1701,7 +1700,6 @@ typedef struct ParallelBitmapHeapState
int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
- char phs_snapshot_data[FLEXIBLE_ARRAY_MEMBER];
} ParallelBitmapHeapState;
/* ----------------
@@ -1721,7 +1719,6 @@ typedef struct ParallelBitmapHeapState
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
- * pscan_len size of the shared memory for parallel bitmap
* initialized is node is ready to iterate
* shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
@@ -1745,7 +1742,6 @@ typedef struct BitmapHeapScanState
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
- Size pscan_len;
bool initialized;
TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
--
2.39.2
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-14 10:55 ` Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 1 reply; 248+ messages in thread
From: Dilip Kumar @ 2024-03-14 10:55 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Robert Haas <robertmhaas@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On Thu, Mar 14, 2024 at 4:07 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
> > Yeah, that's a very valid point. So I think now Heikki/Melanie might
> > have got an answer to their question, about the thought process behind
> > serializing the snapshot for each scan node. And the same thing is
> > followed for BitmapHeapNode as well.
>
> I see. Thanks, understanding the thought process helps.
>
> So when a parallel table or index scan runs in the executor as part of a
> query, we could just use the active snapshot. But there are some other
> callers of parallel table scans that don't use the executor, namely
> parallel index builds. For those it makes sense to pass the snapshot for
> the scan independent of the active snapshot.
Right
> A parallel bitmap heap scan isn't really a parallel scan as far as the
> table AM is concerned, though. It's more like an independent bitmap heap
> scan in each worker process, nodeBitmapHeapscan.c does all the
> coordination of which blocks to scan. So I think that
> table_parallelscan_initialize() was the wrong role model, and we should
> still remove the snapshot serialization code from nodeBitmapHeapscan.c.
I think that seems right.
> Digging deeper into the question of whether es_snapshot ==
> GetActiveSnapshot() is a valid assumption:
>
> <deep dive>
>
> es_snapshot is copied from the QueryDesc in standard_ExecutorStart().
> Looking at the callers of ExecutorStart(), they all get the QueryDesc by
> calling CreateQueryDesc() with GetActiveSnapshot(). And I don't see any
> callers changing the active snapshot between the ExecutorStart() and
> ExecutorRun() calls either. In pquery.c, we explicitly
> PushActiveSnapshot(queryDesc->snapshot) before calling ExecutorRun(). So
> no live bug here AFAICS, es_snapshot == GetActiveSnapshot() holds.
>
> _SPI_execute_plan() has code to deal with the possibility that the
> active snapshot is not set. That seems fishy; do we really support SPI
> without any snapshot? I'm inclined to turn that into an error. I ran the
> regression tests with an "Assert(ActiveSnapshotSet())" there, and
> everything worked.
IMHO, we can call SPI_Connect() and SPI_Execute() from any C
extension, so I don't think there we can guarantee that the snapshot
must be set, do we?
> If es_snapshot was different from the active snapshot, things would get
> weird, even without parallel query. The scans would use es_snapshot for
> the visibility checks, but any functions you execute in quals would use
> the active snapshot.
>
> We could double down on that assumption, and remove es_snapshot
> altogether and use GetActiveSnapshot() instead. And perhaps add
> "PushActiveSnapshot(queryDesc->snapshot)" to ExecutorRun().
>
> </deep dive>
>
> In summary, this es_snapshot stuff is a bit confusing and could use some
> cleanup. But for now, I'd like to just add some assertions and a
> comments about this, and remove the snapshot serialization from bitmap
> heap scan node, to make it consistent with other non-parallel scan nodes
> (it's not really a parallel scan as far as the table AM is concerned).
> See attached patch, which is the same as previous patch with some extra
> assertions.
Maybe for now we can just handle this specific case to remove the
snapshot serializing for the BitmapHeapScan as you are doing in the
patch. After looking into the code your theory seems correct that we
are just copying the ActiveSnapshot while building the query
descriptor and from there we are copying into the Estate so logically
there should not be any reason for these two to be different.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
@ 2024-03-14 13:32 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Heikki Linnakangas @ 2024-03-14 13:32 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On 14/03/2024 12:55, Dilip Kumar wrote:
> On Thu, Mar 14, 2024 at 4:07 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> _SPI_execute_plan() has code to deal with the possibility that the
>> active snapshot is not set. That seems fishy; do we really support SPI
>> without any snapshot? I'm inclined to turn that into an error. I ran the
>> regression tests with an "Assert(ActiveSnapshotSet())" there, and
>> everything worked.
>
> IMHO, we can call SPI_Connect() and SPI_Execute() from any C
> extension, so I don't think there we can guarantee that the snapshot
> must be set, do we?
I suppose, although the things you could do without a snapshot would be
pretty limited. The query couldn't access any tables. Could it even look
up functions in the parser? Not sure.
> Maybe for now we can just handle this specific case to remove the
> snapshot serializing for the BitmapHeapScan as you are doing in the
> patch. After looking into the code your theory seems correct that we
> are just copying the ActiveSnapshot while building the query
> descriptor and from there we are copying into the Estate so logically
> there should not be any reason for these two to be different.
Ok, committed that for now. Thanks for looking!
--
Heikki Linnakangas
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-14 18:16 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-21 14:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 2 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-03-14 18:16 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On Thu, Mar 14, 2024 at 03:32:04PM +0200, Heikki Linnakangas wrote:
> On 14/03/2024 12:55, Dilip Kumar wrote:
> > On Thu, Mar 14, 2024 at 4:07 PM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> > > _SPI_execute_plan() has code to deal with the possibility that the
> > > active snapshot is not set. That seems fishy; do we really support SPI
> > > without any snapshot? I'm inclined to turn that into an error. I ran the
> > > regression tests with an "Assert(ActiveSnapshotSet())" there, and
> > > everything worked.
> >
> > IMHO, we can call SPI_Connect() and SPI_Execute() from any C
> > extension, so I don't think there we can guarantee that the snapshot
> > must be set, do we?
>
> I suppose, although the things you could do without a snapshot would be
> pretty limited. The query couldn't access any tables. Could it even look up
> functions in the parser? Not sure.
>
> > Maybe for now we can just handle this specific case to remove the
> > snapshot serializing for the BitmapHeapScan as you are doing in the
> > patch. After looking into the code your theory seems correct that we
> > are just copying the ActiveSnapshot while building the query
> > descriptor and from there we are copying into the Estate so logically
> > there should not be any reason for these two to be different.
>
> Ok, committed that for now. Thanks for looking!
Attached v6 is rebased over your new commit. It also has the "fix" in
0010 which moves BitmapAdjustPrefetchIterator() back above
table_scan_bitmap_next_block(). I've also updated the Streaming Read API
commit (0013) to Thomas' v7 version from [1]. This has the update that
we theorize should address some of the regressions in the bitmapheapscan
streaming read user in 0014.
- Melanie
[1] https://www.postgresql.org/message-id/CA%2BhUKGLJi%2Bc5jB3j6UvkgMYHky-qu%2BLPCsiNahUGSa5Z4DvyVA%40ma...
Attachments:
[text/x-diff] v6-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/2-v6-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From e05474410a9771479232f25b21f11222ccabcb21 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v6 01/14] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index e57a0b7ea31..e78d793f69c 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index ca548e44eb4..6d756001fdb 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v6-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/3-v6-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From eb824b3cc826fc722ea07eeec45dfcd124aef88b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v6 02/14] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 6d756001fdb..05cf610f9b4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v6-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (14.9K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/4-v6-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From 95e4f932bd9debe72d257498d5a61d289e3e5c76 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v6 03/14] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 123 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 100 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 34bc60f625f..e7bed84f75b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -951,6 +951,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1039,6 +1041,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1058,6 +1066,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 680a50bf8b1..c9b9b4c00f1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2122,6 +2123,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, and the bitmap entries don't need rechecking, and all tuples on
+ * the page are visible to our transaction.
+ */
+ if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2234,6 +2253,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 05cf610f9b4..df382580ac1 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
+ extra_flags |= SO_CAN_SKIP_FETCH;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,8 +211,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -228,32 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!table_scan_bitmap_next_block(scan, tbmres))
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +278,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +476,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +528,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +598,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +607,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +651,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +694,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +704,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..3dfb19ec7d5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8249b37bbf1..c43a8b3dea5 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed.
+ */
+ SO_CAN_SKIP_FETCH = 1 << 10,
} ScanOptions;
/*
@@ -944,9 +951,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 27614ab50fb..4e5aab8472f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1709,10 +1709,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1732,9 +1729,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v6-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (2.2K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/5-v6-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From ed046b2ea088b8294aba1046086a9cf89c4224a7 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v6 04/14] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index df382580ac1..6a34f10646a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -470,15 +470,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
-
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -529,7 +523,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v6-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/6-v6-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From 0f94cdd27e2b4a7550f93de2d59398d47392ae3c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v6 05/14] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 6a34f10646a..61ffc8d0ee8 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -228,7 +228,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
if (!table_scan_bitmap_next_block(scan, tbmres))
{
@@ -337,7 +337,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -356,7 +356,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v6-0006-EXPLAIN-Bitmap-table-scan-also-count-no-visible-t.patch (3.0K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/7-v6-0006-EXPLAIN-Bitmap-table-scan-also-count-no-visible-t.patch)
download | inline diff:
From 2a783e5ce456695ce5ab6da02eab151b655c9597 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 18:35:28 -0500
Subject: [PATCH v6 06/14] EXPLAIN Bitmap table scan also count no visible
tuple pages
Previously, bitmap heap scans only counted lossy and exact pages for
explain when there was at least one visible tuple on the page.
heapam_scan_bitmap_next_block() returned true only if there was a
"valid" page with tuples to be processed. However, the lossy and exact
page counters in EXPLAIN should count the number of pages represented in
a lossy or non-lossy way in the constructed bitmap, regardless of
whether or not the pages ultimately contained visible tuples.
---
src/backend/executor/nodeBitmapHeapscan.c | 15 ++++++++++-----
src/test/regress/expected/partition_prune.out | 4 +++-
2 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 61ffc8d0ee8..38b4596a775 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,6 +211,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
+ bool valid;
+
CHECK_FOR_INTERRUPTS();
/*
@@ -230,17 +232,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- if (!table_scan_bitmap_next_block(scan, tbmres))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
+ valid = table_scan_bitmap_next_block(scan, tbmres);
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
+ if (!valid)
+ {
+ /* AM doesn't think this block is valid, skip */
+ continue;
+ }
+
+
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index bf0657b9f2c..9c20a24982f 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2848,6 +2848,7 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a = ab_a1.a;
Index Cond: (a = 1)
-> Bitmap Heap Scan on ab_a1_b3 ab_a1_3 (actual rows=0 loops=1)
Recheck Cond: (a = 1)
+ Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1 loops=1)
Index Cond: (a = 1)
-> Materialize (actual rows=1 loops=1)
@@ -2863,9 +2864,10 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a = ab_a1.a;
Index Cond: (a = 1)
-> Bitmap Heap Scan on ab_a1_b3 ab_3 (actual rows=0 loops=1)
Recheck Cond: (a = 1)
+ Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1 loops=1)
Index Cond: (a = 1)
-(34 rows)
+(36 rows)
table ab;
a | b
--
2.40.1
[text/x-diff] v6-0007-table_scan_bitmap_next_block-returns-lossy-or-exa.patch (4.4K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/8-v6-0007-table_scan_bitmap_next_block-returns-lossy-or-exa.patch)
download | inline diff:
From 8a0ec9d8b58fb1b1aaa74254ebe046bd85cf5614 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v6 07/14] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c9b9b4c00f1..10c1c3b616b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2112,7 +2112,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2240,6 +2241,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 38b4596a775..08e346ae7b3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index c43a8b3dea5..f1d0d4b78e3 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -796,6 +796,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -811,7 +814,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1951,14 +1955,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1969,7 +1975,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v6-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/9-v6-0008-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From 466846b691f5cfd3928db6b6193699f5bbeb0b35 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v6 08/14] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 08e346ae7b3..248ff90904b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v6-0009-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (4.1K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/10-v6-0009-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From 3734de02a94812b88317c873de5b46ab835de620 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v6 09/14] Remove table_scan_bitmap_next_tuple parameter tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 10c1c3b616b..a1ec50ab7a8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2248,7 +2248,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 248ff90904b..04ad14f70b3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -287,7 +287,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index f1d0d4b78e3..e35bd36e710 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -787,10 +787,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -821,15 +818,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -1988,7 +1980,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2000,7 +1991,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v6-0010-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/11-v6-0010-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 292a2c77fd967eacb34a669f13aa00bbc83d1a11 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v6 10/14] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 194 insertions(+), 150 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index a1ec50ab7a8..e038e60cd8f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2112,18 +2112,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, and the bitmap entries don't need rechecking, and all tuples on
@@ -2142,16 +2175,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2243,7 +2267,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 04ad14f70b3..0924613b247 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,56 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
- {
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -273,45 +232,59 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can validate that the prefetch block stays ahead of
+ * the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ Assert(node->pstate == NULL ||
+ node->prefetch_iterator == NULL ||
+ node->pfblockno > node->blockno);
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -337,13 +310,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -357,14 +334,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -389,7 +369,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -467,6 +450,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -524,6 +508,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
!tbmpre->recheck &&
@@ -585,12 +571,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -598,13 +580,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -635,28 +617,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -689,8 +667,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -698,9 +674,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..92b829cebc7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e35bd36e710..d214abeb201 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -780,19 +781,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -811,8 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -949,9 +945,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1011,6 +1011,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1021,6 +1036,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1944,19 +1974,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1966,8 +1994,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 4e5aab8472f..9ec7eaceeb7 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1707,8 +1707,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1717,9 +1715,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1727,8 +1727,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1737,9 +1735,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v6-0011-Hard-code-TBMIterateResult-offsets-array-size.patch (5.3K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/12-v6-0011-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From e71b4d941eebc3fb4c8589c16004417c4a856844 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v6 11/14] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 29 +++++++----------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fcc..d2bf8f44d50 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1467,8 +1453,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
* Create the TBMSharedIterator struct, with enough trailing space to
* serve the needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v6-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.7K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/13-v6-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From 562a1dd739dfef88c8dc392635b242c6ecc88af3 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v6 12/14] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 38 ++++++------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 108 insertions(+), 91 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3aa457a29e1 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e038e60cd8f..022753e203a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2119,7 +2119,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2132,11 +2132,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2151,11 +2151,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2163,19 +2163,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* the page are visible to our transaction.
*/
if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2204,7 +2204,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2213,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2265,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ *lossy = tbmres.ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 0924613b247..b3b9448627b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -320,7 +320,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -334,8 +334,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -370,8 +370,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -439,10 +439,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -450,7 +452,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -459,13 +461,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -480,7 +482,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -499,8 +501,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -508,17 +510,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index d2bf8f44d50..7d038c2018d 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae1..3b432263bb0 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v6-0013-v7-Streaming-Read-API.patch (56.1K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/14-v6-0013-v7-Streaming-Read-API.patch)
download | inline diff:
From 0ee4d860f46367a0eb005065a9df060564824f67 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:59:42 -0400
Subject: [PATCH v6 13/14] v7 Streaming Read API
---
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 659 +++++++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 642 +++++++++++++++-------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/include/storage/bufmgr.h | 45 ++
src/include/storage/streaming_read.h | 52 ++
src/tools/pgindent/typedefs.list | 3 +
10 files changed, 1227 insertions(+), 210 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..bcab44c802f
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..39aef2a84a2
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 00000000000..d5c29b750d9
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,659 @@
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ */
+typedef struct PgStreamingReadRange
+{
+ bool need_wait;
+ bool advice_issued;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index;
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+ ReadBuffersOperation operation;
+} PgStreamingReadRange;
+
+/*
+ * Streaming read object.
+ */
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int next_tail_buffer;
+ int distance;
+ bool started;
+ bool finished;
+ bool advice_enabled;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ /* Sometimes we need to buffer one block for flow control. */
+ BlockNumber unget_blocknum;
+ void *unget_per_buffer_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /*
+ * Make sure our bmr's smgr and persistent are populated. The caller
+ * asserts that the storage manager will remain valid.
+ */
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+
+ /*
+ * Choose a maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least
+ * MAX_BUFFER_PER_TRANSFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+ pgsr->callback = next_block_cb;
+ pgsr->bmr = bmr;
+ pgsr->forknum = forknum;
+
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out doing full-sized
+ * reads.
+ */
+ if (flags & PGSR_FLAG_FULL)
+ pgsr->distance = Min(MAX_BUFFERS_PER_TRANSFER, pgsr->max_pinned_buffers);
+ else
+ pgsr->distance = 1;
+
+ /*
+ * Space for the callback to store extra data along with each block. Note
+ * that we need one more than max_pinned_buffers, so we can return a
+ * pointer to a slot that can't be overwritten until the next call.
+ */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * size);
+
+ return pgsr;
+}
+
+/*
+ * Find the per-buffer data index for the Nth block of a range.
+ */
+static int
+get_per_buffer_data_index(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ int result;
+
+ /*
+ * Find slot in the circular buffer of per-buffer data, without using the
+ * expensive % operator.
+ */
+ result = range->per_buffer_data_index + n;
+ while (result >= pgsr->size)
+ result -= pgsr->size;
+ Assert(result == (range->per_buffer_data_index + n) % pgsr->size);
+
+ return result;
+}
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static void *
+get_per_buffer_data_by_index(PgStreamingRead *pgsr, int per_buffer_data_index)
+{
+ return (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * per_buffer_data_index;
+}
+
+/*
+ * Return a pointer to the per-buffer data for the Nth block of a range.
+ */
+static void *
+get_per_buffer_data(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ return get_per_buffer_data_by_index(pgsr,
+ get_per_buffer_data_index(pgsr,
+ range,
+ n));
+}
+
+/*
+ * Start reading the head range, and create a new head range. The new head
+ * range is returned. It may not be empty, if StartReadBuffers() couldn't
+ * start the entire range; in that case the returned range contains the
+ * remaining portion of the range.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_start_head_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+ PgStreamingReadRange *new_head_range;
+ int nblocks_pinned;
+ int flags;
+
+ /* Caller should make sure we never exceed max_ios. */
+ Assert((pgsr->ios_in_progress < pgsr->max_ios) ||
+ (pgsr->ios_in_progress == 0 && pgsr->max_ios == 0));
+
+ /* Should only call if the head range has some blocks to read. */
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If advice hasn't been suppressed, and this system supports it, this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (pgsr->advice_enabled &&
+ pgsr->max_ios > 0 &&
+ pgsr->started &&
+ head_range->blocknum != pgsr->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* Suppress advice on the first call, because it's too late to benefit. */
+ if (!pgsr->started)
+ pgsr->started = true;
+
+ /* We shouldn't be trying to pin more buffers that we're allowed to. */
+ Assert(pgsr->pinned_buffers + head_range->nblocks <= pgsr->max_pinned_buffers);
+
+ /* Start reading as many blocks as we can from the head range. */
+ nblocks_pinned = head_range->nblocks;
+ head_range->need_wait =
+ StartReadBuffers(pgsr->bmr,
+ head_range->buffers,
+ pgsr->forknum,
+ head_range->blocknum,
+ &nblocks_pinned,
+ pgsr->strategy,
+ flags,
+ &head_range->operation);
+
+ Assert(pgsr->pinned_buffers <= pgsr->max_pinned_buffers);
+
+ if (head_range->need_wait && (flags & READ_BUFFERS_ISSUE_ADVICE))
+ {
+ /*
+ * Since we've issued advice, we count an I/O in progress until we
+ * call WaitReadBuffers().
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ Assert(pgsr->ios_in_progress <= pgsr->max_ios);
+ }
+
+ /*
+ * StartReadBuffers() might have pinned fewer blocks than we asked it to,
+ * but always at least one.
+ */
+ Assert(nblocks_pinned <= head_range->nblocks);
+ Assert(nblocks_pinned >= 1);
+ pgsr->pinned_buffers += nblocks_pinned;
+
+ /*
+ * Remember where the next block would be after that, so we can detect
+ * sequential access next time.
+ */
+ pgsr->seq_blocknum = head_range->blocknum + nblocks_pinned;
+
+ /*
+ * Create a new head range. There must be space, because we have enough
+ * elements for every range to hold just one block, up to the pin limit.
+ */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ new_head_range = &pgsr->ranges[pgsr->head];
+ new_head_range->nblocks = 0;
+ new_head_range->advice_issued = false;
+
+ /*
+ * If we didn't manage to start the whole read above, we split the range,
+ * moving the remainder into the new head range.
+ */
+ if (nblocks_pinned < head_range->nblocks)
+ {
+ int nblocks_remaining = head_range->nblocks - nblocks_pinned;
+
+ head_range->nblocks = nblocks_pinned;
+
+ new_head_range->blocknum = head_range->blocknum + nblocks_pinned;
+ new_head_range->nblocks = nblocks_remaining;
+ }
+
+ /* The new range has per-buffer data starting after the previous range. */
+ new_head_range->per_buffer_data_index =
+ get_per_buffer_data_index(pgsr, head_range, nblocks_pinned);
+
+ return new_head_range;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow pg_streaming_unget_block() to work.
+ */
+static BlockNumber
+pg_streaming_get_block(PgStreamingRead *pgsr, void *per_buffer_data)
+{
+ BlockNumber result;
+
+ if (unlikely(pgsr->unget_blocknum != InvalidBlockNumber))
+ {
+ /*
+ * If we had to unget a block, now it is time to return that one
+ * again.
+ */
+ result = pgsr->unget_blocknum;
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+ /*
+ * The same per_buffer_data element must have been used, and still
+ * contains whatever data the callback wrote into it. So we just
+ * sanity-check that we were called with the value that
+ * pg_streaming_unget_block() pushed back.
+ */
+ Assert(per_buffer_data == pgsr->unget_per_buffer_data);
+ }
+ else
+ {
+ /* Use the installed callback directly. */
+ result = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ }
+
+ return result;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later. This *must* be called with the
+ * last value returned by pg_streaming_get_block().
+ */
+static void
+pg_streaming_unget_block(PgStreamingRead *pgsr, BlockNumber blocknum, void *per_buffer_data)
+{
+ Assert(pgsr->unget_blocknum == InvalidBlockNumber);
+ pgsr->unget_blocknum = blocknum;
+ pgsr->unget_per_buffer_data = per_buffer_data;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *range;
+
+ /* If we're finished, don't look ahead. */
+ if (pgsr->finished)
+ return;
+
+ /*
+ * We we've already started the maximum allowed number of I/Os, don't look
+ * ahead. There is a special case for max_ios == 0.
+ */
+ if (pgsr->max_ios > 0 && pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /* Can't pin any more buffers. */
+ if (pgsr->pinned_buffers == pgsr->distance)
+ return;
+
+ /*
+ * Keep trying to add new blocks to the end of the head range while doing
+ * so wouldn't exceed the distance limit.
+ */
+ range = &pgsr->ranges[pgsr->head];
+ while (pgsr->pinned_buffers + range->nblocks < pgsr->distance)
+ {
+ BlockNumber blocknum;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ if (range->nblocks == lengthof(range->buffers))
+ {
+ /* Start as much of it as we can. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /* If we're now at the I/O limit, stop here. */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * That might have only been partially started, but always
+ * processes at least one so that'll do for now.
+ */
+ Assert(range->nblocks < lengthof(range->buffers));
+ }
+
+ /* Find per-buffer data slot for the next block. */
+ per_buffer_data = get_per_buffer_data(pgsr, range, range->nblocks);
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pg_streaming_get_block(pgsr, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ pgsr->finished = true;
+ break;
+ }
+
+ /*
+ * Is there a head range that we cannot extend, because the requested
+ * block is not consecutive?
+ */
+ if (range->nblocks > 0 &&
+ range->blocknum + range->nblocks != blocknum)
+ {
+ /* Yes. Start it, so we can begin building a new one. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * It's possible that it was only partially started, and we have a
+ * new range with the remainder. Keep starting I/Os until we get
+ * it all out of the way, or we hit the I/O limit.
+ */
+ while (range->nblocks > 0 && pgsr->ios_in_progress < pgsr->max_ios)
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * We do have to worry about I/O capacity running out if the head
+ * range was split. In that case we have to 'unget' the block
+ * returned by the callback.
+ */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ {
+ pg_streaming_unget_block(pgsr, blocknum, per_buffer_data);
+ return;
+ }
+ }
+
+ /* If we have a new, empty range, initialize the start block. */
+ if (range->nblocks == 0)
+ range->blocknum = blocknum;
+
+ /* This block extends the range by one. */
+ Assert(range->blocknum + range->nblocks == blocknum);
+ range->nblocks++;
+ };
+
+ /*
+ * Normally we don't start the head range, preferring to give it a chance
+ * to grow to full size once more buffers have been consumed. In cases
+ * where that can't possibly happen, we might as well start the read
+ * immediately.
+ */
+ if ((range->nblocks > 0 && pgsr->finished) ||
+ (range->nblocks == pgsr->distance))
+ pg_streaming_read_start_head_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ PgStreamingReadRange *tail_range;
+
+ for (;;)
+ {
+ if (pgsr->tail != pgsr->head)
+ {
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to wait for a ReadBuffers operation to finish before
+ * returning the buffers in this range?
+ */
+ if (tail_range->need_wait)
+ {
+ int distance;
+
+ Assert(pgsr->next_tail_buffer == 0);
+ WaitReadBuffers(&tail_range->operation);
+ tail_range->need_wait = false;
+
+ /*
+ * We don't really know if the kernel generated a physical I/O
+ * when we issued advice, let alone when it finished, but it
+ * has certainly finished now because we've performed the
+ * read.
+ */
+ if (tail_range->advice_issued)
+ {
+
+ Assert(pgsr->ios_in_progress > 0);
+ pgsr->ios_in_progress--;
+
+ /*
+ * Look-ahead distance ramps up rapidly if we're issuing
+ * advice, so we can search for new more I/Os to start.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ else
+ {
+ /*
+ * There is no point in increasing look-ahead distance if
+ * we've already reached the full I/O size, since we're
+ * not issuing advice. Extra distance would only pin more
+ * buffers for no benefit.
+ */
+ if (pgsr->distance > MAX_BUFFERS_PER_TRANSFER)
+ {
+ /*
+ * Look-ahead distance gradually decays to full I/O
+ * size.
+ */
+ pgsr->distance--;
+ }
+ else
+ {
+ /*
+ * Look-ahead distance ramps up rapidly, but not more
+ * that the full I/O size.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, MAX_BUFFERS_PER_TRANSFER);
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ }
+ }
+ else if (pgsr->next_tail_buffer == 0)
+ {
+ /* No I/O necessary. Look-ahead distance gradually decays. */
+ if (pgsr->distance > 1)
+ pgsr->distance--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(pgsr, tail_range, buffer_index);
+
+ /* We may be able to get another I/O started. */
+ pg_streaming_read_look_ahead(pgsr);
+
+ return buffer;
+ }
+
+ /* Advance tail to next range. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+ else
+ {
+ /*
+ * If tail crashed into head, and head is not empty, then it is
+ * time to start that range. Otherwise, force a look-ahead, to
+ * kick start the stream.
+ */
+ Assert(pgsr->tail == pgsr->head);
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ {
+ pg_streaming_read_start_head_range(pgsr);
+ }
+ else
+ {
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* Finished? */
+ if (pgsr->tail == pgsr->head &&
+ pgsr->ranges[pgsr->head].nblocks == 0)
+ break;
+ }
+ }
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ pgsr->finished = true;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(pgsr->pinned_buffers == 0);
+ Assert(pgsr->ios_in_progress == 0);
+
+ /* Release memory. */
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c5..d0e9c7deffa 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -471,10 +476,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +504,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +785,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +797,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +819,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +990,68 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
+/*
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
+ */
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* ReadBuffer_common -- common logic for all ReadBuffer variants
*
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+ BufferAccessStrategy strategy)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int nblocks;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1070,405 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ nblocks = 1;
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
+ else
+ flags = 0;
+ if (StartReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ &nblocks,
+ strategy,
+ flags,
+ &operation))
+ WaitReadBuffers(&operation);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ ZeroBuffer(buffer, mode);
+
+ return buffer;
+}
+static Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested.
+ *
+ * If false is returned, no I/O is necessary and WaitReadBuffers() is not
+ * necessary. If true is returned, one I/O has been started, and
+ * WaitReadBuffers() must be called with the same operation object before the
+ * buffers are accessed. Along with the operation object, the caller-supplied
+ * array of buffers must remain valid until WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+bool
+StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation)
+{
+ int actual_nblocks = *nblocks;
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ operation->bmr = bmr;
+ operation->forknum = forkNum;
+ operation->blocknum = blockNum;
+ operation->buffers = buffers;
+ operation->nblocks = actual_nblocks;
+ operation->strategy = strategy;
+ operation->flags = flags;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ operation->io_buffers_len = 0;
- /*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
- */
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
- else
+ for (int i = 0; i < actual_nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ bool found;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ buffers[i] = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum + i,
+ strategy,
+ &found);
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = operation->nblocks = *nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ operation->io_buffers_len++;
+ }
+ }
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (operation->io_buffers_len > 0)
+ {
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
- {
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
+ /*
+ * In theory we should only do this if PrepareReadBuffers() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(bmr.smgr, forkNum, blockNum, operation->io_buffers_len);
}
+
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
}
+ else
+ {
+ return false;
+ }
+}
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
{
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ /*
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
+ */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+ bmr = operation->bmr;
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
-
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
}
else
{
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
}
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ /*
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
+ */
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
+ else
+ pgBufferUsage.shared_blks_read += nblocks;
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ for (int i = 0; i < nblocks; ++i)
+ {
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- return BufferDescriptorGetBuffer(bufHdr);
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
+
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
+ {
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
+ {
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
+ }
+ else
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
+
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
+
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
+
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
+
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for StartReadBuffers. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1476,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1535,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1603,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1638,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +1994,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2259,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2282,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2372,7 +2597,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2611,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3679,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5184,9 +5414,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5435,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a57..985a2c7049c 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..b57f71f97e3 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,42 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+/*
+ * Private state used by StartReadBuffers() and WaitReadBuffers(). Declared
+ * in public header only to allow inclusion in other structs, but contents
+ * should not be accessed.
+ */
+struct ReadBuffersOperation
+{
+ /* Parameters passed in to StartReadBuffers(). */
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ int nblocks;
+ BufferAccessStrategy strategy;
+ int flags;
+
+ /* Range of buffers, if we need to perform a read. */
+ int io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +292,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 00000000000..c4d3892bb26
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,52 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define PGSR_FLAG_FULL 0x04
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index aa7a25b8f8c..8b5c2a1576d 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2099,6 +2099,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
@@ -2269,6 +2271,7 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
--
2.40.1
[text/x-diff] v6-0014-BitmapHeapScan-uses-streaming-read-API.patch (28.5K, ../../20240314181625.a7uigo5ujaogfd6x@liskov/15-v6-0014-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From 42475042baa09b3fd3ddb8f39f7a6304c49449de Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:04:18 -0500
Subject: [PATCH v6 14/14] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
---
src/backend/access/heap/heapam.c | 68 +++++
src/backend/access/heap/heapam_handler.c | 88 +++---
src/backend/executor/nodeBitmapHeapscan.c | 352 +---------------------
src/include/access/heapam.h | 4 +
src/include/access/tableam.h | 22 +-
src/include/nodes/execnodes.h | 22 --
6 files changed, 116 insertions(+), 440 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e7bed84f75b..0f370bfec3e 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -111,6 +111,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -331,6 +333,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -951,6 +969,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
@@ -1089,6 +1108,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10246,3 +10268,49 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hdesc->rs_base.shared_tbmiterator)
+ tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
+ else
+ tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 022753e203a..9727613e87f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2111,79 +2111,65 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, and the bitmap entries don't need rechecking, and all tuples on
- * the page are visible to our transaction.
- */
- if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2204,7 +2190,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2199,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2251,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres.ntuples < 0;
+ *lossy = tbmres->ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index b3b9448627b..c437718f980 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -86,14 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -109,15 +97,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
}
else
{
@@ -140,20 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
- }
-#endif
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
@@ -161,14 +126,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -212,37 +169,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -264,27 +190,13 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
new_page:
-
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy))
break;
if (lossy)
node->lossy_pages++;
else
node->exact_pages++;
-
- /*
- * If serial, we can validate that the prefetch block stays ahead of
- * the current block.
- */
- Assert(node->pstate == NULL ||
- node->prefetch_iterator == NULL ||
- node->pfblockno > node->blockno);
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -308,225 +220,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- tbm_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -572,23 +265,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -627,16 +309,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -669,18 +345,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -720,13 +389,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -803,14 +465,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -841,11 +499,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d5..1cad9c04f01 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index d214abeb201..5a963f9293f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -792,23 +792,11 @@ typedef struct TableAmRoutine
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool (*scan_bitmap_next_block) (TableScanDesc scan, bool *recheck,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1983,8 +1971,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* used after verifying the presence (at plan time or such).
*/
static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+table_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1994,8 +1981,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, lossy);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 9ec7eaceeb7..6a48f583d7c 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1682,11 +1682,8 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
* mutex mutual exclusion for the prefetching variable
* and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1694,10 +1691,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
@@ -1707,19 +1701,11 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1727,19 +1713,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-14 21:26 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-14 21:26 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/14/24 19:16, Melanie Plageman wrote:
> On Thu, Mar 14, 2024 at 03:32:04PM +0200, Heikki Linnakangas wrote:
>> ...
>>
>> Ok, committed that for now. Thanks for looking!
>
> Attached v6 is rebased over your new commit. It also has the "fix" in
> 0010 which moves BitmapAdjustPrefetchIterator() back above
> table_scan_bitmap_next_block(). I've also updated the Streaming Read API
> commit (0013) to Thomas' v7 version from [1]. This has the update that
> we theorize should address some of the regressions in the bitmapheapscan
> streaming read user in 0014.
>
Should I rerun the benchmarks with these new patches, to see if it
really helps with the regressions?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-14 21:39 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-17 19:21 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 2 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-03-14 21:39 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On Thu, Mar 14, 2024 at 5:26 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 3/14/24 19:16, Melanie Plageman wrote:
> > On Thu, Mar 14, 2024 at 03:32:04PM +0200, Heikki Linnakangas wrote:
> >> ...
> >>
> >> Ok, committed that for now. Thanks for looking!
> >
> > Attached v6 is rebased over your new commit. It also has the "fix" in
> > 0010 which moves BitmapAdjustPrefetchIterator() back above
> > table_scan_bitmap_next_block(). I've also updated the Streaming Read API
> > commit (0013) to Thomas' v7 version from [1]. This has the update that
> > we theorize should address some of the regressions in the bitmapheapscan
> > streaming read user in 0014.
> >
>
> Should I rerun the benchmarks with these new patches, to see if it
> really helps with the regressions?
That would be awesome!
I will soon send out a summary of what we investigated off-list about
0010 (though we didn't end up concluding anything). My "fix" (leaving
BitmapAdjustPrefetchIterator() above table_scan_bitmap_next_block())
eliminates the regression in 0010 on the one example that I repro'd
upthread, but it would be good to know if it eliminates the
regressions across some other tests.
I think it would be worthwhile to run the subset of tests which seemed
to fare the worst on 0010 against the patches 0001-0010-- cyclic
uncached on your xeon machine with 4 parallel workers, IIRC -- even
the 1 million scale would do the trick, I think.
And then separately run the subset of tests which seemed to do the
worst on 0014. There were several groups of issues across the
different tests, but I think that the uniform pages data test would be
relevant to use. It showed the regressions with eic 0.
As for the other regressions showing with 0014, I think we would want
to see at least one with fully-in-shared-buffers and one with fully
uncached. Some of the fixes were around pinning fewer buffers when the
blocks were already in shared buffers.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-15 21:14 ` Andres Freund <andres@anarazel.de>
2024-03-15 22:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Andres Freund @ 2024-03-15 21:14 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
Hi,
On 2024-03-14 17:39:30 -0400, Melanie Plageman wrote:
> I will soon send out a summary of what we investigated off-list about
> 0010 (though we didn't end up concluding anything). My "fix" (leaving
> BitmapAdjustPrefetchIterator() above table_scan_bitmap_next_block())
> eliminates the regression in 0010 on the one example that I repro'd
> upthread, but it would be good to know if it eliminates the
> regressions across some other tests.
I spent a good amount of time looking into this with Melanie. After a bunch of
wrong paths I think I found the issue: We end up prefetching blocks we have
already read. Notably this happens even as-is on master - just not as
frequently as after moving BitmapAdjustPrefetchIterator().
From what I can tell the prefetching in parallel bitmap heap scans is
thoroughly broken. I added some tracking of the last block read, the last
block prefetched to ParallelBitmapHeapState and found that with a small
effective_io_concurrency we end up with ~18% of prefetches being of blocks we
already read! After moving the BitmapAdjustPrefetchIterator() to rises to 86%,
no wonder it's slower...
The race here seems fairly substantial - we're moving the two iterators
independently from each other, in multiple processes, without useful locking.
I'm inclined to think this is a bug we ought to fix in the backbranches.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
@ 2024-03-15 22:42 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-16 19:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-15 22:42 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On Fri, Mar 15, 2024 at 5:14 PM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2024-03-14 17:39:30 -0400, Melanie Plageman wrote:
> > I will soon send out a summary of what we investigated off-list about
> > 0010 (though we didn't end up concluding anything). My "fix" (leaving
> > BitmapAdjustPrefetchIterator() above table_scan_bitmap_next_block())
> > eliminates the regression in 0010 on the one example that I repro'd
> > upthread, but it would be good to know if it eliminates the
> > regressions across some other tests.
>
> I spent a good amount of time looking into this with Melanie. After a bunch of
> wrong paths I think I found the issue: We end up prefetching blocks we have
> already read. Notably this happens even as-is on master - just not as
> frequently as after moving BitmapAdjustPrefetchIterator().
>
> From what I can tell the prefetching in parallel bitmap heap scans is
> thoroughly broken. I added some tracking of the last block read, the last
> block prefetched to ParallelBitmapHeapState and found that with a small
> effective_io_concurrency we end up with ~18% of prefetches being of blocks we
> already read! After moving the BitmapAdjustPrefetchIterator() to rises to 86%,
> no wonder it's slower...
>
> The race here seems fairly substantial - we're moving the two iterators
> independently from each other, in multiple processes, without useful locking.
>
> I'm inclined to think this is a bug we ought to fix in the backbranches.
Thinking about how to fix this, perhaps we could keep the current max
block number in the ParallelBitmapHeapState and then when prefetching,
workers could loop calling tbm_shared_iterate() until they've found a
block at least prefetch_pages ahead of the current block. They
wouldn't need to read the current max value from the parallel state on
each iteration. Even checking it once and storing that value in a
local variable prevented prefetching blocks after reading them in my
example repro of the issue.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-15 22:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-16 19:12 ` Andres Freund <andres@anarazel.de>
2024-03-16 20:25 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Andres Freund @ 2024-03-16 19:12 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
Hi,
On 2024-03-15 18:42:29 -0400, Melanie Plageman wrote:
> On Fri, Mar 15, 2024 at 5:14 PM Andres Freund <andres@anarazel.de> wrote:
> > On 2024-03-14 17:39:30 -0400, Melanie Plageman wrote:
> > I spent a good amount of time looking into this with Melanie. After a bunch of
> > wrong paths I think I found the issue: We end up prefetching blocks we have
> > already read. Notably this happens even as-is on master - just not as
> > frequently as after moving BitmapAdjustPrefetchIterator().
> >
> > From what I can tell the prefetching in parallel bitmap heap scans is
> > thoroughly broken. I added some tracking of the last block read, the last
> > block prefetched to ParallelBitmapHeapState and found that with a small
> > effective_io_concurrency we end up with ~18% of prefetches being of blocks we
> > already read! After moving the BitmapAdjustPrefetchIterator() to rises to 86%,
> > no wonder it's slower...
> >
> > The race here seems fairly substantial - we're moving the two iterators
> > independently from each other, in multiple processes, without useful locking.
> >
> > I'm inclined to think this is a bug we ought to fix in the backbranches.
>
> Thinking about how to fix this, perhaps we could keep the current max
> block number in the ParallelBitmapHeapState and then when prefetching,
> workers could loop calling tbm_shared_iterate() until they've found a
> block at least prefetch_pages ahead of the current block. They
> wouldn't need to read the current max value from the parallel state on
> each iteration. Even checking it once and storing that value in a
> local variable prevented prefetching blocks after reading them in my
> example repro of the issue.
That would address some of the worst behaviour, but it doesn't really seem to
address the underlying problem of the two iterators being modified
independently. ISTM the proper fix would be to protect the state of the
iterators with a single lock, rather than pushing down the locking into the
bitmap code. OTOH, we'll only need one lock going forward, so being economic
in the effort of fixing this is also important.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-15 22:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-16 19:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
@ 2024-03-16 20:25 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-17 16:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-16 20:25 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Melanie Plageman <melanieplageman@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/16/24 20:12, Andres Freund wrote:
> Hi,
>
> On 2024-03-15 18:42:29 -0400, Melanie Plageman wrote:
>> On Fri, Mar 15, 2024 at 5:14 PM Andres Freund <andres@anarazel.de> wrote:
>>> On 2024-03-14 17:39:30 -0400, Melanie Plageman wrote:
>>> I spent a good amount of time looking into this with Melanie. After a bunch of
>>> wrong paths I think I found the issue: We end up prefetching blocks we have
>>> already read. Notably this happens even as-is on master - just not as
>>> frequently as after moving BitmapAdjustPrefetchIterator().
>>>
>>> From what I can tell the prefetching in parallel bitmap heap scans is
>>> thoroughly broken. I added some tracking of the last block read, the last
>>> block prefetched to ParallelBitmapHeapState and found that with a small
>>> effective_io_concurrency we end up with ~18% of prefetches being of blocks we
>>> already read! After moving the BitmapAdjustPrefetchIterator() to rises to 86%,
>>> no wonder it's slower...
>>>
>>> The race here seems fairly substantial - we're moving the two iterators
>>> independently from each other, in multiple processes, without useful locking.
>>>
>>> I'm inclined to think this is a bug we ought to fix in the backbranches.
>>
>> Thinking about how to fix this, perhaps we could keep the current max
>> block number in the ParallelBitmapHeapState and then when prefetching,
>> workers could loop calling tbm_shared_iterate() until they've found a
>> block at least prefetch_pages ahead of the current block. They
>> wouldn't need to read the current max value from the parallel state on
>> each iteration. Even checking it once and storing that value in a
>> local variable prevented prefetching blocks after reading them in my
>> example repro of the issue.
>
> That would address some of the worst behaviour, but it doesn't really seem to
> address the underlying problem of the two iterators being modified
> independently. ISTM the proper fix would be to protect the state of the
> iterators with a single lock, rather than pushing down the locking into the
> bitmap code. OTOH, we'll only need one lock going forward, so being economic
> in the effort of fixing this is also important.
>
Can you share some details about how you identified the problem, counted
the prefetches that happen too late, etc? I'd like to try to reproduce
this to understand the issue better.
If I understand correctly, what may happen is that a worker reads blocks
from the "prefetch" iterator, but before it manages to issue the
posix_fadvise, some other worker already did pread. Or can the iterators
get "out of sync" in a more fundamental way?
If my understanding is correct, why would a single lock solve that? Yes,
we'd advance the iterators at the same time, but surely we'd not issue
the fadvise calls while holding the lock, and the prefetch/fadvise for a
particular block could still happen in different workers.
I suppose a dirty PoC fix should not be too difficult, and it'd allow us
to check if it works.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-15 22:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-16 19:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-16 20:25 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-17 16:38 ` Andres Freund <andres@anarazel.de>
2024-03-17 19:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Andres Freund @ 2024-03-17 16:38 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
Hi,
On 2024-03-16 21:25:18 +0100, Tomas Vondra wrote:
> On 3/16/24 20:12, Andres Freund wrote:
> > That would address some of the worst behaviour, but it doesn't really seem to
> > address the underlying problem of the two iterators being modified
> > independently. ISTM the proper fix would be to protect the state of the
> > iterators with a single lock, rather than pushing down the locking into the
> > bitmap code. OTOH, we'll only need one lock going forward, so being economic
> > in the effort of fixing this is also important.
> >
>
> Can you share some details about how you identified the problem, counted
> the prefetches that happen too late, etc? I'd like to try to reproduce
> this to understand the issue better.
There's two aspects. Originally I couldn't reliably reproduce the regression
with Melanie's repro on my laptop. I finally was able to do so after I
a) changed the block device's read_ahead_kb to 0
b) used effective_io_concurrency=1
That made the difference between the BitmapAdjustPrefetchIterator() locations
very significant, something like 2.3s vs 12s.
Besides a lot of other things, I finally added debugging fprintfs printing the
pid, (prefetch, read), block number. Even looking at tiny excerpts of the
large amount of output that generates shows that two iterators were out of
sync.
> If I understand correctly, what may happen is that a worker reads blocks
> from the "prefetch" iterator, but before it manages to issue the
> posix_fadvise, some other worker already did pread. Or can the iterators
> get "out of sync" in a more fundamental way?
I agree that the current scheme of two shared iterators being used has some
fairly fundamental raciness. But I suspect there's more than that going on
right now.
Moving BitmapAdjustPrefetchIterator() to later drastically increases the
raciness because it means table_scan_bitmap_next_block() happens between
increasing the "real" and the "prefetch" iterators.
An example scenario that, I think, leads to the iterators being out of sync,
without there being races between iterator advancement and completing
prefetching:
start:
real -> block 0
prefetch -> block 0
prefetch_pages = 0
prefetch_target = 1
W1: tbm_shared_iterate(real) -> block 0
W2: tbm_shared_iterate(real) -> block 1
W1: BitmapAdjustPrefetchIterator() -> tbm_shared_iterate(prefetch) -> 0
W2: BitmapAdjustPrefetchIterator() -> tbm_shared_iterate(prefetch) -> 1
W1: read block 0
W2: read block 1
W1: BitmapPrefetch() -> prefetch_pages++ -> 1, tbm_shared_iterate(prefetch) -> 2, prefetch block 2
W2: BitmapPrefetch() -> nothing, as prefetch_pages == prefetch_target
W1: tbm_shared_iterate(real) -> block 2
W2: tbm_shared_iterate(real) -> block 3
W2: BitmapAdjustPrefetchIterator() -> prefetch_pages--
W2: read block 3
W2: BitmapPrefetch() -> prefetch_pages++, tbm_shared_iterate(prefetch) -> 3, prefetch block 3
So afaict here we end up prefetching a block that the *same process* just had
read.
ISTM that the idea of somehow "catching up" in BitmapAdjustPrefetchIterator(),
separately from advancing the "real" iterator, is pretty ugly for non-parallel
BHS and just straight up broken in the parallel case.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-15 22:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-16 19:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-16 20:25 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-17 16:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
@ 2024-03-17 19:36 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 11:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-17 19:36 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/17/24 17:38, Andres Freund wrote:
> Hi,
>
> On 2024-03-16 21:25:18 +0100, Tomas Vondra wrote:
>> On 3/16/24 20:12, Andres Freund wrote:
>>> That would address some of the worst behaviour, but it doesn't really seem to
>>> address the underlying problem of the two iterators being modified
>>> independently. ISTM the proper fix would be to protect the state of the
>>> iterators with a single lock, rather than pushing down the locking into the
>>> bitmap code. OTOH, we'll only need one lock going forward, so being economic
>>> in the effort of fixing this is also important.
>>>
>>
>> Can you share some details about how you identified the problem, counted
>> the prefetches that happen too late, etc? I'd like to try to reproduce
>> this to understand the issue better.
>
> There's two aspects. Originally I couldn't reliably reproduce the regression
> with Melanie's repro on my laptop. I finally was able to do so after I
> a) changed the block device's read_ahead_kb to 0
> b) used effective_io_concurrency=1
>
> That made the difference between the BitmapAdjustPrefetchIterator() locations
> very significant, something like 2.3s vs 12s.
>
Interesting. I haven't thought about read_ahead_kb, but in hindsight it
makes sense it affects these cases. OTOH I did not set it to 0 on either
machine (the 6xSATA RAID0 has it at 12288, for example) and yet that's
how we found the regressions.
For eic it makes perfect sense that setting it to 1 is particularly
vulnerable to this issue - it only takes a small "desynchronization" of
the two iterators for the prefetch to "fall behind" and frequently
prefetch blocks we already read.
> Besides a lot of other things, I finally added debugging fprintfs printing the
> pid, (prefetch, read), block number. Even looking at tiny excerpts of the
> large amount of output that generates shows that two iterators were out of
> sync.
>
Thanks. I did experiment with fprintf, but it's quite cumbersome, so I
was hoping you came up with some smart way to trace this king of stuff.
For example I was wondering if ebpf would be a more convenient way.
>
>> If I understand correctly, what may happen is that a worker reads blocks
>> from the "prefetch" iterator, but before it manages to issue the
>> posix_fadvise, some other worker already did pread. Or can the iterators
>> get "out of sync" in a more fundamental way?
>
> I agree that the current scheme of two shared iterators being used has some
> fairly fundamental raciness. But I suspect there's more than that going on
> right now.
>
> Moving BitmapAdjustPrefetchIterator() to later drastically increases the
> raciness because it means table_scan_bitmap_next_block() happens between
> increasing the "real" and the "prefetch" iterators.
>
> An example scenario that, I think, leads to the iterators being out of sync,
> without there being races between iterator advancement and completing
> prefetching:
>
> start:
> real -> block 0
> prefetch -> block 0
> prefetch_pages = 0
> prefetch_target = 1
>
> W1: tbm_shared_iterate(real) -> block 0
> W2: tbm_shared_iterate(real) -> block 1
> W1: BitmapAdjustPrefetchIterator() -> tbm_shared_iterate(prefetch) -> 0
> W2: BitmapAdjustPrefetchIterator() -> tbm_shared_iterate(prefetch) -> 1
> W1: read block 0
> W2: read block 1
> W1: BitmapPrefetch() -> prefetch_pages++ -> 1, tbm_shared_iterate(prefetch) -> 2, prefetch block 2
> W2: BitmapPrefetch() -> nothing, as prefetch_pages == prefetch_target
>
> W1: tbm_shared_iterate(real) -> block 2
> W2: tbm_shared_iterate(real) -> block 3
>
> W2: BitmapAdjustPrefetchIterator() -> prefetch_pages--
> W2: read block 3
> W2: BitmapPrefetch() -> prefetch_pages++, tbm_shared_iterate(prefetch) -> 3, prefetch block 3
>
> So afaict here we end up prefetching a block that the *same process* just had
> read.
>
Uh, that's very weird. I'd understood if there's some cross-process
issue, but if this happens in a single process ... strange.
> ISTM that the idea of somehow "catching up" in BitmapAdjustPrefetchIterator(),
> separately from advancing the "real" iterator, is pretty ugly for non-parallel
> BHS and just straight up broken in the parallel case.
>
Yeah, I agree with the feeling it's an ugly fix. Definitely seems more
like fixing symptoms than the actual problem.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-15 22:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-16 19:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-16 20:25 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-17 16:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-17 19:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-18 11:34 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 0 replies; 248+ messages in thread
From: Tomas Vondra @ 2024-03-18 11:34 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/17/24 20:36, Tomas Vondra wrote:
>
> ...
>
>> Besides a lot of other things, I finally added debugging fprintfs printing the
>> pid, (prefetch, read), block number. Even looking at tiny excerpts of the
>> large amount of output that generates shows that two iterators were out of
>> sync.
>>
>
> Thanks. I did experiment with fprintf, but it's quite cumbersome, so I
> was hoping you came up with some smart way to trace this king of stuff.
> For example I was wondering if ebpf would be a more convenient way.
>
FWIW I just realized why I failed to identify this "late prefetch" issue
during my investigation. I was experimenting with instrumenting this by
adding a LD_PRELOAD library, logging all pread/fadvise calls. But the
FilePrefetch call is skipped in the page is already in shared buffers,
so this case "disappeared" during processing which matched the two calls
by doing an "inner join".
That being said, I think tracing this using LD_PRELOAD or perf may be
more convenient way to see what's happening. For example I ended up
doing this:
perf record -a -e syscalls:sys_enter_fadvise64 \
-e syscalls:sys_exit_fadvise64 \
-e syscalls:sys_enter_pread64 \
-e syscalls:sys_exit_pread64
perf script -ns
Alternatively, perf-trace can be used and prints the filename too (but
time has ms resolution only). Processing this seems comparable to the
fprintf approach.
It still has the issue that some of the fadvise calls may be absent if
the prefetch iterator gets too far behind, but I think that can be
detected / measured by simply counting the fadvise calls, and comparing
them to pread calls. We expect these to be about the same, so
(#pread - #fadvise) / #fadvise
is a measure of how many were "late" and skipped.
It also seems better than fprintf because it traces the actual syscalls,
not just calls to glibc wrappers. For example I saw this
postgres 54769 [001] 33768.771524828:
syscalls:sys_enter_pread64: ..., pos: 0x30d04000
postgres 54769 [001] 33768.771526867:
syscalls:sys_exit_pread64: 0x2000
postgres 54820 [000] 33768.771527473:
syscalls:sys_enter_fadvise64: ..., offset: 0x30d04000, ...
postgres 54820 [000] 33768.771528320:
syscalls:sys_exit_fadvise64: 0x0
which is clearly a case where we issue fadvise after pread of the same
block already completed.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-17 19:21 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 14:47 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-17 19:21 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/14/24 22:39, Melanie Plageman wrote:
> On Thu, Mar 14, 2024 at 5:26 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 3/14/24 19:16, Melanie Plageman wrote:
>>> On Thu, Mar 14, 2024 at 03:32:04PM +0200, Heikki Linnakangas wrote:
>>>> ...
>>>>
>>>> Ok, committed that for now. Thanks for looking!
>>>
>>> Attached v6 is rebased over your new commit. It also has the "fix" in
>>> 0010 which moves BitmapAdjustPrefetchIterator() back above
>>> table_scan_bitmap_next_block(). I've also updated the Streaming Read API
>>> commit (0013) to Thomas' v7 version from [1]. This has the update that
>>> we theorize should address some of the regressions in the bitmapheapscan
>>> streaming read user in 0014.
>>>
>>
>> Should I rerun the benchmarks with these new patches, to see if it
>> really helps with the regressions?
>
> That would be awesome!
>
OK, here's a couple charts comparing the effect of v6 patches to master.
These are from 1M and 10M data sets, same as the runs presented earlier
in this thread (the 10M is still running, but should be good enough for
this kind of visual comparison).
I have results for individual patches, but 0001-0013 behave virtually
the same, so the charts show only 0012 and 0014 (vs master).
Instead of a table with color scale (used before), I used simple scatter
plots as a more compact / concise visualization. It's impossible to
identify patterns (e.g. serial vs. parallel runs), but for the purpose
of this comparison that does not matter.
And then I'll use a chart plotting "relative" time compared to master (I
find it easier to judge the relative difference than with scatter plot).
1) absolute-all - all runs (scatter plot)
2) absolute-optimal - runs where the planner would pick bitmapscan
3) relative-all - all runs (duration relative to master)
4) relative-optimal - relative, runs where bitmapscan would be picked
The 0012 results are a pretty clear sign the "refactoring patches"
behave exactly the same as master. There are a couple outliers (in
either direction), but I'd attribute those to random noise and too few
runs to smooth it out for a particular combination (especially for 10M).
What is even more obvious is that 0014 behaves *VERY* differently. I'm
not sure if this is a good thing or a problem is debatable/unclear. I'm
sure we don't want to cause regressions, but perhaps those are due to
the prefetch issue discussed elsewhere in this thread (identified by
Andres and Melanie). There are also many cases that got much faster, but
the question is whether this is due to better efficiency or maybe the
new code being more aggressive in some way (not sure).
It's however interesting the differences are way more significant (both
in terms of frequency and scale) on the older machine with SATA SSDs.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[image/png] relative-optimal.png (118.2K, ../../84bf5689-4524-4ba1-b25e-18018045e94d@enterprisedb.com/2-relative-optimal.png)
download | view image
[image/png] relative-all.png (122.0K, ../../84bf5689-4524-4ba1-b25e-18018045e94d@enterprisedb.com/3-relative-all.png)
download | view image
[image/png] absolute-optimal.png (58.3K, ../../84bf5689-4524-4ba1-b25e-18018045e94d@enterprisedb.com/4-absolute-optimal.png)
download | view image
[image/png] absolute-all.png (69.7K, ../../84bf5689-4524-4ba1-b25e-18018045e94d@enterprisedb.com/5-absolute-all.png)
download | view image
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-17 19:21 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-18 14:47 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-18 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-18 14:47 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On Sun, Mar 17, 2024 at 3:21 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 3/14/24 22:39, Melanie Plageman wrote:
> > On Thu, Mar 14, 2024 at 5:26 PM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> >>
> >> On 3/14/24 19:16, Melanie Plageman wrote:
> >>> On Thu, Mar 14, 2024 at 03:32:04PM +0200, Heikki Linnakangas wrote:
> >>>> ...
> >>>>
> >>>> Ok, committed that for now. Thanks for looking!
> >>>
> >>> Attached v6 is rebased over your new commit. It also has the "fix" in
> >>> 0010 which moves BitmapAdjustPrefetchIterator() back above
> >>> table_scan_bitmap_next_block(). I've also updated the Streaming Read API
> >>> commit (0013) to Thomas' v7 version from [1]. This has the update that
> >>> we theorize should address some of the regressions in the bitmapheapscan
> >>> streaming read user in 0014.
> >>>
> >>
> >> Should I rerun the benchmarks with these new patches, to see if it
> >> really helps with the regressions?
> >
> > That would be awesome!
> >
>
> OK, here's a couple charts comparing the effect of v6 patches to master.
> These are from 1M and 10M data sets, same as the runs presented earlier
> in this thread (the 10M is still running, but should be good enough for
> this kind of visual comparison).
Thanks for doing this!
> What is even more obvious is that 0014 behaves *VERY* differently. I'm
> not sure if this is a good thing or a problem is debatable/unclear. I'm
> sure we don't want to cause regressions, but perhaps those are due to
> the prefetch issue discussed elsewhere in this thread (identified by
> Andres and Melanie). There are also many cases that got much faster, but
> the question is whether this is due to better efficiency or maybe the
> new code being more aggressive in some way (not sure).
Are these with the default effective_io_concurrency (1)? If so, the
"effective" prefetch distance in many cases will be higher with the
streaming read code applied. With effective_io_concurrency 1,
"max_ios" will always be 1, but the number of blocks prefetched may
exceed this (up to MAX_BUFFERS_PER_TRANSFER) because the streaming
read code is always trying to build bigger IOs. And, if prefetching,
it will prefetch IOs not yet in shared buffers before reading them.
It's hard to tell without going into a specific repro why this would
cause some queries to be much slower. In the forced bitmapheapscan, it
would make sense that more prefetching is worse -- which is why a
bitmapheapscan plan wouldn't have been chosen. But in the optimal
cases, it is unclear why it would be worse.
I don't think there is any way it could be the issue Andres
identified, because there is only one iterator. Nothing to get out of
sync. It could be that the fadvises are being issued too close to the
reads and aren't effective enough at covering up read latency on
slower, older hardware. But that doesn't explain why master would
sometimes be faster.
Probably the only thing we can do is get into a repro. It would, of
course, be easiest to do this with a serial query. I can dig into the
scripts you shared earlier and try to find a good repro. Because the
regressions may have shifted with Thomas' new version, it would help
if you shared a category (cyclic/uniform/etc, parallel or serial, eic
value, work mem, etc) where you now see the most regressions.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-17 19:21 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 14:47 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-18 15:55 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-19 20:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-18 15:55 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/18/24 15:47, Melanie Plageman wrote:
> On Sun, Mar 17, 2024 at 3:21 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 3/14/24 22:39, Melanie Plageman wrote:
>>> On Thu, Mar 14, 2024 at 5:26 PM Tomas Vondra
>>> <tomas.vondra@enterprisedb.com> wrote:
>>>>
>>>> On 3/14/24 19:16, Melanie Plageman wrote:
>>>>> On Thu, Mar 14, 2024 at 03:32:04PM +0200, Heikki Linnakangas wrote:
>>>>>> ...
>>>>>>
>>>>>> Ok, committed that for now. Thanks for looking!
>>>>>
>>>>> Attached v6 is rebased over your new commit. It also has the "fix" in
>>>>> 0010 which moves BitmapAdjustPrefetchIterator() back above
>>>>> table_scan_bitmap_next_block(). I've also updated the Streaming Read API
>>>>> commit (0013) to Thomas' v7 version from [1]. This has the update that
>>>>> we theorize should address some of the regressions in the bitmapheapscan
>>>>> streaming read user in 0014.
>>>>>
>>>>
>>>> Should I rerun the benchmarks with these new patches, to see if it
>>>> really helps with the regressions?
>>>
>>> That would be awesome!
>>>
>>
>> OK, here's a couple charts comparing the effect of v6 patches to master.
>> These are from 1M and 10M data sets, same as the runs presented earlier
>> in this thread (the 10M is still running, but should be good enough for
>> this kind of visual comparison).
>
> Thanks for doing this!
>
>> What is even more obvious is that 0014 behaves *VERY* differently. I'm
>> not sure if this is a good thing or a problem is debatable/unclear. I'm
>> sure we don't want to cause regressions, but perhaps those are due to
>> the prefetch issue discussed elsewhere in this thread (identified by
>> Andres and Melanie). There are also many cases that got much faster, but
>> the question is whether this is due to better efficiency or maybe the
>> new code being more aggressive in some way (not sure).
>
> Are these with the default effective_io_concurrency (1)? If so, the
> "effective" prefetch distance in many cases will be higher with the
> streaming read code applied. With effective_io_concurrency 1,
> "max_ios" will always be 1, but the number of blocks prefetched may
> exceed this (up to MAX_BUFFERS_PER_TRANSFER) because the streaming
> read code is always trying to build bigger IOs. And, if prefetching,
> it will prefetch IOs not yet in shared buffers before reading them.
>
No, it's a mix of runs with random combinations of these parameters:
dataset: uniform uniform_pages linear linear_fuzz cyclic cyclic_fuzz
workers: 0 4
work_mem: 128kB 4MB 64MB
eic: 0 1 8 16 32
selectivity: 0-100%
I can either share the data (~70MB of CSV) or generate charts for
results with some filter.
> It's hard to tell without going into a specific repro why this would
> cause some queries to be much slower. In the forced bitmapheapscan, it
> would make sense that more prefetching is worse -- which is why a
> bitmapheapscan plan wouldn't have been chosen. But in the optimal
> cases, it is unclear why it would be worse.
>
Yes, not sure about the optimal cases. I'll wait for the 10M runs to
complete, and then we can look for some patterns.
> I don't think there is any way it could be the issue Andres
> identified, because there is only one iterator. Nothing to get out of
> sync. It could be that the fadvises are being issued too close to the
> reads and aren't effective enough at covering up read latency on
> slower, older hardware. But that doesn't explain why master would
> sometimes be faster.
>
Ah, right, thanks for the clarification. I forgot the streaming read API
does not use the two-iterator approach.
> Probably the only thing we can do is get into a repro. It would, of
> course, be easiest to do this with a serial query. I can dig into the
> scripts you shared earlier and try to find a good repro. Because the
> regressions may have shifted with Thomas' new version, it would help
> if you shared a category (cyclic/uniform/etc, parallel or serial, eic
> value, work mem, etc) where you now see the most regressions.
>
OK, I've restarted the tests for only 0012 and 0014 patches, and I'll
wait for these to complete - I don't want to be looking for patterns
until we have enough data to smooth this out.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-17 19:21 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 14:47 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-18 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-19 20:34 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2025-01-30 23:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-19 20:34 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/18/24 16:55, Tomas Vondra wrote:
>
> ...
>
> OK, I've restarted the tests for only 0012 and 0014 patches, and I'll
> wait for these to complete - I don't want to be looking for patterns
> until we have enough data to smooth this out.
>
>
I now have results for 1M and 10M runs on the two builds (0012 and
0014), attached is a chart for relative performance plotting
(0014 timing) / (0012 timing)
for "optimal' runs that would pick bitmapscan on their own. There's
nothing special about the config - I reduced the random_page_cost to
1.5-2.0 to reflect both machines have flash storage, etc.
Overall, the chart is pretty consistent with what I shared on Sunday.
Most of the results are fine (0014 is close to 0012 or faster), but
there's a bunch of cases that are much slower. Interestingly enough,
almost all of them are on the i5 machine, almost none of the xeon. My
guess is this is about the SSD type (SATA vs. NVMe).
Attached if table of ~50 worst regressions (by the metric above), and
it's interesting the worst regressions are with eic=0 and eic=1.
I decided to look at the first case (eic=0), and the timings are quite
stable - there are three runs for each build, with timings close to the
average (see below the table).
Attached is a script that reproduces this on both machines, but the
difference is much more significant on i5 (~5x) compared to xeon (~2x).
I haven't investigated what exactly is happening and why, hopefully the
script will allow you to reproduce this independently. I plan to take a
look, but I don't know when I'll have time for this.
FWIW if the script does not reproduce this on your machines, I might be
able to give you access to the i5 machine. Let me know.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
create table test_table (a bigint, b bigint, c text) with (fillfactor = 25);
insert into test_table select a, b, c from (select a, b, c, generate_series(1,24) from (select a, b, c from (select (416 * random())::int a, i b, md5(random()::text) c from generate_series(1, 1000000/24) s(i)) foo) bar) baz;
create index on test_table (a);
vacuum analyze;
checkpoint;
-- 0012
set effective_io_concurrency = 0;
set work_mem = '4MB';
explain analyze select * from test_table where (a >= 53) AND (a <= 264);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on test_table (cost=6030.91..55240.07 rows=502877 width=49) (actual time=21.821..406.664 rows=508440 loops=1)
Recheck Cond: ((a >= 53) AND (a <= 264))
Heap Blocks: exact=21185
-> Bitmap Index Scan on test_table_a_idx (cost=0.00..5905.20 rows=502877 width=0) (actual time=18.599..18.599 rows=508440 loops=1)
Index Cond: ((a >= 53) AND (a <= 264))
Planning Time: 5.235 ms
Execution Time: 421.185 ms
(7 rows)
-- 0014
set effective_io_concurrency = 0;
set work_mem = '4MB';
explain analyze select * from test_table where (a >= 53) AND (a <= 264);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on test_table (cost=6030.91..55240.07 rows=502877 width=49) (actual time=21.894..2697.074 rows=508440 loops=1)
Recheck Cond: ((a >= 53) AND (a <= 264))
Heap Blocks: exact=21185
-> Bitmap Index Scan on test_table_a_idx (cost=0.00..5905.20 rows=502877 width=0) (actual time=18.714..18.715 rows=508440 loops=1)
Index Cond: ((a >= 53) AND (a <= 264))
Planning Time: 4.943 ms
Execution Time: 2730.409 ms
(7 rows)
with data as (
select machine, build, rows, dataset, workers, wm, eic, matches, caching, count(*), round(avg(timing),2) as timing
from results where optimal = 'bitmapscan'
group by 1, 2, 3, 4, 5, 6, 7, 8, 9
)
select d1.*, d2.timing as timing_0014, round(d2.timing / d1.timing,2) AS change
from data d1
join data d2 on ((d1.machine, d1.rows, d1.dataset, d1.workers, d1.wm, d1.eic, d1.matches, d1.caching) = (d2.machine, d2.rows, d2.dataset, d2.workers, d2.wm, d2.eic, d2.matches, d2.caching))
where d1.build = 'patched-0012'
and d2.build = 'patched-0014'
order by d2.timing / d1.timing desc;
machine | build | rows | dataset | workers | wm | eic | matches | caching | count | timing | timing_0014 | change
---------+--------------+----------+---------------+---------+-------+-----+---------+-----------+-------+----------+-------------+--------
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 4096 | 0 | 211 | uncached | 3 | 449.78 | 2650.20 | 5.89
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 65536 | 0 | 211 | uncached | 3 | 440.61 | 2575.66 | 5.85
i5 | patched-0012 | 10000000 | uniform | 0 | 128 | 0 | 4096 | uncached | 3 | 3962.57 | 21042.76 | 5.31
i5 | patched-0012 | 10000000 | uniform | 0 | 4096 | 0 | 4096 | uncached | 3 | 3896.75 | 20501.01 | 5.26
i5 | patched-0012 | 1000000 | uniform | 0 | 65536 | 0 | 256 | uncached | 3 | 451.27 | 2302.76 | 5.10
i5 | patched-0012 | 1000000 | uniform | 0 | 4096 | 0 | 256 | uncached | 3 | 472.59 | 2364.94 | 5.00
i5 | patched-0012 | 10000000 | uniform_pages | 0 | 65536 | 0 | 1857 | uncached | 3 | 4799.53 | 22771.77 | 4.74
i5 | patched-0012 | 10000000 | uniform | 0 | 65536 | 0 | 4096 | uncached | 3 | 3773.63 | 17691.96 | 4.69
i5 | patched-0012 | 10000000 | uniform | 4 | 65536 | 1 | 16384 | uncached | 3 | 3785.72 | 17514.72 | 4.63
i5 | patched-0012 | 10000000 | uniform | 0 | 65536 | 0 | 2048 | uncached | 3 | 4845.45 | 20687.09 | 4.27
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 0 | 256 | uncached | 3 | 445.31 | 1807.80 | 4.06
i5 | patched-0012 | 10000000 | uniform | 4 | 65536 | 1 | 8192 | uncached | 3 | 4101.52 | 16243.37 | 3.96
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 1 | 1024 | uncached | 3 | 428.64 | 1669.27 | 3.89
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 1 | 2048 | uncached | 3 | 390.48 | 1509.55 | 3.87
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 1 | 2048 | uncached | 3 | 429.78 | 1615.26 | 3.76
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 0 | 256 | uncached | 3 | 476.00 | 1710.61 | 3.59
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 1 | 1024 | uncached | 3 | 441.87 | 1559.04 | 3.53
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 3 | 58.85 | 183.76 | 3.12
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 128 | 32 | 4 | uncached | 3 | 59.39 | 184.68 | 3.11
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 3 | 59.20 | 183.51 | 3.10
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 3 | 105.09 | 313.22 | 2.98
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 3 | 105.81 | 314.44 | 2.97
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 4096 | 0 | 128 | uncached | 3 | 622.12 | 1818.29 | 2.92
xeon | patched-0012 | 1000000 | uniform_pages | 4 | 65536 | 1 | 16 | uncached | 3 | 46.87 | 132.90 | 2.84
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 65536 | 0 | 128 | uncached | 3 | 678.35 | 1884.47 | 2.78
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 32 | 8 | uncached | 3 | 111.51 | 308.55 | 2.77
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 32 | 8 | uncached | 3 | 111.19 | 306.88 | 2.76
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 16 | 8 | uncached | 3 | 120.98 | 323.61 | 2.67
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 32 | 4 | uncached | 3 | 50.19 | 130.33 | 2.60
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 3 | 124.51 | 323.07 | 2.59
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 128 | 16 | 4 | uncached | 3 | 75.20 | 190.16 | 2.53
i5 | patched-0012 | 1000000 | cyclic | 0 | 128 | 32 | 4 | uncached | 3 | 51.18 | 127.96 | 2.50
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 16 | 4 | uncached | 3 | 74.08 | 184.83 | 2.50
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 3 | 128.53 | 317.58 | 2.47
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 3 | 51.86 | 128.12 | 2.47
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 16 | 8 | uncached | 3 | 127.98 | 313.20 | 2.45
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 3 | 77.42 | 189.34 | 2.45
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 0 | 128 | uncached | 3 | 672.17 | 1601.18 | 2.38
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 0 | 128 | uncached | 3 | 690.68 | 1587.87 | 2.30
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 8 | 8 | uncached | 3 | 146.23 | 317.30 | 2.17
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 16 | 4 | uncached | 3 | 63.46 | 137.28 | 2.16
i5 | patched-0012 | 1000000 | uniform | 0 | 65536 | 0 | 128 | uncached | 3 | 724.55 | 1562.10 | 2.16
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 8 | 8 | uncached | 3 | 150.09 | 320.40 | 2.13
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 128 | 8 | 4 | uncached | 3 | 96.47 | 205.49 | 2.13
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 8 | 4 | uncached | 3 | 96.30 | 204.45 | 2.12
i5 | patched-0012 | 1000000 | uniform | 0 | 4096 | 0 | 128 | uncached | 3 | 739.61 | 1564.32 | 2.12
xeon | patched-0012 | 10000000 | uniform | 0 | 65536 | 0 | 4096 | uncached | 3 | 6377.14 | 13482.09 | 2.11
test=# select * from results where machine = 'i5' and build = 'patched-0012' and rows = 1000000 and dataset = 'uniform_pages' and workers = 0 and wm = 4096 and caching = 'uncached' and matches = 211 and eic = 0;
machine | build | rows | dataset | relpages | workers | wm | eic | matches | ndistinct | run | caching | optimal | timing
---------+--------------+---------+---------------+----------+---------+------+-----+---------+-----------+-----+----------+------------+---------
i5 | patched-0012 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 1 | uncached | bitmapscan | 407.605
i5 | patched-0012 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 2 | uncached | bitmapscan | 494.448
i5 | patched-0012 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 3 | uncached | bitmapscan | 447.291
(3 rows)
test=# select * from results where machine = 'i5' and build = 'patched-0014' and rows = 1000000 and dataset = 'uniform_pages' and workers = 0 and wm = 4096 and caching = 'uncached' and matches = 211 and eic = 0;
machine | build | rows | dataset | relpages | workers | wm | eic | matches | ndistinct | run | caching | optimal | timing
---------+--------------+---------+---------------+----------+---------+------+-----+---------+-----------+-----+----------+------------+----------
i5 | patched-0014 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 1 | uncached | bitmapscan | 2693.009
i5 | patched-0014 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 2 | uncached | bitmapscan | 2653.878
i5 | patched-0014 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 3 | uncached | bitmapscan | 2603.713
(3 rows)
Attachments:
[text/plain] reproducer.txt (2.0K, ../../5d5954ed-6f43-4f1a-8e19-ece75b2b7362@enterprisedb.com/2-reproducer.txt)
download | inline:
create table test_table (a bigint, b bigint, c text) with (fillfactor = 25);
insert into test_table select a, b, c from (select a, b, c, generate_series(1,24) from (select a, b, c from (select (416 * random())::int a, i b, md5(random()::text) c from generate_series(1, 1000000/24) s(i)) foo) bar) baz;
create index on test_table (a);
vacuum analyze;
checkpoint;
-- 0012
set effective_io_concurrency = 0;
set work_mem = '4MB';
explain analyze select * from test_table where (a >= 53) AND (a <= 264);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on test_table (cost=6030.91..55240.07 rows=502877 width=49) (actual time=21.821..406.664 rows=508440 loops=1)
Recheck Cond: ((a >= 53) AND (a <= 264))
Heap Blocks: exact=21185
-> Bitmap Index Scan on test_table_a_idx (cost=0.00..5905.20 rows=502877 width=0) (actual time=18.599..18.599 rows=508440 loops=1)
Index Cond: ((a >= 53) AND (a <= 264))
Planning Time: 5.235 ms
Execution Time: 421.185 ms
(7 rows)
-- 0014
set effective_io_concurrency = 0;
set work_mem = '4MB';
explain analyze select * from test_table where (a >= 53) AND (a <= 264);
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on test_table (cost=6030.91..55240.07 rows=502877 width=49) (actual time=21.894..2697.074 rows=508440 loops=1)
Recheck Cond: ((a >= 53) AND (a <= 264))
Heap Blocks: exact=21185
-> Bitmap Index Scan on test_table_a_idx (cost=0.00..5905.20 rows=502877 width=0) (actual time=18.714..18.715 rows=508440 loops=1)
Index Cond: ((a >= 53) AND (a <= 264))
Planning Time: 4.943 ms
Execution Time: 2730.409 ms
(7 rows)
[text/plain] results.txt (9.1K, ../../5d5954ed-6f43-4f1a-8e19-ece75b2b7362@enterprisedb.com/3-results.txt)
download | inline:
with data as (
select machine, build, rows, dataset, workers, wm, eic, matches, caching, count(*), round(avg(timing),2) as timing
from results where optimal = 'bitmapscan'
group by 1, 2, 3, 4, 5, 6, 7, 8, 9
)
select d1.*, d2.timing as timing_0014, round(d2.timing / d1.timing,2) AS change
from data d1
join data d2 on ((d1.machine, d1.rows, d1.dataset, d1.workers, d1.wm, d1.eic, d1.matches, d1.caching) = (d2.machine, d2.rows, d2.dataset, d2.workers, d2.wm, d2.eic, d2.matches, d2.caching))
where d1.build = 'patched-0012'
and d2.build = 'patched-0014'
order by d2.timing / d1.timing desc;
machine | build | rows | dataset | workers | wm | eic | matches | caching | count | timing | timing_0014 | change
---------+--------------+----------+---------------+---------+-------+-----+---------+-----------+-------+----------+-------------+--------
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 4096 | 0 | 211 | uncached | 3 | 449.78 | 2650.20 | 5.89
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 65536 | 0 | 211 | uncached | 3 | 440.61 | 2575.66 | 5.85
i5 | patched-0012 | 10000000 | uniform | 0 | 128 | 0 | 4096 | uncached | 3 | 3962.57 | 21042.76 | 5.31
i5 | patched-0012 | 10000000 | uniform | 0 | 4096 | 0 | 4096 | uncached | 3 | 3896.75 | 20501.01 | 5.26
i5 | patched-0012 | 1000000 | uniform | 0 | 65536 | 0 | 256 | uncached | 3 | 451.27 | 2302.76 | 5.10
i5 | patched-0012 | 1000000 | uniform | 0 | 4096 | 0 | 256 | uncached | 3 | 472.59 | 2364.94 | 5.00
i5 | patched-0012 | 10000000 | uniform_pages | 0 | 65536 | 0 | 1857 | uncached | 3 | 4799.53 | 22771.77 | 4.74
i5 | patched-0012 | 10000000 | uniform | 0 | 65536 | 0 | 4096 | uncached | 3 | 3773.63 | 17691.96 | 4.69
i5 | patched-0012 | 10000000 | uniform | 4 | 65536 | 1 | 16384 | uncached | 3 | 3785.72 | 17514.72 | 4.63
i5 | patched-0012 | 10000000 | uniform | 0 | 65536 | 0 | 2048 | uncached | 3 | 4845.45 | 20687.09 | 4.27
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 0 | 256 | uncached | 3 | 445.31 | 1807.80 | 4.06
i5 | patched-0012 | 10000000 | uniform | 4 | 65536 | 1 | 8192 | uncached | 3 | 4101.52 | 16243.37 | 3.96
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 1 | 1024 | uncached | 3 | 428.64 | 1669.27 | 3.89
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 1 | 2048 | uncached | 3 | 390.48 | 1509.55 | 3.87
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 1 | 2048 | uncached | 3 | 429.78 | 1615.26 | 3.76
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 0 | 256 | uncached | 3 | 476.00 | 1710.61 | 3.59
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 1 | 1024 | uncached | 3 | 441.87 | 1559.04 | 3.53
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 3 | 58.85 | 183.76 | 3.12
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 128 | 32 | 4 | uncached | 3 | 59.39 | 184.68 | 3.11
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 3 | 59.20 | 183.51 | 3.10
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 3 | 105.09 | 313.22 | 2.98
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 3 | 105.81 | 314.44 | 2.97
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 4096 | 0 | 128 | uncached | 3 | 622.12 | 1818.29 | 2.92
xeon | patched-0012 | 1000000 | uniform_pages | 4 | 65536 | 1 | 16 | uncached | 3 | 46.87 | 132.90 | 2.84
i5 | patched-0012 | 1000000 | uniform_pages | 0 | 65536 | 0 | 128 | uncached | 3 | 678.35 | 1884.47 | 2.78
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 32 | 8 | uncached | 3 | 111.51 | 308.55 | 2.77
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 32 | 8 | uncached | 3 | 111.19 | 306.88 | 2.76
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 16 | 8 | uncached | 3 | 120.98 | 323.61 | 2.67
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 32 | 4 | uncached | 3 | 50.19 | 130.33 | 2.60
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 3 | 124.51 | 323.07 | 2.59
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 128 | 16 | 4 | uncached | 3 | 75.20 | 190.16 | 2.53
i5 | patched-0012 | 1000000 | cyclic | 0 | 128 | 32 | 4 | uncached | 3 | 51.18 | 127.96 | 2.50
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 16 | 4 | uncached | 3 | 74.08 | 184.83 | 2.50
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 3 | 128.53 | 317.58 | 2.47
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 3 | 51.86 | 128.12 | 2.47
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 4096 | 16 | 8 | uncached | 3 | 127.98 | 313.20 | 2.45
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 3 | 77.42 | 189.34 | 2.45
i5 | patched-0012 | 1000000 | uniform | 4 | 65536 | 0 | 128 | uncached | 3 | 672.17 | 1601.18 | 2.38
i5 | patched-0012 | 1000000 | uniform | 4 | 4096 | 0 | 128 | uncached | 3 | 690.68 | 1587.87 | 2.30
i5 | patched-0012 | 1000000 | cyclic | 0 | 65536 | 8 | 8 | uncached | 3 | 146.23 | 317.30 | 2.17
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 16 | 4 | uncached | 3 | 63.46 | 137.28 | 2.16
i5 | patched-0012 | 1000000 | uniform | 0 | 65536 | 0 | 128 | uncached | 3 | 724.55 | 1562.10 | 2.16
i5 | patched-0012 | 1000000 | cyclic | 0 | 4096 | 8 | 8 | uncached | 3 | 150.09 | 320.40 | 2.13
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 128 | 8 | 4 | uncached | 3 | 96.47 | 205.49 | 2.13
i5 | patched-0012 | 1000000 | cyclic_fuzz | 0 | 65536 | 8 | 4 | uncached | 3 | 96.30 | 204.45 | 2.12
i5 | patched-0012 | 1000000 | uniform | 0 | 4096 | 0 | 128 | uncached | 3 | 739.61 | 1564.32 | 2.12
xeon | patched-0012 | 10000000 | uniform | 0 | 65536 | 0 | 4096 | uncached | 3 | 6377.14 | 13482.09 | 2.11
test=# select * from results where machine = 'i5' and build = 'patched-0012' and rows = 1000000 and dataset = 'uniform_pages' and workers = 0 and wm = 4096 and caching = 'uncached' and matches = 211 and eic = 0;
machine | build | rows | dataset | relpages | workers | wm | eic | matches | ndistinct | run | caching | optimal | timing
---------+--------------+---------+---------------+----------+---------+------+-----+---------+-----------+-----+----------+------------+---------
i5 | patched-0012 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 1 | uncached | bitmapscan | 407.605
i5 | patched-0012 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 2 | uncached | bitmapscan | 494.448
i5 | patched-0012 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 3 | uncached | bitmapscan | 447.291
(3 rows)
test=# select * from results where machine = 'i5' and build = 'patched-0014' and rows = 1000000 and dataset = 'uniform_pages' and workers = 0 and wm = 4096 and caching = 'uncached' and matches = 211 and eic = 0;
machine | build | rows | dataset | relpages | workers | wm | eic | matches | ndistinct | run | caching | optimal | timing
---------+--------------+---------+---------------+----------+---------+------+-----+---------+-----------+-----+----------+------------+----------
i5 | patched-0014 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 1 | uncached | bitmapscan | 2693.009
i5 | patched-0014 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 2 | uncached | bitmapscan | 2653.878
i5 | patched-0014 | 1000000 | uniform_pages | 41666 | 0 | 4096 | 0 | 211 | 416 | 3 | uncached | bitmapscan | 2603.713
(3 rows)
[image/png] relative-all-optimal.png (90.7K, ../../5d5954ed-6f43-4f1a-8e19-ece75b2b7362@enterprisedb.com/4-relative-all-optimal.png)
download | view image
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-17 19:21 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 14:47 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-18 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-19 20:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2025-01-30 23:10 ` Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 0 replies; 248+ messages in thread
From: Melanie Plageman @ 2025-01-30 23:10 UTC (permalink / raw)
To: Tomas Vondra <tomas@vondra.me>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On Tue, Mar 19, 2024 at 4:34 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 3/18/24 16:55, Tomas Vondra wrote:
> >
> > ...
> >
> > OK, I've restarted the tests for only 0012 and 0014 patches, and I'll
> > wait for these to complete - I don't want to be looking for patterns
> > until we have enough data to smooth this out.
> >
> >
>
> I now have results for 1M and 10M runs on the two builds (0012 and
> 0014), attached is a chart for relative performance plotting
>
> (0014 timing) / (0012 timing)
>
> for "optimal' runs that would pick bitmapscan on their own. There's
> nothing special about the config - I reduced the random_page_cost to
> 1.5-2.0 to reflect both machines have flash storage, etc.
>
> Overall, the chart is pretty consistent with what I shared on Sunday.
> Most of the results are fine (0014 is close to 0012 or faster), but
> there's a bunch of cases that are much slower. Interestingly enough,
> almost all of them are on the i5 machine, almost none of the xeon. My
> guess is this is about the SSD type (SATA vs. NVMe).
>
> Attached if table of ~50 worst regressions (by the metric above), and
> it's interesting the worst regressions are with eic=0 and eic=1.
>
> I decided to look at the first case (eic=0), and the timings are quite
> stable - there are three runs for each build, with timings close to the
> average (see below the table).
>
> Attached is a script that reproduces this on both machines, but the
> difference is much more significant on i5 (~5x) compared to xeon (~2x).
>
> I haven't investigated what exactly is happening and why, hopefully the
> script will allow you to reproduce this independently. I plan to take a
> look, but I don't know when I'll have time for this.
>
> FWIW if the script does not reproduce this on your machines, I might be
> able to give you access to the i5 machine. Let me know.
I had this particular email on this thread bookmarked so I could go
back and investigate the regression. The patch set has changed since
these benchmarks were run. And, I honestly no longer remember what
0014 and 0012 were. There are four remaining patches in the set I
posted earlier today in [1]. All of them are directly related to
bitmap heap scan using the streaming read interface (i.e. not useful
on their own). Therefore, it is time to investigate if we should merge
streaming read bitmap heap scan.
I ran the query included in the reproducer in this mail a dozen times
on master and with the patches in [1] and the average speedup with my
patch is 12%. So, at least for this query, I don't see a regression.
What do you think about rerunning these old benchmarks to see what
they look like now?
- Melanie
[1] https://www.postgresql.org/message-id/CAAKRu_as499kHb9B4B4%3D%2Bc%2B4p%2BOF_Bibd4KEdoqyBgjEaEUdgA%40...
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-21 14:55 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
1 sibling, 0 replies; 248+ messages in thread
From: Tomas Vondra @ 2024-03-21 14:55 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Robert Haas <robertmhaas@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/14/24 19:16, Melanie Plageman wrote:
> ...
>
> Attached v6 is rebased over your new commit. It also has the "fix" in
> 0010 which moves BitmapAdjustPrefetchIterator() back above
> table_scan_bitmap_next_block(). I've also updated the Streaming Read API
> commit (0013) to Thomas' v7 version from [1]. This has the update that
> we theorize should address some of the regressions in the bitmapheapscan
> streaming read user in 0014.
>
Based on the recent discussions in this thread I've been wondering how
does the readahead setting for the device affect the behavior, so I've
modified the script to test different values for this parameter too.
Considering the bug in the v7 patch (reported yesterday elsewhere in
this thread), I had to use the v6 version for now. I don't think it
makes much difference, the important parts of the patch do not change.
The complete results are far too large to include here (multiple MBs),
so I'll only include results for a small subset of parameters (one
dataset on i5), and some scatter charts with all results to show the
overall behavior. (I only have results for 1M rows so far).
Complete results (including the raw CSV etc.) are available in a git
repo, along with jupyter notebooks that I started using for experiments
and building the other charts:
https://github.com/tvondra/jupyterlab-projects/tree/master
If you look at the attached PDF table, the first half is for serial
execution (no parallelism), the second half is with 4 workers. And there
are 3 different readahead settings 0, 1536 and 12288 (and then different
eic values for each readahead value). The readahead values are chosen as
"disabled", 6x128kB and the default that was set by the kernel (or
wherever it comes from).
There are pretty clear patterns:
* serial runs with disabled readahead - The patch causes fairly serious
regressions (compared to master), if eic>0.
* serial runs with enabled readahead - there's still some regression for
lower matches values. Presumably, at higher values (which means larger
fraction of the table matches) the readahead kicks in, leaving the lower
values as if readahead was not enabled.
* parallel runs - The regression is much smaller, either because the
parallel workers issue requests almost as if there was readahead, or
maybe it implicitly disrupts the readahead. Not sure.
The other datasets are quite similar, feel free to check the git repo
for complete results.
One possible caveat is that maybe this affects only cases that would not
actually use bitmap scans? But if you check the attached scatter charts
(PNG), that only show results for cases where the planner would actually
pick bitmap scans on it's own, there are plenty such cases.
For the 0012 patch (chart on left), there's almost no such problem - the
results are very close to master. Similarly, there are regressions even
on the chart with readahead, but it's far less frequent/significant.
The question is whether readahead=0 is even worth worrying about? If
disabling readahead causes serious regressions even on master (clearly
visible in the PDF table), would anyone actually run with it disabled?
But I'm not sure that argument is very sound. Surely there are cases
where readahead may not detect a pattern, or where it's not supported
for some arbitrary reason (e.g. I didn't have much luck with this on
ZFS, perhaps other filesystems have similar limitations). But also what
about direct I/O? Surely that won't have readahead by kernel, right?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/pdf] i5-uncached-cyclic.pdf (150.1K, ../../a31bd142-1d4a-4732-8862-7e009e453b99@enterprisedb.com/2-i5-uncached-cyclic.pdf)
download
[image/png] relative-all-optimal-readahead-small.png (110.3K, ../../a31bd142-1d4a-4732-8862-7e009e453b99@enterprisedb.com/3-relative-all-optimal-readahead-small.png)
download | view image
[image/png] relative-all-optimal-no-readahead-small.png (104.1K, ../../a31bd142-1d4a-4732-8862-7e009e453b99@enterprisedb.com/4-relative-all-optimal-no-readahead-small.png)
download | view image
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-14 12:34 ` Robert Haas <robertmhaas@gmail.com>
2024-03-14 13:00 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
1 sibling, 1 reply; 248+ messages in thread
From: Robert Haas @ 2024-03-14 12:34 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On Thu, Mar 14, 2024 at 6:37 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> If es_snapshot was different from the active snapshot, things would get
> weird, even without parallel query. The scans would use es_snapshot for
> the visibility checks, but any functions you execute in quals would use
> the active snapshot.
Hmm, that's an interesting point.
The case where the query is suspended and resumed - i.e. cursors are
used - probably needs more analysis. In that case, perhaps there's
more room for the snapshots to diverge.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 12:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
@ 2024-03-14 13:00 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 13:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Heikki Linnakangas @ 2024-03-14 13:00 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On 14/03/2024 14:34, Robert Haas wrote:
> On Thu, Mar 14, 2024 at 6:37 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>> If es_snapshot was different from the active snapshot, things would get
>> weird, even without parallel query. The scans would use es_snapshot for
>> the visibility checks, but any functions you execute in quals would use
>> the active snapshot.
>
> Hmm, that's an interesting point.
>
> The case where the query is suspended and resumed - i.e. cursors are
> used - probably needs more analysis. In that case, perhaps there's
> more room for the snapshots to diverge.
The portal code is pretty explicit about it, the ExecutorRun() call in
PortalRunSelect() looks like this:
PushActiveSnapshot(queryDesc->snapshot);
ExecutorRun(queryDesc, direction, (uint64) count,
portal->run_once);
nprocessed = queryDesc->estate->es_processed;
PopActiveSnapshot();
I looked at all the callers of ExecutorRun(), and they all have the
active snapshot equal to queryDesc->snapshot, either because they called
CreateQueryDesc() with the active snapshot before ExecutorRun(), or they
set the active snapshot like above.
--
Heikki Linnakangas
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Re: BitmapHeapScan streaming read user and prelim refactoring Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 12:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Robert Haas <robertmhaas@gmail.com>
2024-03-14 13:00 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-14 13:20 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 248+ messages in thread
From: Robert Haas @ 2024-03-14 13:20 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Tomas Vondra <tomas.vondra@enterprisedb.com>
On Thu, Mar 14, 2024 at 9:00 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> The portal code is pretty explicit about it, the ExecutorRun() call in
> PortalRunSelect() looks like this:
>
> PushActiveSnapshot(queryDesc->snapshot);
> ExecutorRun(queryDesc, direction, (uint64) count,
> portal->run_once);
> nprocessed = queryDesc->estate->es_processed;
> PopActiveSnapshot();
>
> I looked at all the callers of ExecutorRun(), and they all have the
> active snapshot equal to queryDesc->snapshot, either because they called
> CreateQueryDesc() with the active snapshot before ExecutorRun(), or they
> set the active snapshot like above.
Well, maybe there's a bunch of code cleanup possible, then.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 13:34 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-13 18:14 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
1 sibling, 0 replies; 248+ messages in thread
From: Tomas Vondra @ 2024-03-13 18:14 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; Dilip Kumar <dilipbalaut@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; +Cc: Nazir Bilal Yavuz <byavuz81@gmail.com>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>
On 3/13/24 14:34, Heikki Linnakangas wrote:
> ...
>
> Lots of discussion happening on the performance results but it seems
> that there is no performance impact with the preliminary patches up to
> v5-0013-Streaming-Read-API.patch. I'm focusing purely on those
> preliminary patches now, because I think they're worthwhile cleanups
> independent of the streaming read API.
>
Not quite true - the comparison I shared on 29/2 [1] shows a serious
regression caused by the 0010 patch. We've been investigating this with
Melanie off list, but we don't have any clear findings yet (except that
it's clearly due to moving BitmapAdjustPrefetchIterator() a bit down.
But if we revert this (and move the BitmapAdjustPrefetchIterator back),
the regression should disappear, and we can merge these preparatory
patches. We'll have to deal with the regression (or something very
similar) when merging the remaining patches.
regards
[1]
https://www.postgresql.org/message-id/91090d58-7d3f-4447-9425-f24ba66e292a%40enterprisedb.com
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
@ 2024-03-18 12:10 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Heikki Linnakangas @ 2024-03-18 12:10 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Melanie Plageman <melanieplageman@gmail.com>; +Cc: pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 14/02/2024 21:42, Andres Freund wrote:
> On 2024-02-13 18:11:25 -0500, Melanie Plageman wrote:
>> patch 0004 is, I think, a bug fix. see [2].
>
> I'd not quite call it a bugfix, it's not like it leads to wrong
> behaviour. Seems more like an optimization. But whatever :)
It sure looks like bug to me, albeit a very minor one. Certainly not an
optimization, it doesn't affect performance in any way, only what
EXPLAIN reports. So committed and backported that to all supported branches.
--
Heikki Linnakangas
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-18 15:19 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-20 18:13 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 2 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-03-18 15:19 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Mon, Mar 18, 2024 at 02:10:28PM +0200, Heikki Linnakangas wrote:
> On 14/02/2024 21:42, Andres Freund wrote:
> > On 2024-02-13 18:11:25 -0500, Melanie Plageman wrote:
> > > patch 0004 is, I think, a bug fix. see [2].
> >
> > I'd not quite call it a bugfix, it's not like it leads to wrong
> > behaviour. Seems more like an optimization. But whatever :)
>
> It sure looks like bug to me, albeit a very minor one. Certainly not an
> optimization, it doesn't affect performance in any way, only what EXPLAIN
> reports. So committed and backported that to all supported branches.
I've attached v7 rebased over this commit.
- Melanie
Attachments:
[text/x-diff] v7-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240318151947.uu2il7iegi2gkmol@liskov/2-v7-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 2a03eb08e41300ca2a59e2185535fd3480494d20 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v7 01/13] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index e57a0b7ea31..e78d793f69c 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aabe..93fdcd226bf 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v7-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240318151947.uu2il7iegi2gkmol@liskov/3-v7-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 7c3f18736677ca63df36682c89122187ab6234b2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v7 02/13] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226bf..c64530674bd 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (15.0K, ../../20240318151947.uu2il7iegi2gkmol@liskov/4-v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From 97d1a9563293ac6a19b31695ff315efe78f11211 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v7 03/13] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 125 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 103 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 34bc60f625f..e7bed84f75b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -951,6 +951,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1039,6 +1041,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1058,6 +1066,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 680a50bf8b1..c9b9b4c00f1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2122,6 +2123,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, and the bitmap entries don't need rechecking, and all tuples on
+ * the page are visible to our transaction.
+ */
+ if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2234,6 +2253,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674bd..81e3166d51b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
+ extra_flags |= SO_CAN_SKIP_FETCH;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +532,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +602,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +611,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +698,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +708,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f68593..3dfb19ec7d5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8249b37bbf1..c43a8b3dea5 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed.
+ */
+ SO_CAN_SKIP_FETCH = 1 << 10,
} ScanOptions;
/*
@@ -944,9 +951,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 92593526725..76153b63d76 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1712,10 +1712,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1735,9 +1732,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v7-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (2.2K, ../../20240318151947.uu2il7iegi2gkmol@liskov/5-v7-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From 445f49b91876a3eb94f281e9622e23ffdafff47f Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v7 04/13] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 81e3166d51b..dc8c312cafb 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,15 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
-
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -533,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v7-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../20240318151947.uu2il7iegi2gkmol@liskov/6-v7-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From e3c76cfe8f88346e45e30a16b098efc16c4f314c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v7 05/13] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index dc8c312cafb..a6b98fa12a1 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v7-0006-table_scan_bitmap_next_block-returns-lossy-or-exa.patch (4.4K, ../../20240318151947.uu2il7iegi2gkmol@liskov/7-v7-0006-table_scan_bitmap_next_block-returns-lossy-or-exa.patch)
download | inline diff:
From 842be33052d781fc368d1e371448bd28c9fe4bd5 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v7 06/13] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c9b9b4c00f1..10c1c3b616b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2112,7 +2112,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2240,6 +2241,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a6b98fa12a1..b2397fe2054 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index c43a8b3dea5..f1d0d4b78e3 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -796,6 +796,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -811,7 +814,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1951,14 +1955,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1969,7 +1975,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v7-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../20240318151947.uu2il7iegi2gkmol@liskov/8-v7-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From 9488a9cd0704af64dae236c322ec0250a1acc934 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v7 07/13] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index b2397fe2054..94a5e2da17c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v7-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (4.1K, ../../20240318151947.uu2il7iegi2gkmol@liskov/9-v7-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From dd1264816e4612735ac43292d092e6d3c742d6bc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v7 08/13] Remove table_scan_bitmap_next_tuple parameter tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 10c1c3b616b..a1ec50ab7a8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2248,7 +2248,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 94a5e2da17c..bbdaa591891 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index f1d0d4b78e3..e35bd36e710 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -787,10 +787,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -821,15 +818,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -1988,7 +1980,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2000,7 +1991,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v7-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240318151947.uu2il7iegi2gkmol@liskov/10-v7-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 3bcfc23d4eac0581eb10b5e9b1c69a411304ae50 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v7 09/13] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index a1ec50ab7a8..e038e60cd8f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2112,18 +2112,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, and the bitmap entries don't need rechecking, and all tuples on
@@ -2142,16 +2175,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2243,7 +2267,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index bbdaa591891..b1dfa582c7d 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno > node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..92b829cebc7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e35bd36e710..d214abeb201 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -780,19 +781,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -811,8 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -949,9 +945,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1011,6 +1011,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1021,6 +1036,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1944,19 +1974,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1966,8 +1994,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 76153b63d76..3b136782f38 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1710,8 +1710,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1720,9 +1718,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1730,8 +1730,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1740,9 +1738,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v7-0010-Hard-code-TBMIterateResult-offsets-array-size.patch (5.3K, ../../20240318151947.uu2il7iegi2gkmol@liskov/11-v7-0010-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From adc021b783f1f591f6e8e46405aa4c3ac617740a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v7 10/13] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 29 +++++++----------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fcc..d2bf8f44d50 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1467,8 +1453,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
* Create the TBMSharedIterator struct, with enough trailing space to
* serve the needs of the TBMIterateResult sub-struct.
*/
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v7-0011-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.7K, ../../20240318151947.uu2il7iegi2gkmol@liskov/12-v7-0011-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From bea77dce5cbb967edf5961352013283818bf1369 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v7 11/13] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 38 ++++++------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 108 insertions(+), 91 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3aa457a29e1 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e038e60cd8f..022753e203a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2119,7 +2119,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2132,11 +2132,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2151,11 +2151,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2163,19 +2163,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* the page are visible to our transaction.
*/
if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2204,7 +2204,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2213,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2265,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ *lossy = tbmres.ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index b1dfa582c7d..43df0ac9964 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -321,7 +321,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -335,8 +335,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -371,8 +371,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -440,10 +440,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -451,7 +453,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -460,13 +462,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -481,7 +483,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -500,8 +502,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -509,17 +511,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index d2bf8f44d50..7d038c2018d 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae1..3b432263bb0 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v7-0012-v7-Streaming-Read-API.patch (56.1K, ../../20240318151947.uu2il7iegi2gkmol@liskov/13-v7-0012-v7-Streaming-Read-API.patch)
download | inline diff:
From 4126c2da2e963409055de628328419e577f6fc0c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:59:42 -0400
Subject: [PATCH v7 12/13] v7 Streaming Read API
---
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 659 +++++++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 642 +++++++++++++++-------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/include/storage/bufmgr.h | 45 ++
src/include/storage/streaming_read.h | 52 ++
src/tools/pgindent/typedefs.list | 3 +
10 files changed, 1227 insertions(+), 210 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..bcab44c802f
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..39aef2a84a2
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 00000000000..d5c29b750d9
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,659 @@
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ */
+typedef struct PgStreamingReadRange
+{
+ bool need_wait;
+ bool advice_issued;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index;
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+ ReadBuffersOperation operation;
+} PgStreamingReadRange;
+
+/*
+ * Streaming read object.
+ */
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int next_tail_buffer;
+ int distance;
+ bool started;
+ bool finished;
+ bool advice_enabled;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ /* Sometimes we need to buffer one block for flow control. */
+ BlockNumber unget_blocknum;
+ void *unget_per_buffer_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /*
+ * Make sure our bmr's smgr and persistent are populated. The caller
+ * asserts that the storage manager will remain valid.
+ */
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+
+ /*
+ * Choose a maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least
+ * MAX_BUFFER_PER_TRANSFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+ pgsr->callback = next_block_cb;
+ pgsr->bmr = bmr;
+ pgsr->forknum = forknum;
+
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out doing full-sized
+ * reads.
+ */
+ if (flags & PGSR_FLAG_FULL)
+ pgsr->distance = Min(MAX_BUFFERS_PER_TRANSFER, pgsr->max_pinned_buffers);
+ else
+ pgsr->distance = 1;
+
+ /*
+ * Space for the callback to store extra data along with each block. Note
+ * that we need one more than max_pinned_buffers, so we can return a
+ * pointer to a slot that can't be overwritten until the next call.
+ */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * size);
+
+ return pgsr;
+}
+
+/*
+ * Find the per-buffer data index for the Nth block of a range.
+ */
+static int
+get_per_buffer_data_index(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ int result;
+
+ /*
+ * Find slot in the circular buffer of per-buffer data, without using the
+ * expensive % operator.
+ */
+ result = range->per_buffer_data_index + n;
+ while (result >= pgsr->size)
+ result -= pgsr->size;
+ Assert(result == (range->per_buffer_data_index + n) % pgsr->size);
+
+ return result;
+}
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static void *
+get_per_buffer_data_by_index(PgStreamingRead *pgsr, int per_buffer_data_index)
+{
+ return (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * per_buffer_data_index;
+}
+
+/*
+ * Return a pointer to the per-buffer data for the Nth block of a range.
+ */
+static void *
+get_per_buffer_data(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ return get_per_buffer_data_by_index(pgsr,
+ get_per_buffer_data_index(pgsr,
+ range,
+ n));
+}
+
+/*
+ * Start reading the head range, and create a new head range. The new head
+ * range is returned. It may not be empty, if StartReadBuffers() couldn't
+ * start the entire range; in that case the returned range contains the
+ * remaining portion of the range.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_start_head_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+ PgStreamingReadRange *new_head_range;
+ int nblocks_pinned;
+ int flags;
+
+ /* Caller should make sure we never exceed max_ios. */
+ Assert((pgsr->ios_in_progress < pgsr->max_ios) ||
+ (pgsr->ios_in_progress == 0 && pgsr->max_ios == 0));
+
+ /* Should only call if the head range has some blocks to read. */
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If advice hasn't been suppressed, and this system supports it, this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (pgsr->advice_enabled &&
+ pgsr->max_ios > 0 &&
+ pgsr->started &&
+ head_range->blocknum != pgsr->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* Suppress advice on the first call, because it's too late to benefit. */
+ if (!pgsr->started)
+ pgsr->started = true;
+
+ /* We shouldn't be trying to pin more buffers that we're allowed to. */
+ Assert(pgsr->pinned_buffers + head_range->nblocks <= pgsr->max_pinned_buffers);
+
+ /* Start reading as many blocks as we can from the head range. */
+ nblocks_pinned = head_range->nblocks;
+ head_range->need_wait =
+ StartReadBuffers(pgsr->bmr,
+ head_range->buffers,
+ pgsr->forknum,
+ head_range->blocknum,
+ &nblocks_pinned,
+ pgsr->strategy,
+ flags,
+ &head_range->operation);
+
+ Assert(pgsr->pinned_buffers <= pgsr->max_pinned_buffers);
+
+ if (head_range->need_wait && (flags & READ_BUFFERS_ISSUE_ADVICE))
+ {
+ /*
+ * Since we've issued advice, we count an I/O in progress until we
+ * call WaitReadBuffers().
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ Assert(pgsr->ios_in_progress <= pgsr->max_ios);
+ }
+
+ /*
+ * StartReadBuffers() might have pinned fewer blocks than we asked it to,
+ * but always at least one.
+ */
+ Assert(nblocks_pinned <= head_range->nblocks);
+ Assert(nblocks_pinned >= 1);
+ pgsr->pinned_buffers += nblocks_pinned;
+
+ /*
+ * Remember where the next block would be after that, so we can detect
+ * sequential access next time.
+ */
+ pgsr->seq_blocknum = head_range->blocknum + nblocks_pinned;
+
+ /*
+ * Create a new head range. There must be space, because we have enough
+ * elements for every range to hold just one block, up to the pin limit.
+ */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ new_head_range = &pgsr->ranges[pgsr->head];
+ new_head_range->nblocks = 0;
+ new_head_range->advice_issued = false;
+
+ /*
+ * If we didn't manage to start the whole read above, we split the range,
+ * moving the remainder into the new head range.
+ */
+ if (nblocks_pinned < head_range->nblocks)
+ {
+ int nblocks_remaining = head_range->nblocks - nblocks_pinned;
+
+ head_range->nblocks = nblocks_pinned;
+
+ new_head_range->blocknum = head_range->blocknum + nblocks_pinned;
+ new_head_range->nblocks = nblocks_remaining;
+ }
+
+ /* The new range has per-buffer data starting after the previous range. */
+ new_head_range->per_buffer_data_index =
+ get_per_buffer_data_index(pgsr, head_range, nblocks_pinned);
+
+ return new_head_range;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow pg_streaming_unget_block() to work.
+ */
+static BlockNumber
+pg_streaming_get_block(PgStreamingRead *pgsr, void *per_buffer_data)
+{
+ BlockNumber result;
+
+ if (unlikely(pgsr->unget_blocknum != InvalidBlockNumber))
+ {
+ /*
+ * If we had to unget a block, now it is time to return that one
+ * again.
+ */
+ result = pgsr->unget_blocknum;
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+ /*
+ * The same per_buffer_data element must have been used, and still
+ * contains whatever data the callback wrote into it. So we just
+ * sanity-check that we were called with the value that
+ * pg_streaming_unget_block() pushed back.
+ */
+ Assert(per_buffer_data == pgsr->unget_per_buffer_data);
+ }
+ else
+ {
+ /* Use the installed callback directly. */
+ result = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ }
+
+ return result;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later. This *must* be called with the
+ * last value returned by pg_streaming_get_block().
+ */
+static void
+pg_streaming_unget_block(PgStreamingRead *pgsr, BlockNumber blocknum, void *per_buffer_data)
+{
+ Assert(pgsr->unget_blocknum == InvalidBlockNumber);
+ pgsr->unget_blocknum = blocknum;
+ pgsr->unget_per_buffer_data = per_buffer_data;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *range;
+
+ /* If we're finished, don't look ahead. */
+ if (pgsr->finished)
+ return;
+
+ /*
+ * We we've already started the maximum allowed number of I/Os, don't look
+ * ahead. There is a special case for max_ios == 0.
+ */
+ if (pgsr->max_ios > 0 && pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /* Can't pin any more buffers. */
+ if (pgsr->pinned_buffers == pgsr->distance)
+ return;
+
+ /*
+ * Keep trying to add new blocks to the end of the head range while doing
+ * so wouldn't exceed the distance limit.
+ */
+ range = &pgsr->ranges[pgsr->head];
+ while (pgsr->pinned_buffers + range->nblocks < pgsr->distance)
+ {
+ BlockNumber blocknum;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ if (range->nblocks == lengthof(range->buffers))
+ {
+ /* Start as much of it as we can. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /* If we're now at the I/O limit, stop here. */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * That might have only been partially started, but always
+ * processes at least one so that'll do for now.
+ */
+ Assert(range->nblocks < lengthof(range->buffers));
+ }
+
+ /* Find per-buffer data slot for the next block. */
+ per_buffer_data = get_per_buffer_data(pgsr, range, range->nblocks);
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pg_streaming_get_block(pgsr, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ pgsr->finished = true;
+ break;
+ }
+
+ /*
+ * Is there a head range that we cannot extend, because the requested
+ * block is not consecutive?
+ */
+ if (range->nblocks > 0 &&
+ range->blocknum + range->nblocks != blocknum)
+ {
+ /* Yes. Start it, so we can begin building a new one. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * It's possible that it was only partially started, and we have a
+ * new range with the remainder. Keep starting I/Os until we get
+ * it all out of the way, or we hit the I/O limit.
+ */
+ while (range->nblocks > 0 && pgsr->ios_in_progress < pgsr->max_ios)
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * We do have to worry about I/O capacity running out if the head
+ * range was split. In that case we have to 'unget' the block
+ * returned by the callback.
+ */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ {
+ pg_streaming_unget_block(pgsr, blocknum, per_buffer_data);
+ return;
+ }
+ }
+
+ /* If we have a new, empty range, initialize the start block. */
+ if (range->nblocks == 0)
+ range->blocknum = blocknum;
+
+ /* This block extends the range by one. */
+ Assert(range->blocknum + range->nblocks == blocknum);
+ range->nblocks++;
+ };
+
+ /*
+ * Normally we don't start the head range, preferring to give it a chance
+ * to grow to full size once more buffers have been consumed. In cases
+ * where that can't possibly happen, we might as well start the read
+ * immediately.
+ */
+ if ((range->nblocks > 0 && pgsr->finished) ||
+ (range->nblocks == pgsr->distance))
+ pg_streaming_read_start_head_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ PgStreamingReadRange *tail_range;
+
+ for (;;)
+ {
+ if (pgsr->tail != pgsr->head)
+ {
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to wait for a ReadBuffers operation to finish before
+ * returning the buffers in this range?
+ */
+ if (tail_range->need_wait)
+ {
+ int distance;
+
+ Assert(pgsr->next_tail_buffer == 0);
+ WaitReadBuffers(&tail_range->operation);
+ tail_range->need_wait = false;
+
+ /*
+ * We don't really know if the kernel generated a physical I/O
+ * when we issued advice, let alone when it finished, but it
+ * has certainly finished now because we've performed the
+ * read.
+ */
+ if (tail_range->advice_issued)
+ {
+
+ Assert(pgsr->ios_in_progress > 0);
+ pgsr->ios_in_progress--;
+
+ /*
+ * Look-ahead distance ramps up rapidly if we're issuing
+ * advice, so we can search for new more I/Os to start.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ else
+ {
+ /*
+ * There is no point in increasing look-ahead distance if
+ * we've already reached the full I/O size, since we're
+ * not issuing advice. Extra distance would only pin more
+ * buffers for no benefit.
+ */
+ if (pgsr->distance > MAX_BUFFERS_PER_TRANSFER)
+ {
+ /*
+ * Look-ahead distance gradually decays to full I/O
+ * size.
+ */
+ pgsr->distance--;
+ }
+ else
+ {
+ /*
+ * Look-ahead distance ramps up rapidly, but not more
+ * that the full I/O size.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, MAX_BUFFERS_PER_TRANSFER);
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ }
+ }
+ else if (pgsr->next_tail_buffer == 0)
+ {
+ /* No I/O necessary. Look-ahead distance gradually decays. */
+ if (pgsr->distance > 1)
+ pgsr->distance--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(pgsr, tail_range, buffer_index);
+
+ /* We may be able to get another I/O started. */
+ pg_streaming_read_look_ahead(pgsr);
+
+ return buffer;
+ }
+
+ /* Advance tail to next range. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+ else
+ {
+ /*
+ * If tail crashed into head, and head is not empty, then it is
+ * time to start that range. Otherwise, force a look-ahead, to
+ * kick start the stream.
+ */
+ Assert(pgsr->tail == pgsr->head);
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ {
+ pg_streaming_read_start_head_range(pgsr);
+ }
+ else
+ {
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* Finished? */
+ if (pgsr->tail == pgsr->head &&
+ pgsr->ranges[pgsr->head].nblocks == 0)
+ break;
+ }
+ }
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ pgsr->finished = true;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(pgsr->pinned_buffers == 0);
+ Assert(pgsr->ios_in_progress == 0);
+
+ /* Release memory. */
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c5..d0e9c7deffa 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -471,10 +476,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +504,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +785,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +797,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +819,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +990,68 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
+/*
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
+ */
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* ReadBuffer_common -- common logic for all ReadBuffer variants
*
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+ BufferAccessStrategy strategy)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int nblocks;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1070,405 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ nblocks = 1;
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
+ else
+ flags = 0;
+ if (StartReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ &nblocks,
+ strategy,
+ flags,
+ &operation))
+ WaitReadBuffers(&operation);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ ZeroBuffer(buffer, mode);
+
+ return buffer;
+}
+static Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested.
+ *
+ * If false is returned, no I/O is necessary and WaitReadBuffers() is not
+ * necessary. If true is returned, one I/O has been started, and
+ * WaitReadBuffers() must be called with the same operation object before the
+ * buffers are accessed. Along with the operation object, the caller-supplied
+ * array of buffers must remain valid until WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+bool
+StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation)
+{
+ int actual_nblocks = *nblocks;
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ operation->bmr = bmr;
+ operation->forknum = forkNum;
+ operation->blocknum = blockNum;
+ operation->buffers = buffers;
+ operation->nblocks = actual_nblocks;
+ operation->strategy = strategy;
+ operation->flags = flags;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ operation->io_buffers_len = 0;
- /*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
- */
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
- else
+ for (int i = 0; i < actual_nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ bool found;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ buffers[i] = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum + i,
+ strategy,
+ &found);
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = operation->nblocks = *nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ operation->io_buffers_len++;
+ }
+ }
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (operation->io_buffers_len > 0)
+ {
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
- {
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
+ /*
+ * In theory we should only do this if PrepareReadBuffers() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(bmr.smgr, forkNum, blockNum, operation->io_buffers_len);
}
+
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
}
+ else
+ {
+ return false;
+ }
+}
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
{
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ /*
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
+ */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+ bmr = operation->bmr;
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
-
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
}
else
{
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
}
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ /*
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
+ */
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
+ else
+ pgBufferUsage.shared_blks_read += nblocks;
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ for (int i = 0; i < nblocks; ++i)
+ {
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- return BufferDescriptorGetBuffer(bufHdr);
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
+
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
+ {
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
+ {
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
+ }
+ else
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
+
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
+
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
+
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
+
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for StartReadBuffers. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1476,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1535,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1603,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1638,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +1994,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2259,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2282,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2372,7 +2597,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2611,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3679,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5184,9 +5414,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5435,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a57..985a2c7049c 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..b57f71f97e3 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,42 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+/*
+ * Private state used by StartReadBuffers() and WaitReadBuffers(). Declared
+ * in public header only to allow inclusion in other structs, but contents
+ * should not be accessed.
+ */
+struct ReadBuffersOperation
+{
+ /* Parameters passed in to StartReadBuffers(). */
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ int nblocks;
+ BufferAccessStrategy strategy;
+ int flags;
+
+ /* Range of buffers, if we need to perform a read. */
+ int io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +292,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 00000000000..c4d3892bb26
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,52 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define PGSR_FLAG_FULL 0x04
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 042d04c8de2..3ea4d350242 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2102,6 +2102,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
@@ -2273,6 +2275,7 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
--
2.40.1
[text/x-diff] v7-0013-BitmapHeapScan-uses-streaming-read-API.patch (28.6K, ../../20240318151947.uu2il7iegi2gkmol@liskov/14-v7-0013-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From bea56f91232e835f02dd0a6d8de8f373bd9ac397 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:04:18 -0500
Subject: [PATCH v7 13/13] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
---
src/backend/access/heap/heapam.c | 68 +++++
src/backend/access/heap/heapam_handler.c | 88 +++---
src/backend/executor/nodeBitmapHeapscan.c | 353 +---------------------
src/include/access/heapam.h | 4 +
src/include/access/tableam.h | 22 +-
src/include/nodes/execnodes.h | 22 --
6 files changed, 116 insertions(+), 441 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e7bed84f75b..0f370bfec3e 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -111,6 +111,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -331,6 +333,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -951,6 +969,7 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
@@ -1089,6 +1108,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10246,3 +10268,49 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (hdesc->rs_base.shared_tbmiterator)
+ tbm_shared_iterate(hdesc->rs_base.shared_tbmiterator, tbmres);
+ else
+ tbm_iterate(hdesc->rs_base.tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ if (hdesc->rs_base.rs_flags & SO_CAN_SKIP_FETCH &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 022753e203a..9727613e87f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2111,79 +2111,65 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, and the bitmap entries don't need rechecking, and all tuples on
- * the page are visible to our transaction.
- */
- if (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2204,7 +2190,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2199,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2251,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres.ntuples < 0;
+ *lossy = tbmres->ntuples < 0;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 43df0ac9964..c437718f980 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -86,14 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -109,15 +97,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
}
else
{
@@ -140,20 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
- }
-#endif
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
@@ -161,14 +126,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -212,37 +169,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -264,28 +190,13 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
new_page:
-
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy))
break;
if (lossy)
node->lossy_pages++;
else
node->exact_pages++;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->prefetch_iterator &&
- node->pfblockno > node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -309,225 +220,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- tbm_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (scan->rs_flags & SO_CAN_SKIP_FETCH &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -573,23 +265,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -628,16 +309,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -670,18 +345,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -721,13 +389,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -804,14 +465,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -842,11 +499,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d5..1cad9c04f01 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index d214abeb201..5a963f9293f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -792,23 +792,11 @@ typedef struct TableAmRoutine
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool (*scan_bitmap_next_block) (TableScanDesc scan, bool *recheck,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1983,8 +1971,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* used after verifying the presence (at plan time or such).
*/
static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+table_scan_bitmap_next_block(TableScanDesc scan, bool *recheck, bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1994,8 +1981,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck, lossy);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 3b136782f38..bcffe309913 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1685,11 +1685,8 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
* mutex mutual exclusion for the prefetching variable
* and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1697,10 +1694,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
@@ -1710,19 +1704,11 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1730,19 +1716,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-19 12:33 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
1 sibling, 1 reply; 248+ messages in thread
From: Heikki Linnakangas @ 2024-03-19 12:33 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 18/03/2024 17:19, Melanie Plageman wrote:
> I've attached v7 rebased over this commit.
Thanks!
> v7-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch
If we delayed table_beginscan_bm() call further, after starting the TBM
iterator, we could skip it altogether when the iterator is empty.
That's a further improvement, doesn't need to be part of this patch set.
Just caught my eye while reading this.
> v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch
I suggest to avoid the double negative with SO_CAN_SKIP_FETCH, and call
the flag e.g. SO_NEED_TUPLE.
As yet another preliminary patch before the streaming read API, it would
be nice to move the prefetching code to heapam.c too.
What's the point of having separate table_scan_bitmap_next_block() and
table_scan_bitmap_next_tuple() functions anymore? The AM owns the TBM
iterator now. The executor node updates the lossy/exact page counts, but
that's the only per-page thing it does now.
> /*
> * If this is the first scan of the underlying table, create the table
> * scan descriptor and begin the scan.
> */
> if (!scan)
> {
> uint32 extra_flags = 0;
>
> /*
> * We can potentially skip fetching heap pages if we do not need
> * any columns of the table, either for checking non-indexable
> * quals or for returning data. This test is a bit simplistic, as
> * it checks the stronger condition that there's no qual or return
> * tlist at all. But in most cases it's probably not worth working
> * harder than that.
> */
> if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
> extra_flags |= SO_CAN_SKIP_FETCH;
>
> scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
> node->ss.ss_currentRelation,
> node->ss.ps.state->es_snapshot,
> 0,
> NULL,
> extra_flags);
> }
>
> scan->tbmiterator = tbmiterator;
> scan->shared_tbmiterator = shared_tbmiterator;
How about passing the iterator as an argument to table_beginscan_bm()?
You'd then need some other function to change the iterator on rescan,
though. Not sure what exactly to do here, but feels that this part of
the API is not fully thought-out. Needs comments at least, to explain
who sets tbmiterator / shared_tbmiterator and when. For comparison, for
a TID scan there's a separate scan_set_tidrange() table AM function.
Maybe follow that example and introduce scan_set_tbm_iterator().
It's bit awkward to have separate tbmiterator and shared_tbmiterator
fields. Could regular and shared iterators be merged, or wrapped under a
common interface?
--
Heikki Linnakangas
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
@ 2024-03-23 00:22 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-23 00:22 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Tue, Mar 19, 2024 at 02:33:35PM +0200, Heikki Linnakangas wrote:
> On 18/03/2024 17:19, Melanie Plageman wrote:
> > I've attached v7 rebased over this commit.
>
> If we delayed table_beginscan_bm() call further, after starting the TBM
> iterator, we could skip it altogether when the iterator is empty.
>
> That's a further improvement, doesn't need to be part of this patch set.
> Just caught my eye while reading this.
Hmm. You mean like until after the first call to tbm_[shared]_iterate()?
AFAICT, tbm_begin_iterate() doesn't tell us anything about whether or
not the iterator is "empty". Do you mean cases when the bitmap has no
blocks in it? It seems like we should be able to tell that from the
TIDBitmap.
>
> > v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch
>
> I suggest to avoid the double negative with SO_CAN_SKIP_FETCH, and call the
> flag e.g. SO_NEED_TUPLE.
Agreed. Done in attached v8. Though I wondered if it was a bit weird
that the flag is set in the common case and not set in the uncommon
case...
> As yet another preliminary patch before the streaming read API, it would be
> nice to move the prefetching code to heapam.c too.
I've done this, but I can say it is not very pretty. see 0013. I had to
add a bunch of stuff to TableScanDescData and HeapScanDescData which are
only used for bitmapheapscans. I don't know if it makes the BHS
streaming read user patch easier to review, but I don't think what I
have in 0013 is committable to Postgres. Maybe there was another way I
could have approached it. Let me know what you think.
In addition to bloating the table descriptors, note that it was
difficult to avoid one semantic change -- with 0013, we no longer
prefetch or adjust prefetch target when emitting each empty tuple --
though I think this is could actually be desirable.
> What's the point of having separate table_scan_bitmap_next_block() and
> table_scan_bitmap_next_tuple() functions anymore? The AM owns the TBM
> iterator now. The executor node updates the lossy/exact page counts, but
> that's the only per-page thing it does now.
Oh, interesting. Good point. I've done this in 0015. If you like the way
it turned out, I can probably rebase this back into an earlier point in
the set and end up dropping some of the other incremental changes (e.g.
0008).
> > /*
> > * If this is the first scan of the underlying table, create the table
> > * scan descriptor and begin the scan.
> > */
> > if (!scan)
> > {
> > uint32 extra_flags = 0;
> >
> > /*
> > * We can potentially skip fetching heap pages if we do not need
> > * any columns of the table, either for checking non-indexable
> > * quals or for returning data. This test is a bit simplistic, as
> > * it checks the stronger condition that there's no qual or return
> > * tlist at all. But in most cases it's probably not worth working
> > * harder than that.
> > */
> > if (node->ss.ps.plan->qual == NIL && node->ss.ps.plan->targetlist == NIL)
> > extra_flags |= SO_CAN_SKIP_FETCH;
> >
> > scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
> > node->ss.ss_currentRelation,
> > node->ss.ps.state->es_snapshot,
> > 0,
> > NULL,
> > extra_flags);
> > }
> >
> > scan->tbmiterator = tbmiterator;
> > scan->shared_tbmiterator = shared_tbmiterator;
>
> How about passing the iterator as an argument to table_beginscan_bm()? You'd
> then need some other function to change the iterator on rescan, though. Not
> sure what exactly to do here, but feels that this part of the API is not
> fully thought-out. Needs comments at least, to explain who sets tbmiterator
> / shared_tbmiterator and when. For comparison, for a TID scan there's a
> separate scan_set_tidrange() table AM function. Maybe follow that example
> and introduce scan_set_tbm_iterator().
I've spent quite a bit of time playing around with the code trying to
make it less terrible than what I had before.
On rescan, we have to actually make the whole bitmap and iterator. And,
we don't have what we need to do that in table_rescan()/heap_rescan().
From what I can tell, scan_set_tidrange() is useful because it can be
called from both the beginscan and rescan functions without invoking it
directly from TidNext().
In our case, any wrapper function we wrote would basically just assign
the iterator to the scan in BitmapHeapNext().
I've reorganized this code structure a bit, so see if you like it more
now. I rebased the changes into some of the other patches, so you'll
just have to look at the result and see what you think.
> It's bit awkward to have separate tbmiterator and shared_tbmiterator fields.
> Could regular and shared iterators be merged, or wrapped under a common
> interface?
This is a good idea. I've done that in 0014. It made the code nicer, but
I just wonder if it will add too much overhead.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-23 00:26 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-23 00:26 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Fri, Mar 22, 2024 at 08:22:11PM -0400, Melanie Plageman wrote:
> On Tue, Mar 19, 2024 at 02:33:35PM +0200, Heikki Linnakangas wrote:
> > On 18/03/2024 17:19, Melanie Plageman wrote:
> > > I've attached v7 rebased over this commit.
> >
> > If we delayed table_beginscan_bm() call further, after starting the TBM
> > iterator, we could skip it altogether when the iterator is empty.
> >
> > That's a further improvement, doesn't need to be part of this patch set.
> > Just caught my eye while reading this.
>
> Hmm. You mean like until after the first call to tbm_[shared]_iterate()?
> AFAICT, tbm_begin_iterate() doesn't tell us anything about whether or
> not the iterator is "empty". Do you mean cases when the bitmap has no
> blocks in it? It seems like we should be able to tell that from the
> TIDBitmap.
>
> >
> > > v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch
> >
> > I suggest to avoid the double negative with SO_CAN_SKIP_FETCH, and call the
> > flag e.g. SO_NEED_TUPLE.
>
> Agreed. Done in attached v8. Though I wondered if it was a bit weird
> that the flag is set in the common case and not set in the uncommon
> case...
v8 actually attached this time
Attachments:
[text/x-diff] v8-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240323002607.abegehmmjyxrnmm2@liskov/2-v8-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 5a98d9d714e5fb081c34a3a648fffe7f0893d22d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v8 01/17] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index e57a0b7ea3..e78d793f69 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..93fdcd226b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v8-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240323002607.abegehmmjyxrnmm2@liskov/3-v8-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From b4f1ebf5885202601378007b9df1ca31c73604bf Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v8 02/17] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226b..c64530674b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v8-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (15.0K, ../../20240323002607.abegehmmjyxrnmm2@liskov/4-v8-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From a353da0564b478d63408c668c246f9ff77ebc3cc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v8 03/17] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 34bc60f625..e7bed84f75 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -951,6 +951,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1039,6 +1041,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1058,6 +1066,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2b7c702642..7fdccaf613 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2236,6 +2255,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674b..83d9db8f39 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f6859..3dfb19ec7d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 65834caeb1..1bc5f7c057 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -952,9 +959,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1774c56ae3..6871db9b21 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,10 +1785,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1808,9 +1805,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v8-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (2.2K, ../../20240323002607.abegehmmjyxrnmm2@liskov/5-v8-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From 1c6ac92b28885098809b7ec3a4555dac04d23415 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v8 04/17] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39..5df3b5ca46 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v8-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../20240323002607.abegehmmjyxrnmm2@liskov/6-v8-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From 7abe75616ad270c2e822cea81221a9069a59384e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v8 05/17] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46..404de0595e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v8-0006-table_scan_bitmap_next_block-returns-lossy-or-exa.patch (4.4K, ../../20240323002607.abegehmmjyxrnmm2@liskov/7-v8-0006-table_scan_bitmap_next_block-returns-lossy-or-exa.patch)
download | inline diff:
From 42bd8dc7d3d882464f4c655041e35ff31c171117 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v8 06/17] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 7fdccaf613..849cac3947 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2242,6 +2243,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595e..c95e3412da 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1bc5f7c057..b9ba4f9fb3 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -804,6 +804,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -819,7 +822,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1988,14 +1992,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2006,7 +2012,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v8-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../20240323002607.abegehmmjyxrnmm2@liskov/8-v8-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From 85175100810f3389d0a98e2eaf4cc28e695d8003 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v8 07/17] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da..49938c9ed4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v8-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (4.1K, ../../20240323002607.abegehmmjyxrnmm2@liskov/9-v8-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From d97d2366e680c1e830c7437f880a59ab04c84d3b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v8 08/17] Remove table_scan_bitmap_next_tuple parameter tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 849cac3947..cf4387f443 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2250,7 +2250,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed4..282dcb9791 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b9ba4f9fb3..bcf1497f67 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -795,10 +795,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -829,15 +826,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2025,7 +2017,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2037,7 +2028,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v8-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240323002607.abegehmmjyxrnmm2@liskov/10-v8-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 805ddd029fb08f60b84fd4af67ba80f35e2c2cdc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v8 09/17] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index cf4387f443..2ad785e511 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,18 +2114,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2144,16 +2177,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2245,7 +2269,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb9791..7e73583fe5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno > node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index bcf1497f67..a820cc8c99 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -788,19 +789,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -819,8 +815,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -957,9 +953,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1019,6 +1019,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1029,6 +1044,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1981,19 +2011,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2003,8 +2031,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6871db9b21..8688bc5ab0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,8 +1783,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1793,9 +1791,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,8 +1803,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1813,9 +1811,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v8-0010-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240323002607.abegehmmjyxrnmm2@liskov/11-v8-0010-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From 8a43b6676dc25c51013a06e3bebe306443a929c1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v8 10/17] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fc..1dc4c99bf9 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639b..432fae5296 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v8-0011-table_scan_bitmap_next_block-counts-lossy-and-exa.patch (5.2K, ../../20240323002607.abegehmmjyxrnmm2@liskov/12-v8-0011-table_scan_bitmap_next_block-counts-lossy-and-exa.patch)
download | inline diff:
From 9d82872fbadf30f1a3b595aa1cd9bd6fac096bee Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v8 11/17] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2ad785e511..266b34fe6b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2267,7 +2268,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7e73583fe5..96b55507a3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -69,7 +69,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -267,14 +266,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a820cc8c99..1d4b79a73f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -797,8 +797,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -815,8 +815,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2013,15 +2015,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2032,7 +2036,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v8-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.7K, ../../20240323002607.abegehmmjyxrnmm2@liskov/13-v8-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From c020e4afbdaf94a1783ddc8806be165038238bcb Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v8 12/17] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 38 ++++++------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 108 insertions(+), 91 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb..3aa457a29e 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544..033d525339 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 266b34fe6b..adfc77684a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2122,7 +2122,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2135,11 +2135,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2154,11 +2154,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2166,19 +2166,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2207,7 +2207,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2216,9 +2216,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2268,7 +2268,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 96b55507a3..51c4360205 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -316,7 +316,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -330,8 +330,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -366,8 +366,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -435,10 +435,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -446,7 +448,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -455,13 +457,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -476,7 +478,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -495,8 +497,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -504,17 +506,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf9..309a44bdb8 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae..3b432263bb 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae5296..f000c1af28 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v8-0013-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (33.7K, ../../20240323002607.abegehmmjyxrnmm2@liskov/14-v8-0013-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From a654ebbcd43dcc7cdd633c7d60350a58cafd34c2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v8 13/17] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 268 +++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 397 +++-------------------
src/include/access/heapam.h | 12 +
src/include/access/relscan.h | 11 +
src/include/access/tableam.h | 38 ++-
src/include/nodes/execnodes.h | 16 -
7 files changed, 388 insertions(+), 380 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e7bed84f75..c12563a188 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -951,8 +951,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1035,6 +1043,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1047,6 +1061,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1072,6 +1092,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index adfc77684a..efd2784e03 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,6 +55,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2112,6 +2115,76 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
+
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ tbm_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
+
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2130,6 +2203,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2273,6 +2348,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->pf_tbmiterator &&
+ hscan->pfblockno > hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2283,6 +2370,157 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+
+ if (pstate == NULL)
+ {
+ TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
+
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ tbm_end_iterate(prefetch_iterator);
+ scan->rs_base.pf_tbmiterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
+
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ tbm_end_shared_iterate(prefetch_iterator);
+ scan->rs_base.pf_shared_tbmiterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2308,6 +2546,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 51c4360205..f241f4cb2c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -71,7 +67,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -91,83 +86,53 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
+
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
- if (!pstate)
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (init_shared_state)
{
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
- {
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
- }
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
+ BitmapDoneInitializingSharedState(node->pstate);
}
-#endif /* USE_PREFETCH */
}
/*
@@ -197,8 +162,26 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
+ if (!scan->bm_parallel)
+ scan->tbmiterator = tbm_begin_iterate(tbm);
+ else
+ /* Allocate a private iterator and attach the shared state to it */
+ scan->shared_tbmiterator = tbm_attach_shared_iterate(dsa, scan->bm_parallel->tbmiterator);
+
+#ifdef USE_PREFETCH
+ if (scan->prefetch_maximum > 0)
+ {
+ if (!scan->bm_parallel)
+ scan->pf_tbmiterator = tbm_begin_iterate(tbm);
+ else
+ scan->pf_shared_tbmiterator =
+ tbm_attach_shared_iterate(dsa, scan->bm_parallel->prefetch_iterator);
+ }
+#endif /* USE_PREFETCH */
+
node->initialized = true;
@@ -211,36 +194,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -264,23 +217,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->prefetch_iterator &&
- node->pfblockno > node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -304,224 +243,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- tbm_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
@@ -569,22 +291,11 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -625,14 +336,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -665,18 +370,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -716,13 +414,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -806,7 +497,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d..22bdccc2a9 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -81,6 +81,18 @@ typedef struct HeapScanDescData
*/
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..93168bd350 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@ struct ParallelTableScanDescData;
struct TBMIterator;
struct TBMSharedIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -46,6 +47,16 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct TBMIterator *tbmiterator;
struct TBMSharedIterator *shared_tbmiterator;
+ /* Prefetch iterators */
+ struct TBMIterator *pf_tbmiterator;
+ struct TBMSharedIterator *pf_shared_tbmiterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+
+ /* used to validate prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1d4b79a73f..9cab4462d6 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -800,17 +800,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -961,6 +950,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->shared_tbmiterator = NULL;
result->tbmiterator = NULL;
+ result->pf_shared_tbmiterator = NULL;
+ result->pf_tbmiterator = NULL;
+ result->bm_parallel = NULL;
return result;
}
@@ -1029,11 +1021,23 @@ table_endscan(TableScanDesc scan)
scan->shared_tbmiterator = NULL;
}
+ if (scan->pf_shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
+ scan->pf_shared_tbmiterator = NULL;
+ }
+
if (scan->tbmiterator)
{
tbm_end_iterate(scan->tbmiterator);
scan->tbmiterator = NULL;
}
+
+ if (scan->pf_tbmiterator)
+ {
+ tbm_end_iterate(scan->pf_tbmiterator);
+ scan->pf_tbmiterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1054,11 +1058,23 @@ table_rescan(TableScanDesc scan,
scan->shared_tbmiterator = NULL;
}
+ if (scan->pf_shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
+ scan->pf_shared_tbmiterator = NULL;
+ }
+
if (scan->tbmiterator)
{
tbm_end_iterate(scan->tbmiterator);
scan->tbmiterator = NULL;
}
+
+ if (scan->pf_tbmiterator)
+ {
+ tbm_end_iterate(scan->pf_tbmiterator);
+ scan->pf_tbmiterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8688bc5ab0..7a3fdf9cd4 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,19 +1783,11 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,19 +1795,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v8-0014-Unify-parallel-and-serial-BitmapHeapScan-iterator.patch (11.3K, ../../20240323002607.abegehmmjyxrnmm2@liskov/15-v8-0014-Unify-parallel-and-serial-BitmapHeapScan-iterator.patch)
download | inline diff:
From b3dce275a956b109be270a49a0fe3b5adcdf8c10 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 13:43:41 -0400
Subject: [PATCH v8 14/17] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to
both TBMIterator and TBMSharedIterators. This encapsulates the parallel
and serial iterators and their access and makes the bitmap heap scan
code a bit cleaner.
---
src/backend/access/heap/heapam_handler.c | 40 ++++++---------
src/backend/executor/nodeBitmapHeapscan.c | 62 ++++++++++++++++++-----
src/include/access/relscan.h | 20 +++++---
src/include/access/tableam.h | 61 ++++++----------------
src/tools/pgindent/typedefs.list | 1 +
5 files changed, 96 insertions(+), 88 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index efd2784e03..1a9f7b02d1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,6 +55,7 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+
static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
static inline void BitmapPrefetch(HeapScanDesc scan);
@@ -2131,17 +2132,15 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
-
if (scan->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
scan->prefetch_pages--;
}
- else if (prefetch_iterator)
+ else if (scan->rs_base.rs_pf_tbmiterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbm_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
scan->pfblockno = tbmpre.blockno;
}
return;
@@ -2154,8 +2153,6 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
*/
if (scan->rs_base.prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -2175,9 +2172,9 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
* we don't validate the blockno here as we do in non-parallel
* case.
*/
- if (prefetch_iterator)
+ if (scan->rs_base.rs_pf_tbmiterator)
{
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
scan->pfblockno = tbmpre.blockno;
}
}
@@ -2209,10 +2206,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ bhs_iterate(scan->rs_tbmiterator, &tbmres);
if (!BlockNumberIsValid(tbmres.blockno))
{
@@ -2353,7 +2347,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* ahead of the current block.
*/
if (scan->bm_parallel == NULL &&
- scan->pf_tbmiterator &&
+ scan->rs_pf_tbmiterator &&
hscan->pfblockno > hscan->rs_base.blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -2427,22 +2421,20 @@ BitmapPrefetch(HeapScanDesc scan)
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
-
- if (prefetch_iterator)
+ if (scan->rs_base.rs_pf_tbmiterator)
{
while (scan->prefetch_pages < scan->prefetch_target)
{
TBMIterateResult tbmpre;
bool skip_fetch;
- tbm_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- scan->rs_base.pf_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
+ scan->rs_base.rs_pf_tbmiterator = NULL;
break;
}
scan->prefetch_pages++;
@@ -2470,9 +2462,7 @@ BitmapPrefetch(HeapScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
-
- if (prefetch_iterator)
+ if (scan->rs_base.rs_pf_tbmiterator)
{
while (1)
{
@@ -2495,12 +2485,12 @@ BitmapPrefetch(HeapScanDesc scan)
if (!do_prefetch)
return;
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- scan->rs_base.pf_shared_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
+ scan->rs_base.rs_pf_tbmiterator = NULL;
break;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f241f4cb2c..b548642088 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,6 +52,54 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, ParallelBitmapHeapState *pstate, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ if (pstate)
+ result->parallel = tbm_attach_shared_iterate(personal_area, pstate->tbmiterator);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *tbmres)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ tbm_iterate(iterator->serial, tbmres);
+ else
+ tbm_shared_iterate(iterator->parallel, tbmres);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -165,21 +213,11 @@ BitmapHeapNext(BitmapHeapScanState *node)
scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
- if (!scan->bm_parallel)
- scan->tbmiterator = tbm_begin_iterate(tbm);
- else
- /* Allocate a private iterator and attach the shared state to it */
- scan->shared_tbmiterator = tbm_attach_shared_iterate(dsa, scan->bm_parallel->tbmiterator);
+ scan->rs_tbmiterator = bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
#ifdef USE_PREFETCH
if (scan->prefetch_maximum > 0)
- {
- if (!scan->bm_parallel)
- scan->pf_tbmiterator = tbm_begin_iterate(tbm);
- else
- scan->pf_shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, scan->bm_parallel->prefetch_iterator);
- }
+ bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
#endif /* USE_PREFETCH */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 93168bd350..65092d7226 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -27,6 +27,17 @@ struct ParallelTableScanDescData;
struct TBMIterator;
struct TBMSharedIterator;
struct ParallelBitmapHeapState;
+struct TBMIterateResult;
+
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern void bhs_iterate(BitmapHeapIterator *tbmiterator, struct TBMIterateResult *tbmres);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,12 +56,9 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
- /* Prefetch iterators */
- struct TBMIterator *pf_tbmiterator;
- struct TBMSharedIterator *pf_shared_tbmiterator;
-
+ BitmapHeapIterator *rs_tbmiterator;
+ /* Prefetch iterator */
+ BitmapHeapIterator *rs_pf_tbmiterator;
/* maximum value for prefetch_target */
int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 9cab4462d6..2ded1a124b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -948,10 +948,8 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
- result->pf_shared_tbmiterator = NULL;
- result->pf_tbmiterator = NULL;
+ result->rs_tbmiterator = NULL;
+ result->rs_pf_tbmiterator = NULL;
result->bm_parallel = NULL;
return result;
}
@@ -1015,29 +1013,15 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
+ bhs_end_iterate(scan->rs_tbmiterator);
+ scan->rs_tbmiterator = NULL;
+#ifdef USE_PREFETCH
+ if (scan->rs_pf_tbmiterator)
{
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->pf_shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
- scan->pf_shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
-
- if (scan->pf_tbmiterator)
- {
- tbm_end_iterate(scan->pf_tbmiterator);
- scan->pf_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_pf_tbmiterator);
+ scan->rs_pf_tbmiterator = NULL;
}
+#endif
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1052,29 +1036,16 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->pf_shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
- scan->pf_shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_tbmiterator);
+ scan->rs_tbmiterator = NULL;
- if (scan->pf_tbmiterator)
+#ifdef USE_PREFETCH
+ if (scan->rs_pf_tbmiterator)
{
- tbm_end_iterate(scan->pf_tbmiterator);
- scan->pf_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_pf_tbmiterator);
+ scan->rs_pf_tbmiterator = NULL;
}
+#endif
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e2a0525dd4..58317eac44 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v8-0015-Remove-table_scan_bitmap_next_block.patch (11.2K, ../../20240323002607.abegehmmjyxrnmm2@liskov/16-v8-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From e92e2f2f2925bdf79d5fd8a28021ce5b62d2ce06 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v8 15/17] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
5 files changed, 61 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1a9f7b02d1..9dfc99d8fa 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2110,12 +2110,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2148,8 +2142,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2511,30 +2505,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -2916,7 +2923,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index ce637a5a5d..1d6b03d1ca 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -92,8 +92,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index b548642088..2683d8bc0c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -222,44 +222,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2ded1a124b..5ad3eff539 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -788,36 +788,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -2000,44 +1984,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2048,7 +2001,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v8-0016-v7-Streaming-Read-API.patch (56.1K, ../../20240323002607.abegehmmjyxrnmm2@liskov/17-v8-0016-v7-Streaming-Read-API.patch)
download | inline diff:
From d1a23704e59b3a12e3e9b2028b4eaa0147059aeb Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:59:42 -0400
Subject: [PATCH v8 16/17] v7 Streaming Read API
---
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 659 +++++++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 642 +++++++++++++++-------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/include/storage/bufmgr.h | 45 ++
src/include/storage/streaming_read.h | 52 ++
src/tools/pgindent/typedefs.list | 3 +
10 files changed, 1227 insertions(+), 210 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca2..eec03f6f2b 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 0000000000..bcab44c802
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 0000000000..39aef2a84a
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 0000000000..d5c29b750d
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,659 @@
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ */
+typedef struct PgStreamingReadRange
+{
+ bool need_wait;
+ bool advice_issued;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index;
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+ ReadBuffersOperation operation;
+} PgStreamingReadRange;
+
+/*
+ * Streaming read object.
+ */
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int next_tail_buffer;
+ int distance;
+ bool started;
+ bool finished;
+ bool advice_enabled;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ /* Sometimes we need to buffer one block for flow control. */
+ BlockNumber unget_blocknum;
+ void *unget_per_buffer_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /*
+ * Make sure our bmr's smgr and persistent are populated. The caller
+ * asserts that the storage manager will remain valid.
+ */
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+
+ /*
+ * Choose a maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least
+ * MAX_BUFFER_PER_TRANSFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+ pgsr->callback = next_block_cb;
+ pgsr->bmr = bmr;
+ pgsr->forknum = forknum;
+
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out doing full-sized
+ * reads.
+ */
+ if (flags & PGSR_FLAG_FULL)
+ pgsr->distance = Min(MAX_BUFFERS_PER_TRANSFER, pgsr->max_pinned_buffers);
+ else
+ pgsr->distance = 1;
+
+ /*
+ * Space for the callback to store extra data along with each block. Note
+ * that we need one more than max_pinned_buffers, so we can return a
+ * pointer to a slot that can't be overwritten until the next call.
+ */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * size);
+
+ return pgsr;
+}
+
+/*
+ * Find the per-buffer data index for the Nth block of a range.
+ */
+static int
+get_per_buffer_data_index(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ int result;
+
+ /*
+ * Find slot in the circular buffer of per-buffer data, without using the
+ * expensive % operator.
+ */
+ result = range->per_buffer_data_index + n;
+ while (result >= pgsr->size)
+ result -= pgsr->size;
+ Assert(result == (range->per_buffer_data_index + n) % pgsr->size);
+
+ return result;
+}
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static void *
+get_per_buffer_data_by_index(PgStreamingRead *pgsr, int per_buffer_data_index)
+{
+ return (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * per_buffer_data_index;
+}
+
+/*
+ * Return a pointer to the per-buffer data for the Nth block of a range.
+ */
+static void *
+get_per_buffer_data(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ return get_per_buffer_data_by_index(pgsr,
+ get_per_buffer_data_index(pgsr,
+ range,
+ n));
+}
+
+/*
+ * Start reading the head range, and create a new head range. The new head
+ * range is returned. It may not be empty, if StartReadBuffers() couldn't
+ * start the entire range; in that case the returned range contains the
+ * remaining portion of the range.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_start_head_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+ PgStreamingReadRange *new_head_range;
+ int nblocks_pinned;
+ int flags;
+
+ /* Caller should make sure we never exceed max_ios. */
+ Assert((pgsr->ios_in_progress < pgsr->max_ios) ||
+ (pgsr->ios_in_progress == 0 && pgsr->max_ios == 0));
+
+ /* Should only call if the head range has some blocks to read. */
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If advice hasn't been suppressed, and this system supports it, this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (pgsr->advice_enabled &&
+ pgsr->max_ios > 0 &&
+ pgsr->started &&
+ head_range->blocknum != pgsr->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* Suppress advice on the first call, because it's too late to benefit. */
+ if (!pgsr->started)
+ pgsr->started = true;
+
+ /* We shouldn't be trying to pin more buffers that we're allowed to. */
+ Assert(pgsr->pinned_buffers + head_range->nblocks <= pgsr->max_pinned_buffers);
+
+ /* Start reading as many blocks as we can from the head range. */
+ nblocks_pinned = head_range->nblocks;
+ head_range->need_wait =
+ StartReadBuffers(pgsr->bmr,
+ head_range->buffers,
+ pgsr->forknum,
+ head_range->blocknum,
+ &nblocks_pinned,
+ pgsr->strategy,
+ flags,
+ &head_range->operation);
+
+ Assert(pgsr->pinned_buffers <= pgsr->max_pinned_buffers);
+
+ if (head_range->need_wait && (flags & READ_BUFFERS_ISSUE_ADVICE))
+ {
+ /*
+ * Since we've issued advice, we count an I/O in progress until we
+ * call WaitReadBuffers().
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ Assert(pgsr->ios_in_progress <= pgsr->max_ios);
+ }
+
+ /*
+ * StartReadBuffers() might have pinned fewer blocks than we asked it to,
+ * but always at least one.
+ */
+ Assert(nblocks_pinned <= head_range->nblocks);
+ Assert(nblocks_pinned >= 1);
+ pgsr->pinned_buffers += nblocks_pinned;
+
+ /*
+ * Remember where the next block would be after that, so we can detect
+ * sequential access next time.
+ */
+ pgsr->seq_blocknum = head_range->blocknum + nblocks_pinned;
+
+ /*
+ * Create a new head range. There must be space, because we have enough
+ * elements for every range to hold just one block, up to the pin limit.
+ */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ new_head_range = &pgsr->ranges[pgsr->head];
+ new_head_range->nblocks = 0;
+ new_head_range->advice_issued = false;
+
+ /*
+ * If we didn't manage to start the whole read above, we split the range,
+ * moving the remainder into the new head range.
+ */
+ if (nblocks_pinned < head_range->nblocks)
+ {
+ int nblocks_remaining = head_range->nblocks - nblocks_pinned;
+
+ head_range->nblocks = nblocks_pinned;
+
+ new_head_range->blocknum = head_range->blocknum + nblocks_pinned;
+ new_head_range->nblocks = nblocks_remaining;
+ }
+
+ /* The new range has per-buffer data starting after the previous range. */
+ new_head_range->per_buffer_data_index =
+ get_per_buffer_data_index(pgsr, head_range, nblocks_pinned);
+
+ return new_head_range;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow pg_streaming_unget_block() to work.
+ */
+static BlockNumber
+pg_streaming_get_block(PgStreamingRead *pgsr, void *per_buffer_data)
+{
+ BlockNumber result;
+
+ if (unlikely(pgsr->unget_blocknum != InvalidBlockNumber))
+ {
+ /*
+ * If we had to unget a block, now it is time to return that one
+ * again.
+ */
+ result = pgsr->unget_blocknum;
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+ /*
+ * The same per_buffer_data element must have been used, and still
+ * contains whatever data the callback wrote into it. So we just
+ * sanity-check that we were called with the value that
+ * pg_streaming_unget_block() pushed back.
+ */
+ Assert(per_buffer_data == pgsr->unget_per_buffer_data);
+ }
+ else
+ {
+ /* Use the installed callback directly. */
+ result = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ }
+
+ return result;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later. This *must* be called with the
+ * last value returned by pg_streaming_get_block().
+ */
+static void
+pg_streaming_unget_block(PgStreamingRead *pgsr, BlockNumber blocknum, void *per_buffer_data)
+{
+ Assert(pgsr->unget_blocknum == InvalidBlockNumber);
+ pgsr->unget_blocknum = blocknum;
+ pgsr->unget_per_buffer_data = per_buffer_data;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *range;
+
+ /* If we're finished, don't look ahead. */
+ if (pgsr->finished)
+ return;
+
+ /*
+ * We we've already started the maximum allowed number of I/Os, don't look
+ * ahead. There is a special case for max_ios == 0.
+ */
+ if (pgsr->max_ios > 0 && pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /* Can't pin any more buffers. */
+ if (pgsr->pinned_buffers == pgsr->distance)
+ return;
+
+ /*
+ * Keep trying to add new blocks to the end of the head range while doing
+ * so wouldn't exceed the distance limit.
+ */
+ range = &pgsr->ranges[pgsr->head];
+ while (pgsr->pinned_buffers + range->nblocks < pgsr->distance)
+ {
+ BlockNumber blocknum;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ if (range->nblocks == lengthof(range->buffers))
+ {
+ /* Start as much of it as we can. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /* If we're now at the I/O limit, stop here. */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * That might have only been partially started, but always
+ * processes at least one so that'll do for now.
+ */
+ Assert(range->nblocks < lengthof(range->buffers));
+ }
+
+ /* Find per-buffer data slot for the next block. */
+ per_buffer_data = get_per_buffer_data(pgsr, range, range->nblocks);
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pg_streaming_get_block(pgsr, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ pgsr->finished = true;
+ break;
+ }
+
+ /*
+ * Is there a head range that we cannot extend, because the requested
+ * block is not consecutive?
+ */
+ if (range->nblocks > 0 &&
+ range->blocknum + range->nblocks != blocknum)
+ {
+ /* Yes. Start it, so we can begin building a new one. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * It's possible that it was only partially started, and we have a
+ * new range with the remainder. Keep starting I/Os until we get
+ * it all out of the way, or we hit the I/O limit.
+ */
+ while (range->nblocks > 0 && pgsr->ios_in_progress < pgsr->max_ios)
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * We do have to worry about I/O capacity running out if the head
+ * range was split. In that case we have to 'unget' the block
+ * returned by the callback.
+ */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ {
+ pg_streaming_unget_block(pgsr, blocknum, per_buffer_data);
+ return;
+ }
+ }
+
+ /* If we have a new, empty range, initialize the start block. */
+ if (range->nblocks == 0)
+ range->blocknum = blocknum;
+
+ /* This block extends the range by one. */
+ Assert(range->blocknum + range->nblocks == blocknum);
+ range->nblocks++;
+ };
+
+ /*
+ * Normally we don't start the head range, preferring to give it a chance
+ * to grow to full size once more buffers have been consumed. In cases
+ * where that can't possibly happen, we might as well start the read
+ * immediately.
+ */
+ if ((range->nblocks > 0 && pgsr->finished) ||
+ (range->nblocks == pgsr->distance))
+ pg_streaming_read_start_head_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ PgStreamingReadRange *tail_range;
+
+ for (;;)
+ {
+ if (pgsr->tail != pgsr->head)
+ {
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to wait for a ReadBuffers operation to finish before
+ * returning the buffers in this range?
+ */
+ if (tail_range->need_wait)
+ {
+ int distance;
+
+ Assert(pgsr->next_tail_buffer == 0);
+ WaitReadBuffers(&tail_range->operation);
+ tail_range->need_wait = false;
+
+ /*
+ * We don't really know if the kernel generated a physical I/O
+ * when we issued advice, let alone when it finished, but it
+ * has certainly finished now because we've performed the
+ * read.
+ */
+ if (tail_range->advice_issued)
+ {
+
+ Assert(pgsr->ios_in_progress > 0);
+ pgsr->ios_in_progress--;
+
+ /*
+ * Look-ahead distance ramps up rapidly if we're issuing
+ * advice, so we can search for new more I/Os to start.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ else
+ {
+ /*
+ * There is no point in increasing look-ahead distance if
+ * we've already reached the full I/O size, since we're
+ * not issuing advice. Extra distance would only pin more
+ * buffers for no benefit.
+ */
+ if (pgsr->distance > MAX_BUFFERS_PER_TRANSFER)
+ {
+ /*
+ * Look-ahead distance gradually decays to full I/O
+ * size.
+ */
+ pgsr->distance--;
+ }
+ else
+ {
+ /*
+ * Look-ahead distance ramps up rapidly, but not more
+ * that the full I/O size.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, MAX_BUFFERS_PER_TRANSFER);
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ }
+ }
+ else if (pgsr->next_tail_buffer == 0)
+ {
+ /* No I/O necessary. Look-ahead distance gradually decays. */
+ if (pgsr->distance > 1)
+ pgsr->distance--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(pgsr, tail_range, buffer_index);
+
+ /* We may be able to get another I/O started. */
+ pg_streaming_read_look_ahead(pgsr);
+
+ return buffer;
+ }
+
+ /* Advance tail to next range. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+ else
+ {
+ /*
+ * If tail crashed into head, and head is not empty, then it is
+ * time to start that range. Otherwise, force a look-ahead, to
+ * kick start the stream.
+ */
+ Assert(pgsr->tail == pgsr->head);
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ {
+ pg_streaming_read_start_head_range(pgsr);
+ }
+ else
+ {
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* Finished? */
+ if (pgsr->tail == pgsr->head &&
+ pgsr->ranges[pgsr->head].nblocks == 0)
+ break;
+ }
+ }
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ pgsr->finished = true;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(pgsr->pinned_buffers == 0);
+ Assert(pgsr->ios_in_progress == 0);
+
+ /* Release memory. */
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c..d0e9c7deff 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -471,10 +476,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +504,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +785,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +797,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +819,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +990,68 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
+/*
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
+ */
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* ReadBuffer_common -- common logic for all ReadBuffer variants
*
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+ BufferAccessStrategy strategy)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int nblocks;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1070,405 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ nblocks = 1;
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
+ else
+ flags = 0;
+ if (StartReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ &nblocks,
+ strategy,
+ flags,
+ &operation))
+ WaitReadBuffers(&operation);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ ZeroBuffer(buffer, mode);
+
+ return buffer;
+}
+static Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested.
+ *
+ * If false is returned, no I/O is necessary and WaitReadBuffers() is not
+ * necessary. If true is returned, one I/O has been started, and
+ * WaitReadBuffers() must be called with the same operation object before the
+ * buffers are accessed. Along with the operation object, the caller-supplied
+ * array of buffers must remain valid until WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+bool
+StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation)
+{
+ int actual_nblocks = *nblocks;
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ operation->bmr = bmr;
+ operation->forknum = forkNum;
+ operation->blocknum = blockNum;
+ operation->buffers = buffers;
+ operation->nblocks = actual_nblocks;
+ operation->strategy = strategy;
+ operation->flags = flags;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ operation->io_buffers_len = 0;
- /*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
- */
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
- else
+ for (int i = 0; i < actual_nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ bool found;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ buffers[i] = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum + i,
+ strategy,
+ &found);
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = operation->nblocks = *nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ operation->io_buffers_len++;
+ }
+ }
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (operation->io_buffers_len > 0)
+ {
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
- {
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
+ /*
+ * In theory we should only do this if PrepareReadBuffers() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(bmr.smgr, forkNum, blockNum, operation->io_buffers_len);
}
+
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
}
+ else
+ {
+ return false;
+ }
+}
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
{
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ /*
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
+ */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+ bmr = operation->bmr;
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
-
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
}
else
{
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
}
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ /*
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
+ */
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
+ else
+ pgBufferUsage.shared_blks_read += nblocks;
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ for (int i = 0; i < nblocks; ++i)
+ {
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- return BufferDescriptorGetBuffer(bufHdr);
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
+
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
+ {
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
+ {
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
+ }
+ else
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
+
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
+
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
+
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
+
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for StartReadBuffers. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1476,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1535,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1603,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1638,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +1994,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2259,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2282,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2372,7 +2597,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2611,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3679,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5184,9 +5414,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5435,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a5..985a2c7049 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca2..739d13293f 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d335..b57f71f97e 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,42 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+/*
+ * Private state used by StartReadBuffers() and WaitReadBuffers(). Declared
+ * in public header only to allow inclusion in other structs, but contents
+ * should not be accessed.
+ */
+struct ReadBuffersOperation
+{
+ /* Parameters passed in to StartReadBuffers(). */
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ int nblocks;
+ BufferAccessStrategy strategy;
+ int flags;
+
+ /* Range of buffers, if we need to perform a read. */
+ int io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +292,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 0000000000..c4d3892bb2
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,52 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define PGSR_FLAG_FULL 0x04
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 58317eac44..162fe9300a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2116,6 +2116,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
@@ -2287,6 +2289,7 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
--
2.40.1
[text/x-diff] v8-0017-BitmapHeapScan-uses-streaming-read-API.patch (26.5K, ../../20240323002607.abegehmmjyxrnmm2@liskov/18-v8-0017-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From 830584b381e185e824350c2750ddfd7b7f091d41 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v8 17/17] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
ci-os-only:
---
src/backend/access/heap/heapam.c | 96 ++++--
src/backend/access/heap/heapam_handler.c | 346 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 44 +--
src/include/access/heapam.h | 16 +-
src/include/access/relscan.h | 7 -
src/include/access/tableam.h | 16 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 116 insertions(+), 418 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index c12563a188..1beb0bb99b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -111,6 +111,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -331,6 +333,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -951,16 +969,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1043,12 +1054,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1061,12 +1066,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1092,12 +1091,6 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1115,6 +1108,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10272,3 +10268,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 9dfc99d8fa..fe9ee5976f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -56,10 +56,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
-
static const TableAmRoutine heapam_methods;
@@ -2110,146 +2106,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (scan->rs_base.rs_pf_tbmiterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (scan->rs_base.rs_pf_tbmiterator)
- {
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2270,7 +2188,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2279,9 +2197,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2331,23 +2249,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_tbmiterator &&
- hscan->pfblockno > hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2358,152 +2264,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
-
- if (pstate == NULL)
- {
- if (scan->rs_base.rs_pf_tbmiterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
- scan->rs_base.rs_pf_tbmiterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (scan->rs_base.rs_pf_tbmiterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
- scan->rs_base.rs_pf_tbmiterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2538,41 +2298,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2683d8bc0c..c21dae466a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -128,14 +128,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -146,15 +138,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -171,13 +154,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -210,17 +187,10 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_tbmiterator = bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
-#endif /* USE_PREFETCH */
-
-
node->initialized = true;
}
@@ -319,7 +289,7 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
node->tbm = NULL;
@@ -363,7 +333,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
if (node->tbm)
tbm_free(node->tbm);
@@ -519,14 +489,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -557,11 +523,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 22bdccc2a9..1cad9c04f0 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -81,18 +85,6 @@ typedef struct HeapScanDescData
*/
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used for prefetching in bitmap table scans
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 65092d7226..00a123ee92 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -57,15 +57,8 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
BitmapHeapIterator *rs_tbmiterator;
- /* Prefetch iterator */
- BitmapHeapIterator *rs_pf_tbmiterator;
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate prefetch and current block stay in sync */
- BlockNumber blockno;
-
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5ad3eff539..4bc021f0eb 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -933,7 +933,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_tbmiterator = NULL;
- result->rs_pf_tbmiterator = NULL;
result->bm_parallel = NULL;
return result;
}
@@ -999,13 +998,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_tbmiterator);
scan->rs_tbmiterator = NULL;
-#ifdef USE_PREFETCH
- if (scan->rs_pf_tbmiterator)
- {
- bhs_end_iterate(scan->rs_pf_tbmiterator);
- scan->rs_pf_tbmiterator = NULL;
- }
-#endif
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1022,14 +1014,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_tbmiterator);
scan->rs_tbmiterator = NULL;
-
-#ifdef USE_PREFETCH
- if (scan->rs_pf_tbmiterator)
- {
- bhs_end_iterate(scan->rs_pf_tbmiterator);
- scan->rs_pf_tbmiterator = NULL;
- }
-#endif
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 7a3fdf9cd4..c1778653c7 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1758,11 +1758,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1770,10 +1766,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-24 12:36 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-24 12:36 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/23/24 01:26, Melanie Plageman wrote:
> On Fri, Mar 22, 2024 at 08:22:11PM -0400, Melanie Plageman wrote:
>> On Tue, Mar 19, 2024 at 02:33:35PM +0200, Heikki Linnakangas wrote:
>>> On 18/03/2024 17:19, Melanie Plageman wrote:
>>>> I've attached v7 rebased over this commit.
>>>
>>> If we delayed table_beginscan_bm() call further, after starting the TBM
>>> iterator, we could skip it altogether when the iterator is empty.
>>>
>>> That's a further improvement, doesn't need to be part of this patch set.
>>> Just caught my eye while reading this.
>>
>> Hmm. You mean like until after the first call to tbm_[shared]_iterate()?
>> AFAICT, tbm_begin_iterate() doesn't tell us anything about whether or
>> not the iterator is "empty". Do you mean cases when the bitmap has no
>> blocks in it? It seems like we should be able to tell that from the
>> TIDBitmap.
>>
>>>
>>>> v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch
>>>
>>> I suggest to avoid the double negative with SO_CAN_SKIP_FETCH, and call the
>>> flag e.g. SO_NEED_TUPLE.
>>
>> Agreed. Done in attached v8. Though I wondered if it was a bit weird
>> that the flag is set in the common case and not set in the uncommon
>> case...
>
> v8 actually attached this time
I tried to run the benchmarks with v8, but unfortunately it crashes for
me very quickly (I've only seen 0015 to crash, so I guess the bug is in
that patch).
The backtrace attached, this doesn't seem right:
(gdb) p hscan->rs_cindex
$1 = 543516018
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Core was generated by `postgres: postgres test-1000000 [local] SELECT '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000556f99ca7cd0 in heapam_scan_bitmap_next_tuple (scan=0x556f9b9e6960, slot=0x556f9b9ef440, recheck=0x556f9b9eec10, lossy_pages=0x556f9b9eebf8, exact_pages=0x556f9b9eebf0)
at heapam_handler.c:2576
warning: Source file is more recent than executable.
2576 targoffset = hscan->rs_vistuples[hscan->rs_cindex];
(gdb) bt
#0 0x0000556f99ca7cd0 in heapam_scan_bitmap_next_tuple (scan=0x556f9b9e6960, slot=0x556f9b9ef440, recheck=0x556f9b9eec10, lossy_pages=0x556f9b9eebf8, exact_pages=0x556f9b9eebf0)
at heapam_handler.c:2576
#1 0x0000556f99e17adb in table_scan_bitmap_next_tuple (exact_pages=0x556f9b9eebf0, lossy_pages=0x556f9b9eebf8, recheck=0x556f9b9eec10, slot=0x556f9b9ef440, scan=0x556f9b9e6960)
at ../../../src/include/access/tableam.h:2003
#2 BitmapHeapNext (node=0x556f9b9eeb00) at nodeBitmapHeapscan.c:227
#3 0x0000556f99e1a331 in ExecProcNode (node=0x556f9b9eeb00) at ../../../src/include/executor/executor.h:274
#4 gather_getnext (gatherstate=0x556f9b9ee970) at nodeGather.c:287
#5 ExecGather (pstate=0x556f9b9ee970) at nodeGather.c:222
#6 0x0000556f99e24ac3 in ExecProcNode (node=0x556f9b9ee970) at ../../../src/include/executor/executor.h:274
#7 ExecLimit (pstate=0x556f9b9ee698) at nodeLimit.c:95
#8 0x0000556f99e0174a in ExecProcNode (node=0x556f9b9ee698) at ../../../src/include/executor/executor.h:274
#9 ExecutePlan (execute_once=<optimized out>, dest=0x556f9b9f2360, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>,
planstate=0x556f9b9ee698, estate=0x556f9b9ee458) at execMain.c:1644
#10 standard_ExecutorRun (queryDesc=0x556f9b944f88, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
#11 0x0000556f99f9cc7f in PortalRunSelect (portal=portal@entry=0x556f9b997008, forward=forward@entry=true, count=0, count@entry=9223372036854775807, dest=dest@entry=0x556f9b9f2360)
at pquery.c:924
#12 0x0000556f99f9dffa in PortalRun (portal=portal@entry=0x556f9b997008, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true,
dest=dest@entry=0x556f9b9f2360, altdest=altdest@entry=0x556f9b9f2360, qc=0x7ffc123a0b60) at pquery.c:768
#13 0x0000556f99f9a57c in exec_simple_query (query_string=0x556f9b91ab18 "SELECT * FROM linear WHERE (a BETWEEN 2013 AND 8061) OFFSET 1000000;") at postgres.c:1274
#14 0x0000556f99f9c051 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
#15 0x0000556f99f96def in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
#16 0x0000556f99f0c564 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffc123a0f90 "", startup_data_len=startup_data_len@entry=4,
client_sock=client_sock@entry=0x7ffc123a0fb0) at launch_backend.c:265
#17 0x0000556f99f0fea9 in BackendStartup (client_sock=0x7ffc123a0fb0) at postmaster.c:3593
#18 ServerLoop () at postmaster.c:1674
#19 0x0000556f99f11b50 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x556f9b915260) at postmaster.c:1372
#20 0x0000556f99c5b0c3 in main (argc=3, argv=0x556f9b915260) at main.c:197
(gdb) p hscan->rs_cindex
$1 = 543516018
(gdb) p hscan
$2 = (HeapScanDesc) 0x556f9b9e6960
(gdb) p *hscan
$3 = {rs_base = {rs_rd = 0x7f65fb83e130, rs_snapshot = 0x556f9b94cf60, rs_nkeys = 0, rs_key = 0x0, rs_mintid = {ip_blkid = {bi_hi = 31090, bi_lo = 25673}, ip_posid = 12320}, rs_maxtid = {
ip_blkid = {bi_hi = 14880, bi_lo = 24936}, ip_posid = 21107}, rs_tbmiterator = 0x556f9ba18f80, rs_pf_tbmiterator = 0x0, prefetch_maximum = 8, bm_parallel = 0x7f65fb7991c0,
blockno = 4294967295, rs_flags = 1282, rs_parallel = 0x0}, rs_nblocks = 41667, rs_startblock = 0, rs_numblocks = 4294967295, rs_inited = false, rs_coffset = 28257,
rs_cblock = 4294967295, rs_cbuf = 0, rs_strategy = 0x0, rs_ctup = {t_len = 1819234926, t_self = {ip_blkid = {bi_hi = 65535, bi_lo = 65535}, ip_posid = 0}, t_tableOid = 16416,
t_data = 0x0}, rs_parallelworkerdata = 0x0, rs_vmbuffer = 0, rs_empty_tuples_pending = 0, pvmbuffer = 0, prefetch_target = -1, prefetch_pages = 0, pfblockno = 4294967295,
rs_cindex = 543516018, rs_ntuples = 1414087291, rs_vistuples = {16717, 18512, 16709, 21328, 16707, 8270, 29498, 24931, 11886, 27760, 28257, 29486, 24948, 29810, 28789, 25439, 29551,
8308, 14646, 13364, 14638, 13362, 13618, 14880, 25459, 28257, 28718, 24940, 11886, 28532, 24948, 24428, 28515, 29811, 13600, 13106, 13618, 13102, 13365, 13618, 12336, 12336, 12336,
8244, 29498, 24931, 11886, 27760, 28257, 28718, 24940, 24430, 28530, 29559, 12576, 12341, 12856, 8249, 29498, 24931, 11886, 27760, 28257, 28718, 24940, 24430, 26999, 29796, 8296,
14644, 14880, 25459, 28257, 28718, 24940, 11886, 24944, 24946, 27756, 27749, 24927, 24951, 25970, 29728, 30066, 8293, 29498, 24931, 11886, 27760, 28257, 28718, 29281, 27745, 25964,
24428, 24947, 25958, 29728, 30066, 8293, 29498, 24931, 11886, 27760, 28257, 24878, 31091, 25454, 25439, 28769, 25185, 25964, 26144, 27745, 25971, 14880, 25459, 28257, 28718, 24940,
11886, 27760, 28257, 28255, 25711, 24421, 25705, 12832, 14880, 25459, 28257, 28718, 24940, 11886, 24948, 26482, 29797, 26988, 29811, 10272, 21627, 21057, 17735, 17748, 21582, 22866,
14880, 30821, 29296, 31520, 16726, 8274, 30266, 29281, 28526, 12576, 14880, 24950, 24946, 29812, 28526, 12576, 14880, 24950, 29810, 28793, 8293, 12338, 14880, 24950, 29810, 28793,
28525, 8292, 12589, 14880, 24950, 25458, 27759, 26988, 8292, 8240, 30266, 29281, 30062, 27756, 28265, 29287, 27749, 8307, 25128, 8233, 30266, 29281, 25964, 25974, 29548, 28789,
12320...}}
(gdb) bt full
#0 0x000055e7d81c6cd0 in heapam_scan_bitmap_next_tuple (scan=0x55e7d98fd140, slot=0x55e7d98eb440, recheck=0x55e7d98eac10, lossy_pages=0x55e7d98eabf8, exact_pages=0x55e7d98eabf0)
at heapam_handler.c:2576
hscan = 0x55e7d98fd140
targoffset = <optimized out>
page = <optimized out>
lp = <optimized out>
__func__ = "heapam_scan_bitmap_next_tuple"
#1 0x000055e7d8336adb in table_scan_bitmap_next_tuple (exact_pages=0x55e7d98eabf0, lossy_pages=0x55e7d98eabf8, recheck=0x55e7d98eac10, slot=0x55e7d98eb440, scan=0x55e7d98fd140)
at ../../../src/include/access/tableam.h:2003
__func__ = "table_scan_bitmap_next_tuple"
__errno_location = <optimized out>
#2 BitmapHeapNext (node=0x55e7d98eab00) at nodeBitmapHeapscan.c:227
econtext = 0x55e7d98ead08
scan = 0x55e7d98fd140
tbm = <optimized out>
slot = 0x55e7d98eb440
dsa = <optimized out>
__func__ = "BitmapHeapNext"
#3 0x000055e7d8339331 in ExecProcNode (node=0x55e7d98eab00) at ../../../src/include/executor/executor.h:274
No locals.
#4 gather_getnext (gatherstate=0x55e7d98ea970) at nodeGather.c:287
estate = 0x55e7d98ea458
outerPlan = 0x55e7d98eab00
outerTupleSlot = <optimized out>
fslot = 0x55e7d98ebf10
tup = <optimized out>
outerPlan = <optimized out>
outerTupleSlot = <optimized out>
fslot = <optimized out>
tup = <optimized out>
estate = <optimized out>
#5 ExecGather (pstate=0x55e7d98ea970) at nodeGather.c:222
node = <optimized out>
slot = <optimized out>
econtext = 0x55e7d98eaa78
#6 0x000055e7d8343ac3 in ExecProcNode (node=0x55e7d98ea970) at ../../../src/include/executor/executor.h:274
No locals.
#7 ExecLimit (pstate=0x55e7d98ea698) at nodeLimit.c:95
node = 0x55e7d98ea698
econtext = 0x55e7d98ea8a0
direction = <optimized out>
slot = <optimized out>
outerPlan = 0x55e7d98ea970
__func__ = "ExecLimit"
#8 0x000055e7d832074a in ExecProcNode (node=0x55e7d98ea698) at ../../../src/include/executor/executor.h:274
No locals.
#9 ExecutePlan (execute_once=<optimized out>, dest=0x55e7d98ee380, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>,
planstate=0x55e7d98ea698, estate=0x55e7d98ea458) at execMain.c:1644
slot = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
current_tuple_count = 0
slot = <optimized out>
current_tuple_count = <optimized out>
#10 standard_ExecutorRun (queryDesc=0x55e7d9840f88, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
estate = 0x55e7d98ea458
operation = CMD_SELECT
dest = 0x55e7d98ee380
sendTuples = <optimized out>
oldcontext = 0x55e7d9840e90
__func__ = "standard_ExecutorRun"
#11 0x000055e7d84bbc7f in PortalRunSelect (portal=portal@entry=0x55e7d9893008, forward=forward@entry=true, count=0, count@entry=9223372036854775807, dest=dest@entry=0x55e7d98ee380)
at pquery.c:924
queryDesc = 0x55e7d9840f88
direction = <optimized out>
nprocessed = <optimized out>
__func__ = "PortalRunSelect"
#12 0x000055e7d84bcffa in PortalRun (portal=portal@entry=0x55e7d9893008, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true,
dest=dest@entry=0x55e7d98ee380, altdest=altdest@entry=0x55e7d98ee380, qc=0x7ffd7aceec20) at pquery.c:768
_save_exception_stack = 0x7ffd7aceee60
_save_context_stack = 0x0
_local_sigjmp_buf = {{__jmpbuf = {1, 6450011384570971824, 140726663834656, 94454275826560, 94454275452936, 94454274947640, 988062591367775920, 6450008691990202032},
__mask_was_saved = 0, __saved_mask = {__val = {94453442954861, 94450625806337, 0, 94450625806336, 94454256175041, 0, 94454275461144, 94454257765830, 94454275452936,
94454257765830, 1, 94454275825720, 0, 94454274947640, 11208266538747310592, 94454274947976}}}}
_do_rethrow = <optimized out>
result = <optimized out>
nprocessed = <optimized out>
saveTopTransactionResourceOwner = 0x55e7d984e6e0
saveTopTransactionContext = 0x55e7d9848e40
saveActivePortal = 0x0
saveResourceOwner = 0x55e7d984e6e0
savePortalContext = 0x0
saveMemoryContext = 0x55e7d9848e40
__func__ = "PortalRun"
#13 0x000055e7d84b957c in exec_simple_query (query_string=0x55e7d9816b18 "SELECT * FROM cyclic_fuzz WHERE (a BETWEEN 66 AND 82) OFFSET 1000000;") at postgres.c:1274
cmdtaglen = 6
snapshot_set = <optimized out>
per_parsetree_context = 0x0
plantree_list = <optimized out>
parsetree = 0x55e7d9817a10
commandTag = <optimized out>
qc = {commandTag = CMDTAG_UNKNOWN, nprocessed = 0}
querytree_list = <optimized out>
portal = 0x55e7d9893008
receiver = 0x55e7d98ee380
format = 0
cmdtagname = <optimized out>
parsetree_item__state = {l = 0x55e7d9817a38, i = 0}
dest = DestRemote
oldcontext = 0x55e7d9848e40
--Type <RET> for more, q to quit, c to continue without paging--
parsetree_list = 0x55e7d9817a38
parsetree_item = <optimized out>
save_log_statement_stats = false
was_logged = false
use_implicit_block = false
msec_str = "\000\000\000\000\000\000\000\000\0002\356u\bË\233\340\216\224\330\347U\000\000\000 \000\000\000\000\000"
__func__ = "exec_simple_query"
#14 0x000055e7d84bb051 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
query_string = 0x55e7d9816b18 "SELECT * FROM cyclic_fuzz WHERE (a BETWEEN 66 AND 82) OFFSET 1000000;"
firstchar = <optimized out>
input_message = {data = 0x55e7d9816b18 "SELECT * FROM cyclic_fuzz WHERE (a BETWEEN 66 AND 82) OFFSET 1000000;", len = 70, maxlen = 1024, cursor = 70}
local_sigjmp_buf = {{__jmpbuf = {140726663835040, 6450011346656560816, 94454254631152, 140726663835760, 140726663835760, 140726663835928, 988062591193712304, 6450008694811133616},
__mask_was_saved = 1, __saved_mask = {__val = {4194304, 0, 0, 0, 0, 0, 0, 0, 140311975264413, 94454275173984, 0, 0, 140311975264413, 94454275190384, 140311976438464, 1023}}}}
send_ready_for_query = false
idle_in_transaction_timeout_enabled = false
idle_session_timeout_enabled = false
__func__ = "PostgresMain"
#15 0x000055e7d84b5def in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
bsdata = <optimized out>
#16 0x000055e7d842b564 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffd7acef050 "", startup_data_len=startup_data_len@entry=4,
client_sock=client_sock@entry=0x7ffd7acef070) at launch_backend.c:265
pid = <optimized out>
#17 0x000055e7d842eea9 in BackendStartup (client_sock=0x7ffd7acef070) at postmaster.c:3593
bn = 0x55e7d9818940
pid = <optimized out>
startup_data = {canAcceptConnections = CAC_OK}
bn = <optimized out>
pid = <optimized out>
startup_data = <optimized out>
__func__ = "BackendStartup"
__errno_location = <optimized out>
__errno_location = <optimized out>
save_errno = <optimized out>
__errno_location = <optimized out>
__errno_location = <optimized out>
#18 ServerLoop () at postmaster.c:1674
s = {sock = 9, raddr = {addr = {ss_family = 1,
__ss_padding = "\000\000\347U\000\000\254gy\330\347U\000\0008709", '\000' <repeats 21 times>, "\004\000\000\000\000\000\000\272Ja\330\347U\000\000\360\360\316z\375\177\000\000`\251s\330\347U\000\000\360\360\316z\375\177\000\000\357\001_\330\347U", '\000' <repeats 18 times>, " j\201\331\347U\000\000ZJ\a\330\347U\000", __ss_align = 4269206038}, salen = 2}}
i = 0
now = <optimized out>
last_lockfile_recheck_time = 1711283252
last_touch_time = 1711283252
events = {{pos = 2, events = 2, fd = 7, user_data = 0x0}, {pos = -303599753, events = 32668, fd = 5, user_data = 0x7f9ced682d70}, {pos = 1, events = 0, fd = 0, user_data = 0x1}, {
pos = -303439168, events = 32668, fd = 0, user_data = 0x7f9cede9e2c0}, {pos = 1, events = 21991, fd = 2060382680, user_data = 0x7f9cede9e648}, {pos = 0, events = 0, fd = 0,
user_data = 0x55e7d9840910}, {pos = -1, events = 0, fd = 2060382680, user_data = 0x7ffd7acef1e7}, {pos = 0, events = 0, fd = 2060382692,
user_data = 0x7f9ced797d0e <inet_aton_end+110>}, {pos = 2060382780, events = 32765, fd = 2060382784, user_data = 0x55e7d9841d80}, {pos = -645658351, events = 21991, fd = 6,
user_data = 0x9b8bc30875ee3200}, {pos = 268, events = 0, fd = 2060383156, user_data = 0x7ffd7acef9d0}, {pos = -310803038, events = 32668, fd = 0,
user_data = 0x100000000000000}, {pos = -645648096, events = 21991, fd = 1978544640, user_data = 0x55e7d9818740}, {pos = -645643304, events = 21991, fd = 10,
--Type <RET> for more, q to quit, c to continue without paging--
user_data = 0x7f9ced7495dc <gaih_result_reset+28>}, {pos = 10, events = 0, fd = -311115396, user_data = 0x55e7d9811260}, {pos = 3, events = 0, fd = -664429546,
user_data = 0x7f9cede7f35a <_dl_runtime_resolve_xsave+138>}, {pos = 2060384808, events = 32765, fd = -645667952, user_data = 0x37}, {pos = 1, events = 0, fd = 2060383216,
user_data = 0x0}, {pos = 0, events = 0, fd = 2060383376, user_data = 0x7ffd7acef330}, {pos = -645643304, events = 21991, fd = 1, user_data = 0x7ffd7acefa40}, {pos = 10,
events = 32765, fd = 76, user_data = 0x2e2f706d742f}, {pos = 1364412240, events = 875900492, fd = -645653120, user_data = 0x0}, {pos = 0, events = 0, fd = -661294368,
user_data = 0x55e7d874e4b5}, {pos = -664536538, events = 21991, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x7f9cede6c540}, {pos = 2060383256,
events = 32765, fd = 1918989871, user_data = 0x65726774736f702f}, {pos = 796091500, events = 1882023792, fd = 1751348321, user_data = 0xff}, {pos = 0, events = 0, fd = 255,
user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0xa}, {pos = 0, events = 0, fd = 16777216, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {
pos = 0, events = 0, fd = 1, user_data = 0x100000000}, {pos = 2, events = 17, fd = -303603420, user_data = 0x7f9cede6c540}, {pos = -311915736, events = 32668, fd = 2090263927,
user_data = 0x7f9ced687e9c}, {pos = 2060383508, events = 32765, fd = -303602277, user_data = 0x8e3}, {pos = -311841864, events = 32668, fd = -303643328,
user_data = 0x7ffd7acef518}, {pos = 2060383508, events = 32765, fd = 0, user_data = 0x55e7d8074923}, {pos = 0, events = 0, fd = -670870984, user_data = 0x1f25b95}, {
pos = 2060383728, events = 32765, fd = 2060383720, user_data = 0x7f9ced682ce8}, {pos = 0, events = 0, fd = 10, user_data = 0x0}, {pos = 0, events = 0, fd = -303603420,
user_data = 0x7f9cede6c540}, {pos = -311915736, events = 32668, fd = 259272455, user_data = 0x7f9ced686728}, {pos = 2060383700, events = 32765, fd = -303602277,
user_data = 0x306}, {pos = -311841864, events = 32668, fd = -303643328, user_data = 0x0}, {pos = 2060383700, events = 32765, fd = -670611165, user_data = 0x7c96e577}, {
pos = -311939728, events = 32668, fd = -303438264, user_data = 0x7ffd7acef5e8}, {pos = 2060383872, events = 32765, fd = -303599753, user_data = 0x5}, {pos = -311939728,
events = 32668, fd = 1, user_data = 0x0}, {pos = 1, events = 0, fd = -303439168, user_data = 0x55e7d875284d}, {pos = -303439168, events = 32668, fd = 1,
user_data = 0x7ffd7acef5f0}, {pos = -303438264, events = 32668, fd = 0, user_data = 0x55e7d8659a89}, {pos = -311163099, events = 32668, fd = 0, user_data = 0x8}, {
pos = 2060383728, events = 32765, fd = -1, user_data = 0x7f9ced696070}, {pos = -303643328, events = 32668, fd = 2, user_data = 0x46000041c0}, {pos = 70, events = 0, fd = 0,
user_data = 0x1000}, {pos = -303439168, events = 32668, fd = 245, user_data = 0x55e7d80f3750}, {pos = -661461080, events = 21991, fd = 3, user_data = 0x55e7d875284d}, {
pos = -303574238, events = 32668, fd = 1, user_data = 0x0}, {pos = -663359120, events = 21991, fd = -663359139, user_data = 0x7f9ced696070}, {pos = -311128592, events = 32668,
fd = -661290912, user_data = 0x7ffd7acefa40}, {pos = 2060384864, events = 32765, fd = 0, user_data = 0x0}, {pos = -303565990, events = 32668, fd = 0,
user_data = 0x7f9ced70fd84 <__GI___pthread_sigmask+68>}, {pos = 0, events = 0, fd = -661344352, user_data = 0x2}, {pos = 739479668, events = 0, fd = 2060746880,
user_data = 0x55e7d80f3330}}
nevents = <optimized out>
__func__ = "ServerLoop"
#19 0x000055e7d8430b50 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x55e7d9811260) at postmaster.c:1372
opt = <optimized out>
status = <optimized out>
userDoption = <optimized out>
listen_addr_saved = <optimized out>
output_config_variable = <optimized out>
__func__ = "PostmasterMain"
#20 0x000055e7d817a0c3 in main (argc=3, argv=0x55e7d9811260) at main.c:197
do_check_root = <optimized out>
Attachments:
[text/plain] crash.txt (19.1K, ../../e472f0cb-bdcd-4fe9-9bf6-23a041dedc5d@enterprisedb.com/2-crash.txt)
download | inline:
Core was generated by `postgres: postgres test-1000000 [local] SELECT '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000556f99ca7cd0 in heapam_scan_bitmap_next_tuple (scan=0x556f9b9e6960, slot=0x556f9b9ef440, recheck=0x556f9b9eec10, lossy_pages=0x556f9b9eebf8, exact_pages=0x556f9b9eebf0)
at heapam_handler.c:2576
warning: Source file is more recent than executable.
2576 targoffset = hscan->rs_vistuples[hscan->rs_cindex];
(gdb) bt
#0 0x0000556f99ca7cd0 in heapam_scan_bitmap_next_tuple (scan=0x556f9b9e6960, slot=0x556f9b9ef440, recheck=0x556f9b9eec10, lossy_pages=0x556f9b9eebf8, exact_pages=0x556f9b9eebf0)
at heapam_handler.c:2576
#1 0x0000556f99e17adb in table_scan_bitmap_next_tuple (exact_pages=0x556f9b9eebf0, lossy_pages=0x556f9b9eebf8, recheck=0x556f9b9eec10, slot=0x556f9b9ef440, scan=0x556f9b9e6960)
at ../../../src/include/access/tableam.h:2003
#2 BitmapHeapNext (node=0x556f9b9eeb00) at nodeBitmapHeapscan.c:227
#3 0x0000556f99e1a331 in ExecProcNode (node=0x556f9b9eeb00) at ../../../src/include/executor/executor.h:274
#4 gather_getnext (gatherstate=0x556f9b9ee970) at nodeGather.c:287
#5 ExecGather (pstate=0x556f9b9ee970) at nodeGather.c:222
#6 0x0000556f99e24ac3 in ExecProcNode (node=0x556f9b9ee970) at ../../../src/include/executor/executor.h:274
#7 ExecLimit (pstate=0x556f9b9ee698) at nodeLimit.c:95
#8 0x0000556f99e0174a in ExecProcNode (node=0x556f9b9ee698) at ../../../src/include/executor/executor.h:274
#9 ExecutePlan (execute_once=<optimized out>, dest=0x556f9b9f2360, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>,
planstate=0x556f9b9ee698, estate=0x556f9b9ee458) at execMain.c:1644
#10 standard_ExecutorRun (queryDesc=0x556f9b944f88, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
#11 0x0000556f99f9cc7f in PortalRunSelect (portal=portal@entry=0x556f9b997008, forward=forward@entry=true, count=0, count@entry=9223372036854775807, dest=dest@entry=0x556f9b9f2360)
at pquery.c:924
#12 0x0000556f99f9dffa in PortalRun (portal=portal@entry=0x556f9b997008, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true,
dest=dest@entry=0x556f9b9f2360, altdest=altdest@entry=0x556f9b9f2360, qc=0x7ffc123a0b60) at pquery.c:768
#13 0x0000556f99f9a57c in exec_simple_query (query_string=0x556f9b91ab18 "SELECT * FROM linear WHERE (a BETWEEN 2013 AND 8061) OFFSET 1000000;") at postgres.c:1274
#14 0x0000556f99f9c051 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
#15 0x0000556f99f96def in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
#16 0x0000556f99f0c564 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffc123a0f90 "", startup_data_len=startup_data_len@entry=4,
client_sock=client_sock@entry=0x7ffc123a0fb0) at launch_backend.c:265
#17 0x0000556f99f0fea9 in BackendStartup (client_sock=0x7ffc123a0fb0) at postmaster.c:3593
#18 ServerLoop () at postmaster.c:1674
#19 0x0000556f99f11b50 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x556f9b915260) at postmaster.c:1372
#20 0x0000556f99c5b0c3 in main (argc=3, argv=0x556f9b915260) at main.c:197
(gdb) p hscan->rs_cindex
$1 = 543516018
(gdb) p hscan
$2 = (HeapScanDesc) 0x556f9b9e6960
(gdb) p *hscan
$3 = {rs_base = {rs_rd = 0x7f65fb83e130, rs_snapshot = 0x556f9b94cf60, rs_nkeys = 0, rs_key = 0x0, rs_mintid = {ip_blkid = {bi_hi = 31090, bi_lo = 25673}, ip_posid = 12320}, rs_maxtid = {
ip_blkid = {bi_hi = 14880, bi_lo = 24936}, ip_posid = 21107}, rs_tbmiterator = 0x556f9ba18f80, rs_pf_tbmiterator = 0x0, prefetch_maximum = 8, bm_parallel = 0x7f65fb7991c0,
blockno = 4294967295, rs_flags = 1282, rs_parallel = 0x0}, rs_nblocks = 41667, rs_startblock = 0, rs_numblocks = 4294967295, rs_inited = false, rs_coffset = 28257,
rs_cblock = 4294967295, rs_cbuf = 0, rs_strategy = 0x0, rs_ctup = {t_len = 1819234926, t_self = {ip_blkid = {bi_hi = 65535, bi_lo = 65535}, ip_posid = 0}, t_tableOid = 16416,
t_data = 0x0}, rs_parallelworkerdata = 0x0, rs_vmbuffer = 0, rs_empty_tuples_pending = 0, pvmbuffer = 0, prefetch_target = -1, prefetch_pages = 0, pfblockno = 4294967295,
rs_cindex = 543516018, rs_ntuples = 1414087291, rs_vistuples = {16717, 18512, 16709, 21328, 16707, 8270, 29498, 24931, 11886, 27760, 28257, 29486, 24948, 29810, 28789, 25439, 29551,
8308, 14646, 13364, 14638, 13362, 13618, 14880, 25459, 28257, 28718, 24940, 11886, 28532, 24948, 24428, 28515, 29811, 13600, 13106, 13618, 13102, 13365, 13618, 12336, 12336, 12336,
8244, 29498, 24931, 11886, 27760, 28257, 28718, 24940, 24430, 28530, 29559, 12576, 12341, 12856, 8249, 29498, 24931, 11886, 27760, 28257, 28718, 24940, 24430, 26999, 29796, 8296,
14644, 14880, 25459, 28257, 28718, 24940, 11886, 24944, 24946, 27756, 27749, 24927, 24951, 25970, 29728, 30066, 8293, 29498, 24931, 11886, 27760, 28257, 28718, 29281, 27745, 25964,
24428, 24947, 25958, 29728, 30066, 8293, 29498, 24931, 11886, 27760, 28257, 24878, 31091, 25454, 25439, 28769, 25185, 25964, 26144, 27745, 25971, 14880, 25459, 28257, 28718, 24940,
11886, 27760, 28257, 28255, 25711, 24421, 25705, 12832, 14880, 25459, 28257, 28718, 24940, 11886, 24948, 26482, 29797, 26988, 29811, 10272, 21627, 21057, 17735, 17748, 21582, 22866,
14880, 30821, 29296, 31520, 16726, 8274, 30266, 29281, 28526, 12576, 14880, 24950, 24946, 29812, 28526, 12576, 14880, 24950, 29810, 28793, 8293, 12338, 14880, 24950, 29810, 28793,
28525, 8292, 12589, 14880, 24950, 25458, 27759, 26988, 8292, 8240, 30266, 29281, 30062, 27756, 28265, 29287, 27749, 8307, 25128, 8233, 30266, 29281, 25964, 25974, 29548, 28789,
12320...}}
(gdb) bt full
#0 0x000055e7d81c6cd0 in heapam_scan_bitmap_next_tuple (scan=0x55e7d98fd140, slot=0x55e7d98eb440, recheck=0x55e7d98eac10, lossy_pages=0x55e7d98eabf8, exact_pages=0x55e7d98eabf0)
at heapam_handler.c:2576
hscan = 0x55e7d98fd140
targoffset = <optimized out>
page = <optimized out>
lp = <optimized out>
__func__ = "heapam_scan_bitmap_next_tuple"
#1 0x000055e7d8336adb in table_scan_bitmap_next_tuple (exact_pages=0x55e7d98eabf0, lossy_pages=0x55e7d98eabf8, recheck=0x55e7d98eac10, slot=0x55e7d98eb440, scan=0x55e7d98fd140)
at ../../../src/include/access/tableam.h:2003
__func__ = "table_scan_bitmap_next_tuple"
__errno_location = <optimized out>
#2 BitmapHeapNext (node=0x55e7d98eab00) at nodeBitmapHeapscan.c:227
econtext = 0x55e7d98ead08
scan = 0x55e7d98fd140
tbm = <optimized out>
slot = 0x55e7d98eb440
dsa = <optimized out>
__func__ = "BitmapHeapNext"
#3 0x000055e7d8339331 in ExecProcNode (node=0x55e7d98eab00) at ../../../src/include/executor/executor.h:274
No locals.
#4 gather_getnext (gatherstate=0x55e7d98ea970) at nodeGather.c:287
estate = 0x55e7d98ea458
outerPlan = 0x55e7d98eab00
outerTupleSlot = <optimized out>
fslot = 0x55e7d98ebf10
tup = <optimized out>
outerPlan = <optimized out>
outerTupleSlot = <optimized out>
fslot = <optimized out>
tup = <optimized out>
estate = <optimized out>
#5 ExecGather (pstate=0x55e7d98ea970) at nodeGather.c:222
node = <optimized out>
slot = <optimized out>
econtext = 0x55e7d98eaa78
#6 0x000055e7d8343ac3 in ExecProcNode (node=0x55e7d98ea970) at ../../../src/include/executor/executor.h:274
No locals.
#7 ExecLimit (pstate=0x55e7d98ea698) at nodeLimit.c:95
node = 0x55e7d98ea698
econtext = 0x55e7d98ea8a0
direction = <optimized out>
slot = <optimized out>
outerPlan = 0x55e7d98ea970
__func__ = "ExecLimit"
#8 0x000055e7d832074a in ExecProcNode (node=0x55e7d98ea698) at ../../../src/include/executor/executor.h:274
No locals.
#9 ExecutePlan (execute_once=<optimized out>, dest=0x55e7d98ee380, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>,
planstate=0x55e7d98ea698, estate=0x55e7d98ea458) at execMain.c:1644
slot = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
current_tuple_count = 0
slot = <optimized out>
current_tuple_count = <optimized out>
#10 standard_ExecutorRun (queryDesc=0x55e7d9840f88, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
estate = 0x55e7d98ea458
operation = CMD_SELECT
dest = 0x55e7d98ee380
sendTuples = <optimized out>
oldcontext = 0x55e7d9840e90
__func__ = "standard_ExecutorRun"
#11 0x000055e7d84bbc7f in PortalRunSelect (portal=portal@entry=0x55e7d9893008, forward=forward@entry=true, count=0, count@entry=9223372036854775807, dest=dest@entry=0x55e7d98ee380)
at pquery.c:924
queryDesc = 0x55e7d9840f88
direction = <optimized out>
nprocessed = <optimized out>
__func__ = "PortalRunSelect"
#12 0x000055e7d84bcffa in PortalRun (portal=portal@entry=0x55e7d9893008, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true,
dest=dest@entry=0x55e7d98ee380, altdest=altdest@entry=0x55e7d98ee380, qc=0x7ffd7aceec20) at pquery.c:768
_save_exception_stack = 0x7ffd7aceee60
_save_context_stack = 0x0
_local_sigjmp_buf = {{__jmpbuf = {1, 6450011384570971824, 140726663834656, 94454275826560, 94454275452936, 94454274947640, 988062591367775920, 6450008691990202032},
__mask_was_saved = 0, __saved_mask = {__val = {94453442954861, 94450625806337, 0, 94450625806336, 94454256175041, 0, 94454275461144, 94454257765830, 94454275452936,
94454257765830, 1, 94454275825720, 0, 94454274947640, 11208266538747310592, 94454274947976}}}}
_do_rethrow = <optimized out>
result = <optimized out>
nprocessed = <optimized out>
saveTopTransactionResourceOwner = 0x55e7d984e6e0
saveTopTransactionContext = 0x55e7d9848e40
saveActivePortal = 0x0
saveResourceOwner = 0x55e7d984e6e0
savePortalContext = 0x0
saveMemoryContext = 0x55e7d9848e40
__func__ = "PortalRun"
#13 0x000055e7d84b957c in exec_simple_query (query_string=0x55e7d9816b18 "SELECT * FROM cyclic_fuzz WHERE (a BETWEEN 66 AND 82) OFFSET 1000000;") at postgres.c:1274
cmdtaglen = 6
snapshot_set = <optimized out>
per_parsetree_context = 0x0
plantree_list = <optimized out>
parsetree = 0x55e7d9817a10
commandTag = <optimized out>
qc = {commandTag = CMDTAG_UNKNOWN, nprocessed = 0}
querytree_list = <optimized out>
portal = 0x55e7d9893008
receiver = 0x55e7d98ee380
format = 0
cmdtagname = <optimized out>
parsetree_item__state = {l = 0x55e7d9817a38, i = 0}
dest = DestRemote
oldcontext = 0x55e7d9848e40
--Type <RET> for more, q to quit, c to continue without paging--
parsetree_list = 0x55e7d9817a38
parsetree_item = <optimized out>
save_log_statement_stats = false
was_logged = false
use_implicit_block = false
msec_str = "\000\000\000\000\000\000\000\000\0002\356u\bË\233\340\216\224\330\347U\000\000\000 \000\000\000\000\000"
__func__ = "exec_simple_query"
#14 0x000055e7d84bb051 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
query_string = 0x55e7d9816b18 "SELECT * FROM cyclic_fuzz WHERE (a BETWEEN 66 AND 82) OFFSET 1000000;"
firstchar = <optimized out>
input_message = {data = 0x55e7d9816b18 "SELECT * FROM cyclic_fuzz WHERE (a BETWEEN 66 AND 82) OFFSET 1000000;", len = 70, maxlen = 1024, cursor = 70}
local_sigjmp_buf = {{__jmpbuf = {140726663835040, 6450011346656560816, 94454254631152, 140726663835760, 140726663835760, 140726663835928, 988062591193712304, 6450008694811133616},
__mask_was_saved = 1, __saved_mask = {__val = {4194304, 0, 0, 0, 0, 0, 0, 0, 140311975264413, 94454275173984, 0, 0, 140311975264413, 94454275190384, 140311976438464, 1023}}}}
send_ready_for_query = false
idle_in_transaction_timeout_enabled = false
idle_session_timeout_enabled = false
__func__ = "PostgresMain"
#15 0x000055e7d84b5def in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
bsdata = <optimized out>
#16 0x000055e7d842b564 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffd7acef050 "", startup_data_len=startup_data_len@entry=4,
client_sock=client_sock@entry=0x7ffd7acef070) at launch_backend.c:265
pid = <optimized out>
#17 0x000055e7d842eea9 in BackendStartup (client_sock=0x7ffd7acef070) at postmaster.c:3593
bn = 0x55e7d9818940
pid = <optimized out>
startup_data = {canAcceptConnections = CAC_OK}
bn = <optimized out>
pid = <optimized out>
startup_data = <optimized out>
__func__ = "BackendStartup"
__errno_location = <optimized out>
__errno_location = <optimized out>
save_errno = <optimized out>
__errno_location = <optimized out>
__errno_location = <optimized out>
#18 ServerLoop () at postmaster.c:1674
s = {sock = 9, raddr = {addr = {ss_family = 1,
__ss_padding = "\000\000\347U\000\000\254gy\330\347U\000\0008709", '\000' <repeats 21 times>, "\004\000\000\000\000\000\000\272Ja\330\347U\000\000\360\360\316z\375\177\000\000`\251s\330\347U\000\000\360\360\316z\375\177\000\000\357\001_\330\347U", '\000' <repeats 18 times>, " j\201\331\347U\000\000ZJ\a\330\347U\000", __ss_align = 4269206038}, salen = 2}}
i = 0
now = <optimized out>
last_lockfile_recheck_time = 1711283252
last_touch_time = 1711283252
events = {{pos = 2, events = 2, fd = 7, user_data = 0x0}, {pos = -303599753, events = 32668, fd = 5, user_data = 0x7f9ced682d70}, {pos = 1, events = 0, fd = 0, user_data = 0x1}, {
pos = -303439168, events = 32668, fd = 0, user_data = 0x7f9cede9e2c0}, {pos = 1, events = 21991, fd = 2060382680, user_data = 0x7f9cede9e648}, {pos = 0, events = 0, fd = 0,
user_data = 0x55e7d9840910}, {pos = -1, events = 0, fd = 2060382680, user_data = 0x7ffd7acef1e7}, {pos = 0, events = 0, fd = 2060382692,
user_data = 0x7f9ced797d0e <inet_aton_end+110>}, {pos = 2060382780, events = 32765, fd = 2060382784, user_data = 0x55e7d9841d80}, {pos = -645658351, events = 21991, fd = 6,
user_data = 0x9b8bc30875ee3200}, {pos = 268, events = 0, fd = 2060383156, user_data = 0x7ffd7acef9d0}, {pos = -310803038, events = 32668, fd = 0,
user_data = 0x100000000000000}, {pos = -645648096, events = 21991, fd = 1978544640, user_data = 0x55e7d9818740}, {pos = -645643304, events = 21991, fd = 10,
--Type <RET> for more, q to quit, c to continue without paging--
user_data = 0x7f9ced7495dc <gaih_result_reset+28>}, {pos = 10, events = 0, fd = -311115396, user_data = 0x55e7d9811260}, {pos = 3, events = 0, fd = -664429546,
user_data = 0x7f9cede7f35a <_dl_runtime_resolve_xsave+138>}, {pos = 2060384808, events = 32765, fd = -645667952, user_data = 0x37}, {pos = 1, events = 0, fd = 2060383216,
user_data = 0x0}, {pos = 0, events = 0, fd = 2060383376, user_data = 0x7ffd7acef330}, {pos = -645643304, events = 21991, fd = 1, user_data = 0x7ffd7acefa40}, {pos = 10,
events = 32765, fd = 76, user_data = 0x2e2f706d742f}, {pos = 1364412240, events = 875900492, fd = -645653120, user_data = 0x0}, {pos = 0, events = 0, fd = -661294368,
user_data = 0x55e7d874e4b5}, {pos = -664536538, events = 21991, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x7f9cede6c540}, {pos = 2060383256,
events = 32765, fd = 1918989871, user_data = 0x65726774736f702f}, {pos = 796091500, events = 1882023792, fd = 1751348321, user_data = 0xff}, {pos = 0, events = 0, fd = 255,
user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0xa}, {pos = 0, events = 0, fd = 16777216, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {
pos = 0, events = 0, fd = 1, user_data = 0x100000000}, {pos = 2, events = 17, fd = -303603420, user_data = 0x7f9cede6c540}, {pos = -311915736, events = 32668, fd = 2090263927,
user_data = 0x7f9ced687e9c}, {pos = 2060383508, events = 32765, fd = -303602277, user_data = 0x8e3}, {pos = -311841864, events = 32668, fd = -303643328,
user_data = 0x7ffd7acef518}, {pos = 2060383508, events = 32765, fd = 0, user_data = 0x55e7d8074923}, {pos = 0, events = 0, fd = -670870984, user_data = 0x1f25b95}, {
pos = 2060383728, events = 32765, fd = 2060383720, user_data = 0x7f9ced682ce8}, {pos = 0, events = 0, fd = 10, user_data = 0x0}, {pos = 0, events = 0, fd = -303603420,
user_data = 0x7f9cede6c540}, {pos = -311915736, events = 32668, fd = 259272455, user_data = 0x7f9ced686728}, {pos = 2060383700, events = 32765, fd = -303602277,
user_data = 0x306}, {pos = -311841864, events = 32668, fd = -303643328, user_data = 0x0}, {pos = 2060383700, events = 32765, fd = -670611165, user_data = 0x7c96e577}, {
pos = -311939728, events = 32668, fd = -303438264, user_data = 0x7ffd7acef5e8}, {pos = 2060383872, events = 32765, fd = -303599753, user_data = 0x5}, {pos = -311939728,
events = 32668, fd = 1, user_data = 0x0}, {pos = 1, events = 0, fd = -303439168, user_data = 0x55e7d875284d}, {pos = -303439168, events = 32668, fd = 1,
user_data = 0x7ffd7acef5f0}, {pos = -303438264, events = 32668, fd = 0, user_data = 0x55e7d8659a89}, {pos = -311163099, events = 32668, fd = 0, user_data = 0x8}, {
pos = 2060383728, events = 32765, fd = -1, user_data = 0x7f9ced696070}, {pos = -303643328, events = 32668, fd = 2, user_data = 0x46000041c0}, {pos = 70, events = 0, fd = 0,
user_data = 0x1000}, {pos = -303439168, events = 32668, fd = 245, user_data = 0x55e7d80f3750}, {pos = -661461080, events = 21991, fd = 3, user_data = 0x55e7d875284d}, {
pos = -303574238, events = 32668, fd = 1, user_data = 0x0}, {pos = -663359120, events = 21991, fd = -663359139, user_data = 0x7f9ced696070}, {pos = -311128592, events = 32668,
fd = -661290912, user_data = 0x7ffd7acefa40}, {pos = 2060384864, events = 32765, fd = 0, user_data = 0x0}, {pos = -303565990, events = 32668, fd = 0,
user_data = 0x7f9ced70fd84 <__GI___pthread_sigmask+68>}, {pos = 0, events = 0, fd = -661344352, user_data = 0x2}, {pos = 739479668, events = 0, fd = 2060746880,
user_data = 0x55e7d80f3330}}
nevents = <optimized out>
__func__ = "ServerLoop"
#19 0x000055e7d8430b50 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x55e7d9811260) at postmaster.c:1372
opt = <optimized out>
status = <optimized out>
userDoption = <optimized out>
listen_addr_saved = <optimized out>
output_config_variable = <optimized out>
__func__ = "PostmasterMain"
#20 0x000055e7d817a0c3 in main (argc=3, argv=0x55e7d9811260) at main.c:197
do_check_root = <optimized out>
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-24 17:38 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-24 17:38 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sun, Mar 24, 2024 at 01:36:19PM +0100, Tomas Vondra wrote:
>
>
> On 3/23/24 01:26, Melanie Plageman wrote:
> > On Fri, Mar 22, 2024 at 08:22:11PM -0400, Melanie Plageman wrote:
> >> On Tue, Mar 19, 2024 at 02:33:35PM +0200, Heikki Linnakangas wrote:
> >>> On 18/03/2024 17:19, Melanie Plageman wrote:
> >>>> I've attached v7 rebased over this commit.
> >>>
> >>> If we delayed table_beginscan_bm() call further, after starting the TBM
> >>> iterator, we could skip it altogether when the iterator is empty.
> >>>
> >>> That's a further improvement, doesn't need to be part of this patch set.
> >>> Just caught my eye while reading this.
> >>
> >> Hmm. You mean like until after the first call to tbm_[shared]_iterate()?
> >> AFAICT, tbm_begin_iterate() doesn't tell us anything about whether or
> >> not the iterator is "empty". Do you mean cases when the bitmap has no
> >> blocks in it? It seems like we should be able to tell that from the
> >> TIDBitmap.
> >>
> >>>
> >>>> v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch
> >>>
> >>> I suggest to avoid the double negative with SO_CAN_SKIP_FETCH, and call the
> >>> flag e.g. SO_NEED_TUPLE.
> >>
> >> Agreed. Done in attached v8. Though I wondered if it was a bit weird
> >> that the flag is set in the common case and not set in the uncommon
> >> case...
> >
> > v8 actually attached this time
>
> I tried to run the benchmarks with v8, but unfortunately it crashes for
> me very quickly (I've only seen 0015 to crash, so I guess the bug is in
> that patch).
>
> The backtrace attached, this doesn't seem right:
>
> (gdb) p hscan->rs_cindex
> $1 = 543516018
Thanks for reporting this! I hadn't seen it crash on my machine, so I
didn't realize that I was no longer initializing rs_cindex and
rs_ntuples on the first call to heapam_bitmap_next_tuple() (since
heapam_bitmap_next_block() wasn't being called first). I've done this in
attached v9.
I haven't had a chance yet to reproduce the regressions you saw in the
streaming read user patch or to look closely at the performance results.
I don't anticipate the streaming read user will have any performance
differences in this v9 from v6, since I haven't yet rebased in Thomas'
latest streaming read API changes nor addressed any other potential
regression sources.
I tried rebasing in Thomas' latest version today and something is
causing a crash that I have yet to figure out. v10 of this patchset will
have his latest version once I get that fixed. I wanted to share this
version with what I think is a bug fix for the crash you saw first.
- Melanie
Attachments:
[text/x-diff] v9-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/2-v9-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 5d45746e8602559f60fea6e32b108dc23a9f41d7 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v9 01/17] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index e57a0b7ea3..e78d793f69 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..93fdcd226b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v9-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/3-v9-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 321b320c40e6e5d7ef81cd62710e0d06e80ce9fc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v9 02/17] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226b..c64530674b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v9-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch (15.0K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/4-v9-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch)
download | inline diff:
From c110baa11fbb6a5ee2639a34169e557a36498f05 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v9 03/17] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 34bc60f625..e7bed84f75 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -951,6 +951,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1039,6 +1041,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1058,6 +1066,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2b7c702642..7fdccaf613 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2236,6 +2255,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674b..83d9db8f39 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4b133f6859..3dfb19ec7d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 65834caeb1..1bc5f7c057 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -952,9 +959,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1774c56ae3..6871db9b21 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,10 +1785,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1808,9 +1805,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v9-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch (2.2K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/5-v9-0004-BitmapPrefetch-use-prefetch-block-recheck-for-ski.patch)
download | inline diff:
From c07c20bce287f6266f58cc2179b1f9dc16da8850 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v9 04/17] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39..5df3b5ca46 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v9-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch (2.3K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/6-v9-0005-Update-BitmapAdjustPrefetchIterator-parameter-typ.patch)
download | inline diff:
From 8bc7c42a7d6264de7f9857f6541ab12b0f3ddbe0 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v9 05/17] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46..404de0595e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v9-0006-table_scan_bitmap_next_block-returns-lossy-or-exa.patch (4.4K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/7-v9-0006-table_scan_bitmap_next_block-returns-lossy-or-exa.patch)
download | inline diff:
From 17ee3d0b26b631c0e2edec8242219adf5532280c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v9 06/17] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 7fdccaf613..849cac3947 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2242,6 +2243,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595e..c95e3412da 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1bc5f7c057..b9ba4f9fb3 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -804,6 +804,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -819,7 +822,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1988,14 +1992,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2006,7 +2012,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v9-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch (2.9K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/8-v9-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local-.patch)
download | inline diff:
From 8bfd073e2f59ce4f5979fd433290e51e54597d85 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v9 07/17] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da..49938c9ed4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v9-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch (4.1K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/9-v9-0008-Remove-table_scan_bitmap_next_tuple-parameter-tbm.patch)
download | inline diff:
From 8d60908ce61e967376b961081b973df9baeae5ab Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v9 08/17] Remove table_scan_bitmap_next_tuple parameter tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 849cac3947..cf4387f443 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2250,7 +2250,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed4..282dcb9791 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b9ba4f9fb3..bcf1497f67 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -795,10 +795,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -829,15 +826,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2025,7 +2017,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2037,7 +2028,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v9-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/10-v9-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 0febaee773a45cf19065a23d19023196bf7b4628 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v9 09/17] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index cf4387f443..2ad785e511 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,18 +2114,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2144,16 +2177,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2245,7 +2269,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb9791..7e73583fe5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno > node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index bcf1497f67..a820cc8c99 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -788,19 +789,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -819,8 +815,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -957,9 +953,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1019,6 +1019,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1029,6 +1044,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1981,19 +2011,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2003,8 +2031,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6871db9b21..8688bc5ab0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,8 +1783,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1793,9 +1791,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,8 +1803,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1813,9 +1811,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v9-0010-table_scan_bitmap_next_block-counts-lossy-and-exa.patch (5.2K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/11-v9-0010-table_scan_bitmap_next_block-counts-lossy-and-exa.patch)
download | inline diff:
From cf97448fb2c952354d85f43ef6a6ef1ffb654166 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v9 10/17] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2ad785e511..266b34fe6b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2267,7 +2268,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7e73583fe5..96b55507a3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -69,7 +69,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -267,14 +266,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a820cc8c99..1d4b79a73f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -797,8 +797,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -815,8 +815,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2013,15 +2015,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2032,7 +2036,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v9-0011-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/12-v9-0011-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From 095b063651c492b7040440e448dc375e36a31fa1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v9 11/17] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fc..1dc4c99bf9 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639b..432fae5296 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v9-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch (19.7K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/13-v9-0012-Separate-TBM-Shared-Iterator-and-TBMIterateResult.patch)
download | inline diff:
From afd06b79a7a0260bf6b3194a5c608be0f843a037 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v9 12/17] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 32 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 38 ++++++------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
7 files changed, 108 insertions(+), 91 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb..3aa457a29e 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544..033d525339 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 266b34fe6b..adfc77684a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2122,7 +2122,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2135,11 +2135,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
CHECK_FOR_INTERRUPTS();
if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbm_iterate(scan->tbmiterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2154,11 +2154,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2166,19 +2166,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2207,7 +2207,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2216,9 +2216,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2268,7 +2268,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 96b55507a3..51c4360205 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -316,7 +316,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -330,8 +330,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -366,8 +366,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -435,10 +435,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_iterate(prefetch_iterator);
@@ -446,7 +448,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -455,13 +457,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -476,7 +478,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -495,8 +497,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
tbm_end_shared_iterate(prefetch_iterator);
@@ -504,17 +506,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf9..309a44bdb8 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae..3b432263bb 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae5296..f000c1af28 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v9-0013-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (33.7K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/14-v9-0013-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From 5152ee50a21ea8e39a0f33efc0c83519f418ed8d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v9 13/17] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 268 +++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 397 +++-------------------
src/include/access/heapam.h | 12 +
src/include/access/relscan.h | 11 +
src/include/access/tableam.h | 38 ++-
src/include/nodes/execnodes.h | 16 -
7 files changed, 388 insertions(+), 380 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e7bed84f75..c12563a188 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -951,8 +951,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1035,6 +1043,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1047,6 +1061,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1072,6 +1092,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index adfc77684a..efd2784e03 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,6 +55,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2112,6 +2115,76 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
+
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ tbm_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
+
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2130,6 +2203,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2273,6 +2348,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->pf_tbmiterator &&
+ hscan->pfblockno > hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2283,6 +2370,157 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+
+ if (pstate == NULL)
+ {
+ TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
+
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ tbm_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ tbm_end_iterate(prefetch_iterator);
+ scan->rs_base.pf_tbmiterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
+
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ tbm_end_shared_iterate(prefetch_iterator);
+ scan->rs_base.pf_shared_tbmiterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2308,6 +2546,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 51c4360205..f241f4cb2c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
@@ -71,7 +67,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -91,83 +86,53 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
+
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
- if (!pstate)
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (init_shared_state)
{
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
- {
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
- }
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
+ BitmapDoneInitializingSharedState(node->pstate);
}
-#endif /* USE_PREFETCH */
}
/*
@@ -197,8 +162,26 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
+ if (!scan->bm_parallel)
+ scan->tbmiterator = tbm_begin_iterate(tbm);
+ else
+ /* Allocate a private iterator and attach the shared state to it */
+ scan->shared_tbmiterator = tbm_attach_shared_iterate(dsa, scan->bm_parallel->tbmiterator);
+
+#ifdef USE_PREFETCH
+ if (scan->prefetch_maximum > 0)
+ {
+ if (!scan->bm_parallel)
+ scan->pf_tbmiterator = tbm_begin_iterate(tbm);
+ else
+ scan->pf_shared_tbmiterator =
+ tbm_attach_shared_iterate(dsa, scan->bm_parallel->prefetch_iterator);
+ }
+#endif /* USE_PREFETCH */
+
node->initialized = true;
@@ -211,36 +194,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -264,23 +217,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->prefetch_iterator &&
- node->pfblockno > node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -304,224 +243,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- tbm_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- tbm_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
@@ -569,22 +291,11 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -625,14 +336,8 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -665,18 +370,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -716,13 +414,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -806,7 +497,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 3dfb19ec7d..22bdccc2a9 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -81,6 +81,18 @@ typedef struct HeapScanDescData
*/
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..93168bd350 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@ struct ParallelTableScanDescData;
struct TBMIterator;
struct TBMSharedIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -46,6 +47,16 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct TBMIterator *tbmiterator;
struct TBMSharedIterator *shared_tbmiterator;
+ /* Prefetch iterators */
+ struct TBMIterator *pf_tbmiterator;
+ struct TBMSharedIterator *pf_shared_tbmiterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+
+ /* used to validate prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1d4b79a73f..9cab4462d6 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -800,17 +800,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -961,6 +950,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->shared_tbmiterator = NULL;
result->tbmiterator = NULL;
+ result->pf_shared_tbmiterator = NULL;
+ result->pf_tbmiterator = NULL;
+ result->bm_parallel = NULL;
return result;
}
@@ -1029,11 +1021,23 @@ table_endscan(TableScanDesc scan)
scan->shared_tbmiterator = NULL;
}
+ if (scan->pf_shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
+ scan->pf_shared_tbmiterator = NULL;
+ }
+
if (scan->tbmiterator)
{
tbm_end_iterate(scan->tbmiterator);
scan->tbmiterator = NULL;
}
+
+ if (scan->pf_tbmiterator)
+ {
+ tbm_end_iterate(scan->pf_tbmiterator);
+ scan->pf_tbmiterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1054,11 +1058,23 @@ table_rescan(TableScanDesc scan,
scan->shared_tbmiterator = NULL;
}
+ if (scan->pf_shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
+ scan->pf_shared_tbmiterator = NULL;
+ }
+
if (scan->tbmiterator)
{
tbm_end_iterate(scan->tbmiterator);
scan->tbmiterator = NULL;
}
+
+ if (scan->pf_tbmiterator)
+ {
+ tbm_end_iterate(scan->pf_tbmiterator);
+ scan->pf_tbmiterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8688bc5ab0..7a3fdf9cd4 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,19 +1783,11 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,19 +1795,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v9-0014-Unify-parallel-and-serial-BitmapHeapScan-iterator.patch (11.3K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/15-v9-0014-Unify-parallel-and-serial-BitmapHeapScan-iterator.patch)
download | inline diff:
From 9972e4838b67ea63550505577d1447925d7f97e1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 13:43:41 -0400
Subject: [PATCH v9 14/17] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to
both TBMIterator and TBMSharedIterators. This encapsulates the parallel
and serial iterators and their access and makes the bitmap heap scan
code a bit cleaner.
---
src/backend/access/heap/heapam_handler.c | 40 ++++++---------
src/backend/executor/nodeBitmapHeapscan.c | 62 ++++++++++++++++++-----
src/include/access/relscan.h | 20 +++++---
src/include/access/tableam.h | 61 ++++++----------------
src/tools/pgindent/typedefs.list | 1 +
5 files changed, 96 insertions(+), 88 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index efd2784e03..1a9f7b02d1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,6 +55,7 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+
static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
static inline void BitmapPrefetch(HeapScanDesc scan);
@@ -2131,17 +2132,15 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
-
if (scan->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
scan->prefetch_pages--;
}
- else if (prefetch_iterator)
+ else if (scan->rs_base.rs_pf_tbmiterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbm_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
scan->pfblockno = tbmpre.blockno;
}
return;
@@ -2154,8 +2153,6 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
*/
if (scan->rs_base.prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -2175,9 +2172,9 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
* we don't validate the blockno here as we do in non-parallel
* case.
*/
- if (prefetch_iterator)
+ if (scan->rs_base.rs_pf_tbmiterator)
{
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
scan->pfblockno = tbmpre.blockno;
}
}
@@ -2209,10 +2206,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbm_shared_iterate(scan->shared_tbmiterator, &tbmres);
- else
- tbm_iterate(scan->tbmiterator, &tbmres);
+ bhs_iterate(scan->rs_tbmiterator, &tbmres);
if (!BlockNumberIsValid(tbmres.blockno))
{
@@ -2353,7 +2347,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* ahead of the current block.
*/
if (scan->bm_parallel == NULL &&
- scan->pf_tbmiterator &&
+ scan->rs_pf_tbmiterator &&
hscan->pfblockno > hscan->rs_base.blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -2427,22 +2421,20 @@ BitmapPrefetch(HeapScanDesc scan)
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = scan->rs_base.pf_tbmiterator;
-
- if (prefetch_iterator)
+ if (scan->rs_base.rs_pf_tbmiterator)
{
while (scan->prefetch_pages < scan->prefetch_target)
{
TBMIterateResult tbmpre;
bool skip_fetch;
- tbm_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- scan->rs_base.pf_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
+ scan->rs_base.rs_pf_tbmiterator = NULL;
break;
}
scan->prefetch_pages++;
@@ -2470,9 +2462,7 @@ BitmapPrefetch(HeapScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = scan->rs_base.pf_shared_tbmiterator;
-
- if (prefetch_iterator)
+ if (scan->rs_base.rs_pf_tbmiterator)
{
while (1)
{
@@ -2495,12 +2485,12 @@ BitmapPrefetch(HeapScanDesc scan)
if (!do_prefetch)
return;
- tbm_shared_iterate(prefetch_iterator, &tbmpre);
+ bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- scan->rs_base.pf_shared_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
+ scan->rs_base.rs_pf_tbmiterator = NULL;
break;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f241f4cb2c..b548642088 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,6 +52,54 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, ParallelBitmapHeapState *pstate, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ if (pstate)
+ result->parallel = tbm_attach_shared_iterate(personal_area, pstate->tbmiterator);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *tbmres)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ tbm_iterate(iterator->serial, tbmres);
+ else
+ tbm_shared_iterate(iterator->parallel, tbmres);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -165,21 +213,11 @@ BitmapHeapNext(BitmapHeapScanState *node)
scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
- if (!scan->bm_parallel)
- scan->tbmiterator = tbm_begin_iterate(tbm);
- else
- /* Allocate a private iterator and attach the shared state to it */
- scan->shared_tbmiterator = tbm_attach_shared_iterate(dsa, scan->bm_parallel->tbmiterator);
+ scan->rs_tbmiterator = bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
#ifdef USE_PREFETCH
if (scan->prefetch_maximum > 0)
- {
- if (!scan->bm_parallel)
- scan->pf_tbmiterator = tbm_begin_iterate(tbm);
- else
- scan->pf_shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, scan->bm_parallel->prefetch_iterator);
- }
+ bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
#endif /* USE_PREFETCH */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 93168bd350..65092d7226 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -27,6 +27,17 @@ struct ParallelTableScanDescData;
struct TBMIterator;
struct TBMSharedIterator;
struct ParallelBitmapHeapState;
+struct TBMIterateResult;
+
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern void bhs_iterate(BitmapHeapIterator *tbmiterator, struct TBMIterateResult *tbmres);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,12 +56,9 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
- /* Prefetch iterators */
- struct TBMIterator *pf_tbmiterator;
- struct TBMSharedIterator *pf_shared_tbmiterator;
-
+ BitmapHeapIterator *rs_tbmiterator;
+ /* Prefetch iterator */
+ BitmapHeapIterator *rs_pf_tbmiterator;
/* maximum value for prefetch_target */
int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 9cab4462d6..2ded1a124b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -948,10 +948,8 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
- result->pf_shared_tbmiterator = NULL;
- result->pf_tbmiterator = NULL;
+ result->rs_tbmiterator = NULL;
+ result->rs_pf_tbmiterator = NULL;
result->bm_parallel = NULL;
return result;
}
@@ -1015,29 +1013,15 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
+ bhs_end_iterate(scan->rs_tbmiterator);
+ scan->rs_tbmiterator = NULL;
+#ifdef USE_PREFETCH
+ if (scan->rs_pf_tbmiterator)
{
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->pf_shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
- scan->pf_shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
-
- if (scan->pf_tbmiterator)
- {
- tbm_end_iterate(scan->pf_tbmiterator);
- scan->pf_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_pf_tbmiterator);
+ scan->rs_pf_tbmiterator = NULL;
}
+#endif
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1052,29 +1036,16 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->pf_shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->pf_shared_tbmiterator);
- scan->pf_shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_tbmiterator);
+ scan->rs_tbmiterator = NULL;
- if (scan->pf_tbmiterator)
+#ifdef USE_PREFETCH
+ if (scan->rs_pf_tbmiterator)
{
- tbm_end_iterate(scan->pf_tbmiterator);
- scan->pf_tbmiterator = NULL;
+ bhs_end_iterate(scan->rs_pf_tbmiterator);
+ scan->rs_pf_tbmiterator = NULL;
}
+#endif
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e2a0525dd4..58317eac44 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v9-0015-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/16-v9-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 4f69998bd52da1e6dc4e9e019072db839e24c550 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v9 15/17] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index c12563a188..9ddbaff4e2 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -322,6 +322,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 1a9f7b02d1..9dfc99d8fa 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2110,12 +2110,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2148,8 +2142,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2511,30 +2505,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -2916,7 +2923,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index ce637a5a5d..1d6b03d1ca 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -92,8 +92,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index b548642088..2683d8bc0c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -222,44 +222,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2ded1a124b..5ad3eff539 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -788,36 +788,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -2000,44 +1984,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2048,7 +2001,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v9-0016-v7-Streaming-Read-API.patch (56.1K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/17-v9-0016-v7-Streaming-Read-API.patch)
download | inline diff:
From 8eebac7992268b45eed0042abec10cd86795a540 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:59:42 -0400
Subject: [PATCH v9 16/17] v7 Streaming Read API
---
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 659 +++++++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 642 +++++++++++++++-------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/include/storage/bufmgr.h | 45 ++
src/include/storage/streaming_read.h | 52 ++
src/tools/pgindent/typedefs.list | 3 +
10 files changed, 1227 insertions(+), 210 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca2..eec03f6f2b 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 0000000000..bcab44c802
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 0000000000..39aef2a84a
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 0000000000..d5c29b750d
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,659 @@
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ */
+typedef struct PgStreamingReadRange
+{
+ bool need_wait;
+ bool advice_issued;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index;
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+ ReadBuffersOperation operation;
+} PgStreamingReadRange;
+
+/*
+ * Streaming read object.
+ */
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int next_tail_buffer;
+ int distance;
+ bool started;
+ bool finished;
+ bool advice_enabled;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ /* Sometimes we need to buffer one block for flow control. */
+ BlockNumber unget_blocknum;
+ void *unget_per_buffer_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /*
+ * Make sure our bmr's smgr and persistent are populated. The caller
+ * asserts that the storage manager will remain valid.
+ */
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+
+ /*
+ * Choose a maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least
+ * MAX_BUFFER_PER_TRANSFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+ pgsr->callback = next_block_cb;
+ pgsr->bmr = bmr;
+ pgsr->forknum = forknum;
+
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out doing full-sized
+ * reads.
+ */
+ if (flags & PGSR_FLAG_FULL)
+ pgsr->distance = Min(MAX_BUFFERS_PER_TRANSFER, pgsr->max_pinned_buffers);
+ else
+ pgsr->distance = 1;
+
+ /*
+ * Space for the callback to store extra data along with each block. Note
+ * that we need one more than max_pinned_buffers, so we can return a
+ * pointer to a slot that can't be overwritten until the next call.
+ */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * size);
+
+ return pgsr;
+}
+
+/*
+ * Find the per-buffer data index for the Nth block of a range.
+ */
+static int
+get_per_buffer_data_index(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ int result;
+
+ /*
+ * Find slot in the circular buffer of per-buffer data, without using the
+ * expensive % operator.
+ */
+ result = range->per_buffer_data_index + n;
+ while (result >= pgsr->size)
+ result -= pgsr->size;
+ Assert(result == (range->per_buffer_data_index + n) % pgsr->size);
+
+ return result;
+}
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static void *
+get_per_buffer_data_by_index(PgStreamingRead *pgsr, int per_buffer_data_index)
+{
+ return (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * per_buffer_data_index;
+}
+
+/*
+ * Return a pointer to the per-buffer data for the Nth block of a range.
+ */
+static void *
+get_per_buffer_data(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ return get_per_buffer_data_by_index(pgsr,
+ get_per_buffer_data_index(pgsr,
+ range,
+ n));
+}
+
+/*
+ * Start reading the head range, and create a new head range. The new head
+ * range is returned. It may not be empty, if StartReadBuffers() couldn't
+ * start the entire range; in that case the returned range contains the
+ * remaining portion of the range.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_start_head_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+ PgStreamingReadRange *new_head_range;
+ int nblocks_pinned;
+ int flags;
+
+ /* Caller should make sure we never exceed max_ios. */
+ Assert((pgsr->ios_in_progress < pgsr->max_ios) ||
+ (pgsr->ios_in_progress == 0 && pgsr->max_ios == 0));
+
+ /* Should only call if the head range has some blocks to read. */
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If advice hasn't been suppressed, and this system supports it, this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (pgsr->advice_enabled &&
+ pgsr->max_ios > 0 &&
+ pgsr->started &&
+ head_range->blocknum != pgsr->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* Suppress advice on the first call, because it's too late to benefit. */
+ if (!pgsr->started)
+ pgsr->started = true;
+
+ /* We shouldn't be trying to pin more buffers that we're allowed to. */
+ Assert(pgsr->pinned_buffers + head_range->nblocks <= pgsr->max_pinned_buffers);
+
+ /* Start reading as many blocks as we can from the head range. */
+ nblocks_pinned = head_range->nblocks;
+ head_range->need_wait =
+ StartReadBuffers(pgsr->bmr,
+ head_range->buffers,
+ pgsr->forknum,
+ head_range->blocknum,
+ &nblocks_pinned,
+ pgsr->strategy,
+ flags,
+ &head_range->operation);
+
+ Assert(pgsr->pinned_buffers <= pgsr->max_pinned_buffers);
+
+ if (head_range->need_wait && (flags & READ_BUFFERS_ISSUE_ADVICE))
+ {
+ /*
+ * Since we've issued advice, we count an I/O in progress until we
+ * call WaitReadBuffers().
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ Assert(pgsr->ios_in_progress <= pgsr->max_ios);
+ }
+
+ /*
+ * StartReadBuffers() might have pinned fewer blocks than we asked it to,
+ * but always at least one.
+ */
+ Assert(nblocks_pinned <= head_range->nblocks);
+ Assert(nblocks_pinned >= 1);
+ pgsr->pinned_buffers += nblocks_pinned;
+
+ /*
+ * Remember where the next block would be after that, so we can detect
+ * sequential access next time.
+ */
+ pgsr->seq_blocknum = head_range->blocknum + nblocks_pinned;
+
+ /*
+ * Create a new head range. There must be space, because we have enough
+ * elements for every range to hold just one block, up to the pin limit.
+ */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ new_head_range = &pgsr->ranges[pgsr->head];
+ new_head_range->nblocks = 0;
+ new_head_range->advice_issued = false;
+
+ /*
+ * If we didn't manage to start the whole read above, we split the range,
+ * moving the remainder into the new head range.
+ */
+ if (nblocks_pinned < head_range->nblocks)
+ {
+ int nblocks_remaining = head_range->nblocks - nblocks_pinned;
+
+ head_range->nblocks = nblocks_pinned;
+
+ new_head_range->blocknum = head_range->blocknum + nblocks_pinned;
+ new_head_range->nblocks = nblocks_remaining;
+ }
+
+ /* The new range has per-buffer data starting after the previous range. */
+ new_head_range->per_buffer_data_index =
+ get_per_buffer_data_index(pgsr, head_range, nblocks_pinned);
+
+ return new_head_range;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow pg_streaming_unget_block() to work.
+ */
+static BlockNumber
+pg_streaming_get_block(PgStreamingRead *pgsr, void *per_buffer_data)
+{
+ BlockNumber result;
+
+ if (unlikely(pgsr->unget_blocknum != InvalidBlockNumber))
+ {
+ /*
+ * If we had to unget a block, now it is time to return that one
+ * again.
+ */
+ result = pgsr->unget_blocknum;
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+ /*
+ * The same per_buffer_data element must have been used, and still
+ * contains whatever data the callback wrote into it. So we just
+ * sanity-check that we were called with the value that
+ * pg_streaming_unget_block() pushed back.
+ */
+ Assert(per_buffer_data == pgsr->unget_per_buffer_data);
+ }
+ else
+ {
+ /* Use the installed callback directly. */
+ result = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ }
+
+ return result;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later. This *must* be called with the
+ * last value returned by pg_streaming_get_block().
+ */
+static void
+pg_streaming_unget_block(PgStreamingRead *pgsr, BlockNumber blocknum, void *per_buffer_data)
+{
+ Assert(pgsr->unget_blocknum == InvalidBlockNumber);
+ pgsr->unget_blocknum = blocknum;
+ pgsr->unget_per_buffer_data = per_buffer_data;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *range;
+
+ /* If we're finished, don't look ahead. */
+ if (pgsr->finished)
+ return;
+
+ /*
+ * We we've already started the maximum allowed number of I/Os, don't look
+ * ahead. There is a special case for max_ios == 0.
+ */
+ if (pgsr->max_ios > 0 && pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /* Can't pin any more buffers. */
+ if (pgsr->pinned_buffers == pgsr->distance)
+ return;
+
+ /*
+ * Keep trying to add new blocks to the end of the head range while doing
+ * so wouldn't exceed the distance limit.
+ */
+ range = &pgsr->ranges[pgsr->head];
+ while (pgsr->pinned_buffers + range->nblocks < pgsr->distance)
+ {
+ BlockNumber blocknum;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ if (range->nblocks == lengthof(range->buffers))
+ {
+ /* Start as much of it as we can. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /* If we're now at the I/O limit, stop here. */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * That might have only been partially started, but always
+ * processes at least one so that'll do for now.
+ */
+ Assert(range->nblocks < lengthof(range->buffers));
+ }
+
+ /* Find per-buffer data slot for the next block. */
+ per_buffer_data = get_per_buffer_data(pgsr, range, range->nblocks);
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pg_streaming_get_block(pgsr, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ pgsr->finished = true;
+ break;
+ }
+
+ /*
+ * Is there a head range that we cannot extend, because the requested
+ * block is not consecutive?
+ */
+ if (range->nblocks > 0 &&
+ range->blocknum + range->nblocks != blocknum)
+ {
+ /* Yes. Start it, so we can begin building a new one. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * It's possible that it was only partially started, and we have a
+ * new range with the remainder. Keep starting I/Os until we get
+ * it all out of the way, or we hit the I/O limit.
+ */
+ while (range->nblocks > 0 && pgsr->ios_in_progress < pgsr->max_ios)
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * We do have to worry about I/O capacity running out if the head
+ * range was split. In that case we have to 'unget' the block
+ * returned by the callback.
+ */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ {
+ pg_streaming_unget_block(pgsr, blocknum, per_buffer_data);
+ return;
+ }
+ }
+
+ /* If we have a new, empty range, initialize the start block. */
+ if (range->nblocks == 0)
+ range->blocknum = blocknum;
+
+ /* This block extends the range by one. */
+ Assert(range->blocknum + range->nblocks == blocknum);
+ range->nblocks++;
+ };
+
+ /*
+ * Normally we don't start the head range, preferring to give it a chance
+ * to grow to full size once more buffers have been consumed. In cases
+ * where that can't possibly happen, we might as well start the read
+ * immediately.
+ */
+ if ((range->nblocks > 0 && pgsr->finished) ||
+ (range->nblocks == pgsr->distance))
+ pg_streaming_read_start_head_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ PgStreamingReadRange *tail_range;
+
+ for (;;)
+ {
+ if (pgsr->tail != pgsr->head)
+ {
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to wait for a ReadBuffers operation to finish before
+ * returning the buffers in this range?
+ */
+ if (tail_range->need_wait)
+ {
+ int distance;
+
+ Assert(pgsr->next_tail_buffer == 0);
+ WaitReadBuffers(&tail_range->operation);
+ tail_range->need_wait = false;
+
+ /*
+ * We don't really know if the kernel generated a physical I/O
+ * when we issued advice, let alone when it finished, but it
+ * has certainly finished now because we've performed the
+ * read.
+ */
+ if (tail_range->advice_issued)
+ {
+
+ Assert(pgsr->ios_in_progress > 0);
+ pgsr->ios_in_progress--;
+
+ /*
+ * Look-ahead distance ramps up rapidly if we're issuing
+ * advice, so we can search for new more I/Os to start.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ else
+ {
+ /*
+ * There is no point in increasing look-ahead distance if
+ * we've already reached the full I/O size, since we're
+ * not issuing advice. Extra distance would only pin more
+ * buffers for no benefit.
+ */
+ if (pgsr->distance > MAX_BUFFERS_PER_TRANSFER)
+ {
+ /*
+ * Look-ahead distance gradually decays to full I/O
+ * size.
+ */
+ pgsr->distance--;
+ }
+ else
+ {
+ /*
+ * Look-ahead distance ramps up rapidly, but not more
+ * that the full I/O size.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, MAX_BUFFERS_PER_TRANSFER);
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ }
+ }
+ else if (pgsr->next_tail_buffer == 0)
+ {
+ /* No I/O necessary. Look-ahead distance gradually decays. */
+ if (pgsr->distance > 1)
+ pgsr->distance--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(pgsr, tail_range, buffer_index);
+
+ /* We may be able to get another I/O started. */
+ pg_streaming_read_look_ahead(pgsr);
+
+ return buffer;
+ }
+
+ /* Advance tail to next range. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+ else
+ {
+ /*
+ * If tail crashed into head, and head is not empty, then it is
+ * time to start that range. Otherwise, force a look-ahead, to
+ * kick start the stream.
+ */
+ Assert(pgsr->tail == pgsr->head);
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ {
+ pg_streaming_read_start_head_range(pgsr);
+ }
+ else
+ {
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* Finished? */
+ if (pgsr->tail == pgsr->head &&
+ pgsr->ranges[pgsr->head].nblocks == 0)
+ break;
+ }
+ }
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ pgsr->finished = true;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(pgsr->pinned_buffers == 0);
+ Assert(pgsr->ios_in_progress == 0);
+
+ /* Release memory. */
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c..d0e9c7deff 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -471,10 +476,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +504,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +785,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +797,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +819,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +990,68 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
+/*
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
+ */
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* ReadBuffer_common -- common logic for all ReadBuffer variants
*
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+ BufferAccessStrategy strategy)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int nblocks;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1070,405 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ nblocks = 1;
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
+ else
+ flags = 0;
+ if (StartReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ &nblocks,
+ strategy,
+ flags,
+ &operation))
+ WaitReadBuffers(&operation);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ ZeroBuffer(buffer, mode);
+
+ return buffer;
+}
+static Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested.
+ *
+ * If false is returned, no I/O is necessary and WaitReadBuffers() is not
+ * necessary. If true is returned, one I/O has been started, and
+ * WaitReadBuffers() must be called with the same operation object before the
+ * buffers are accessed. Along with the operation object, the caller-supplied
+ * array of buffers must remain valid until WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+bool
+StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation)
+{
+ int actual_nblocks = *nblocks;
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ operation->bmr = bmr;
+ operation->forknum = forkNum;
+ operation->blocknum = blockNum;
+ operation->buffers = buffers;
+ operation->nblocks = actual_nblocks;
+ operation->strategy = strategy;
+ operation->flags = flags;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ operation->io_buffers_len = 0;
- /*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
- */
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
- else
+ for (int i = 0; i < actual_nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ bool found;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ buffers[i] = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum + i,
+ strategy,
+ &found);
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = operation->nblocks = *nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ operation->io_buffers_len++;
+ }
+ }
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (operation->io_buffers_len > 0)
+ {
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
- {
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
+ /*
+ * In theory we should only do this if PrepareReadBuffers() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(bmr.smgr, forkNum, blockNum, operation->io_buffers_len);
}
+
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
}
+ else
+ {
+ return false;
+ }
+}
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
{
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ /*
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
+ */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+ bmr = operation->bmr;
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
-
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
}
else
{
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
}
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ /*
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
+ */
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
+ else
+ pgBufferUsage.shared_blks_read += nblocks;
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ for (int i = 0; i < nblocks; ++i)
+ {
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- return BufferDescriptorGetBuffer(bufHdr);
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
+
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
+ {
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
+ {
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
+ }
+ else
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
+
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
+
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
+
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
+
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for StartReadBuffers. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1476,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1535,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1603,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1638,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +1994,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2259,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2282,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2372,7 +2597,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2611,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3679,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5184,9 +5414,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5435,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a5..985a2c7049 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca2..739d13293f 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d335..b57f71f97e 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,42 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+/*
+ * Private state used by StartReadBuffers() and WaitReadBuffers(). Declared
+ * in public header only to allow inclusion in other structs, but contents
+ * should not be accessed.
+ */
+struct ReadBuffersOperation
+{
+ /* Parameters passed in to StartReadBuffers(). */
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ int nblocks;
+ BufferAccessStrategy strategy;
+ int flags;
+
+ /* Range of buffers, if we need to perform a read. */
+ int io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +292,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 0000000000..c4d3892bb2
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,52 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define PGSR_FLAG_FULL 0x04
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 58317eac44..162fe9300a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2116,6 +2116,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
@@ -2287,6 +2289,7 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
--
2.40.1
[text/x-diff] v9-0017-BitmapHeapScan-uses-streaming-read-API.patch (26.5K, ../../20240324173833.rdtw42aur2q5hhb2@liskov/18-v9-0017-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From f206f41cf66d80882724a96ab3ac9970e25bffc1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v9 17/17] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
ci-os-only:
---
src/backend/access/heap/heapam.c | 96 ++++--
src/backend/access/heap/heapam_handler.c | 346 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 44 +--
src/include/access/heapam.h | 16 +-
src/include/access/relscan.h | 7 -
src/include/access/tableam.h | 16 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 116 insertions(+), 418 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 9ddbaff4e2..0ed820e045 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -111,6 +111,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -333,6 +335,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -953,16 +971,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1045,12 +1056,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1063,12 +1068,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1094,12 +1093,6 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1117,6 +1110,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10274,3 +10270,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_tbmiterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 9dfc99d8fa..fe9ee5976f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -56,10 +56,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
-
static const TableAmRoutine heapam_methods;
@@ -2110,146 +2106,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (scan->rs_base.rs_pf_tbmiterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (scan->rs_base.rs_pf_tbmiterator)
- {
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_tbmiterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2270,7 +2188,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2279,9 +2197,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2331,23 +2249,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_tbmiterator &&
- hscan->pfblockno > hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2358,152 +2264,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
-
- if (pstate == NULL)
- {
- if (scan->rs_base.rs_pf_tbmiterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
- scan->rs_base.rs_pf_tbmiterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (scan->rs_base.rs_pf_tbmiterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(scan->rs_base.rs_pf_tbmiterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(scan->rs_base.rs_pf_tbmiterator);
- scan->rs_base.rs_pf_tbmiterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2538,41 +2298,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2683d8bc0c..c21dae466a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -128,14 +128,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -146,15 +138,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -171,13 +154,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -210,17 +187,10 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_tbmiterator = bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- bhs_begin_iterate(tbm, scan->bm_parallel, dsa);
-#endif /* USE_PREFETCH */
-
-
node->initialized = true;
}
@@ -319,7 +289,7 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
node->tbm = NULL;
@@ -363,7 +333,7 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
if (node->tbm)
tbm_free(node->tbm);
@@ -519,14 +489,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -557,11 +523,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 22bdccc2a9..1cad9c04f0 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -81,18 +85,6 @@ typedef struct HeapScanDescData
*/
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used for prefetching in bitmap table scans
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 65092d7226..00a123ee92 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -57,15 +57,8 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
BitmapHeapIterator *rs_tbmiterator;
- /* Prefetch iterator */
- BitmapHeapIterator *rs_pf_tbmiterator;
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate prefetch and current block stay in sync */
- BlockNumber blockno;
-
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5ad3eff539..4bc021f0eb 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -933,7 +933,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_tbmiterator = NULL;
- result->rs_pf_tbmiterator = NULL;
result->bm_parallel = NULL;
return result;
}
@@ -999,13 +998,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_tbmiterator);
scan->rs_tbmiterator = NULL;
-#ifdef USE_PREFETCH
- if (scan->rs_pf_tbmiterator)
- {
- bhs_end_iterate(scan->rs_pf_tbmiterator);
- scan->rs_pf_tbmiterator = NULL;
- }
-#endif
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1022,14 +1014,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_tbmiterator);
scan->rs_tbmiterator = NULL;
-
-#ifdef USE_PREFETCH
- if (scan->rs_pf_tbmiterator)
- {
- bhs_end_iterate(scan->rs_pf_tbmiterator);
- scan->rs_pf_tbmiterator = NULL;
- }
-#endif
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 7a3fdf9cd4..c1778653c7 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1758,11 +1758,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1770,10 +1766,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-24 18:22 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-24 18:22 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/24/24 18:38, Melanie Plageman wrote:
> On Sun, Mar 24, 2024 at 01:36:19PM +0100, Tomas Vondra wrote:
>>
>>
>> On 3/23/24 01:26, Melanie Plageman wrote:
>>> On Fri, Mar 22, 2024 at 08:22:11PM -0400, Melanie Plageman wrote:
>>>> On Tue, Mar 19, 2024 at 02:33:35PM +0200, Heikki Linnakangas wrote:
>>>>> On 18/03/2024 17:19, Melanie Plageman wrote:
>>>>>> I've attached v7 rebased over this commit.
>>>>>
>>>>> If we delayed table_beginscan_bm() call further, after starting the TBM
>>>>> iterator, we could skip it altogether when the iterator is empty.
>>>>>
>>>>> That's a further improvement, doesn't need to be part of this patch set.
>>>>> Just caught my eye while reading this.
>>>>
>>>> Hmm. You mean like until after the first call to tbm_[shared]_iterate()?
>>>> AFAICT, tbm_begin_iterate() doesn't tell us anything about whether or
>>>> not the iterator is "empty". Do you mean cases when the bitmap has no
>>>> blocks in it? It seems like we should be able to tell that from the
>>>> TIDBitmap.
>>>>
>>>>>
>>>>>> v7-0003-Push-BitmapHeapScan-skip-fetch-optimization-into-.patch
>>>>>
>>>>> I suggest to avoid the double negative with SO_CAN_SKIP_FETCH, and call the
>>>>> flag e.g. SO_NEED_TUPLE.
>>>>
>>>> Agreed. Done in attached v8. Though I wondered if it was a bit weird
>>>> that the flag is set in the common case and not set in the uncommon
>>>> case...
>>>
>>> v8 actually attached this time
>>
>> I tried to run the benchmarks with v8, but unfortunately it crashes for
>> me very quickly (I've only seen 0015 to crash, so I guess the bug is in
>> that patch).
>>
>> The backtrace attached, this doesn't seem right:
>>
>> (gdb) p hscan->rs_cindex
>> $1 = 543516018
>
> Thanks for reporting this! I hadn't seen it crash on my machine, so I
> didn't realize that I was no longer initializing rs_cindex and
> rs_ntuples on the first call to heapam_bitmap_next_tuple() (since
> heapam_bitmap_next_block() wasn't being called first). I've done this in
> attached v9.
>
OK, I've restarted the tests with v9.
> I haven't had a chance yet to reproduce the regressions you saw in the
> streaming read user patch or to look closely at the performance results.
So you tried to reproduce it and didn't hit the issue? Or didn't have
time to look into that yet? FWIW with v7 it failed almost immediately
(only a couple queries until hitting one triggering the issue), but v9
that's not the case (hundreds of queries without an error).
> I don't anticipate the streaming read user will have any performance
> differences in this v9 from v6, since I haven't yet rebased in Thomas'
> latest streaming read API changes nor addressed any other potential
> regression sources.
>
OK, understood. It'll be interesting to see the behavior with the new
version of Thomas' patch.
I however wonder what the plan with these patches is - do we still plan
to get some of this into v17? It seems to me we're getting uncomfortably
close to the end of the cycle, with a fairly incomplete idea of how it
affects performance.
Which is why I've been focusing more on the refactoring patches (up to
0015), to make sure those don't cause regressions if committed. And I
think that's generally true.
But for the main StreamingRead API the situation is very different.
> I tried rebasing in Thomas' latest version today and something is
> causing a crash that I have yet to figure out. v10 of this patchset will
> have his latest version once I get that fixed. I wanted to share this
> version with what I think is a bug fix for the crash you saw first.
>
Understood. I'll let the tests with v9 run for now.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-24 20:12 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-24 20:12 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sun, Mar 24, 2024 at 2:22 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> On 3/24/24 18:38, Melanie Plageman wrote:
> > I haven't had a chance yet to reproduce the regressions you saw in the
> > streaming read user patch or to look closely at the performance results.
>
> So you tried to reproduce it and didn't hit the issue? Or didn't have
> time to look into that yet? FWIW with v7 it failed almost immediately
> (only a couple queries until hitting one triggering the issue), but v9
> that's not the case (hundreds of queries without an error).
I haven't started trying to reproduce it yet.
> I however wonder what the plan with these patches is - do we still plan
> to get some of this into v17? It seems to me we're getting uncomfortably
> close to the end of the cycle, with a fairly incomplete idea of how it
> affects performance.
>
> Which is why I've been focusing more on the refactoring patches (up to
> 0015), to make sure those don't cause regressions if committed. And I
> think that's generally true.
Thank you for testing the refactoring patches with this in mind! Out
of the refactoring patches, I think there is a subset of them that
have independent value without the streaming read user. I think it is
worth committing the first few patches because they remove a table AM
layering violation. IMHO, all of the patches up to "Make
table_scan_bitmap_next_block() async friendly" make the code nicer and
better. And, if folks like the patch "Remove
table_scan_bitmap_next_block()", then I think I could rebase that back
in on top of "Make table_scan_bitmap_next_block() async friendly".
This would mean table AMs would only have to implement one callback
(table_scan_bitmap_next_tuple()) which I also think is a net
improvement and simplification.
The other refactoring patches may not be interesting without the
streaming read user.
> But for the main StreamingRead API the situation is very different.
My intent for the bitmapheapscan streaming read user was to get it
into 17, but I'm not sure that looks likely. The main issues Thomas is
looking into right now are related to regressions for a fully cached
scan (noticeable with the pg_prewarm streaming read user). With all of
these fixed, I anticipate we will still see enough behavioral
differences with the bitmapheap scan streaming read user that it may
not be committable in time. Though, I have yet to work on reproducing
the regressions with the BHS streaming read user mostly because I was
focused on getting the refactoring ready and not as much because the
streaming read API is unstable.
- Melanie
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-24 21:59 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-24 21:59 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/24/24 21:12, Melanie Plageman wrote:
> On Sun, Mar 24, 2024 at 2:22 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>>
>> On 3/24/24 18:38, Melanie Plageman wrote:
>>> I haven't had a chance yet to reproduce the regressions you saw in the
>>> streaming read user patch or to look closely at the performance results.
>>
>> So you tried to reproduce it and didn't hit the issue? Or didn't have
>> time to look into that yet? FWIW with v7 it failed almost immediately
>> (only a couple queries until hitting one triggering the issue), but v9
>> that's not the case (hundreds of queries without an error).
>
> I haven't started trying to reproduce it yet.
>
>> I however wonder what the plan with these patches is - do we still plan
>> to get some of this into v17? It seems to me we're getting uncomfortably
>> close to the end of the cycle, with a fairly incomplete idea of how it
>> affects performance.
>>
>> Which is why I've been focusing more on the refactoring patches (up to
>> 0015), to make sure those don't cause regressions if committed. And I
>> think that's generally true.
>
> Thank you for testing the refactoring patches with this in mind! Out
> of the refactoring patches, I think there is a subset of them that
> have independent value without the streaming read user. I think it is
> worth committing the first few patches because they remove a table AM
> layering violation. IMHO, all of the patches up to "Make
> table_scan_bitmap_next_block() async friendly" make the code nicer and
> better. And, if folks like the patch "Remove
> table_scan_bitmap_next_block()", then I think I could rebase that back
> in on top of "Make table_scan_bitmap_next_block() async friendly".
> This would mean table AMs would only have to implement one callback
> (table_scan_bitmap_next_tuple()) which I also think is a net
> improvement and simplification.
>
> The other refactoring patches may not be interesting without the
> streaming read user.
>
I admit not reviewing the individual patches very closely yet, but this
matches how I understood them - that at least some are likely an
improvement on their own, not just as a refactoring preparing for the
switch to streaming reads.
We only have ~2 weeks left, so it's probably time to focus on getting at
least those improvements committed. I see Heikki was paying way more
attention to the patches than me, though ...
BTW when you say "up to 'Make table_scan_bitmap_next_block() async
friendly'" do you mean including that patch, or that this is the first
patch that is not one of the independently useful patches.
(I took a quick look at the first couple patches and I appreciate that
you keep separate patches with small cosmetic changes to keep the actual
patch smaller and easier to understand.)
>> But for the main StreamingRead API the situation is very different.
>
> My intent for the bitmapheapscan streaming read user was to get it
> into 17, but I'm not sure that looks likely. The main issues Thomas is
> looking into right now are related to regressions for a fully cached
> scan (noticeable with the pg_prewarm streaming read user). With all of
> these fixed, I anticipate we will still see enough behavioral
> differences with the bitmapheap scan streaming read user that it may
> not be committable in time. Though, I have yet to work on reproducing
> the regressions with the BHS streaming read user mostly because I was
> focused on getting the refactoring ready and not as much because the
> streaming read API is unstable.
>
I don't have a very good intuition regarding impact of the streaming API
patch on performance. I haven't been following that thread very closely,
but AFAICS there wasn't much discussion about that - perhaps it happened
offlist, not sure. So who knows, really?
Which is why I started looking at this patch instead - it seemed easier
to benchmark with a somewhat realistic workload.
But yeah, there certainly were significant behavior changes, and it's
unlikely that whatever Thomas did in v8 made them go away.
FWIW I certainly am *not* suggesting there must be no behavior changes,
that's simply not possible. I'm not even suggesting no queries must get
slower - given the dependence on storage, I think some regressions are
pretty much inevitable. But it's still be good to know the regressions
are reasonably rare exceptions rather than the common case, and that's
not what I'm seeing ...
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-24 22:37 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-25 16:07 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-24 22:37 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sun, Mar 24, 2024 at 5:59 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
>
> BTW when you say "up to 'Make table_scan_bitmap_next_block() async
> friendly'" do you mean including that patch, or that this is the first
> patch that is not one of the independently useful patches.
I think the code is easier to understand with "Make
table_scan_bitmap_next_block() async friendly". Prior to that commit,
table_scan_bitmap_next_block() could return false even when the bitmap
has more blocks and expects the caller to handle this and invoke it
again. I think that interface is very confusing. The downside of the
code in that state is that the code for prefetching is still in the
BitmapHeapNext() code and the code for getting the current block is in
the heap AM-specific code. I took a stab at fixing this in v9's 0013,
but the outcome wasn't very attractive.
What I will do tomorrow is reorder and group the commits such that all
of the commits that are useful independent of streaming read are first
(I think 0014 and 0015 are independently valuable but they are on top
of some things that are only useful to streaming read because they are
more recently requested changes). I think I can actually do a bit of
simplification in terms of how many commits there are and what is in
each. Just to be clear, v9 is still reviewable. I am just going to go
back and change what is included in each commit.
> (I took a quick look at the first couple patches and I appreciate that
> you keep separate patches with small cosmetic changes to keep the actual
> patch smaller and easier to understand.)
Thanks!
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-25 16:07 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Melanie Plageman @ 2024-03-25 16:07 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Sun, Mar 24, 2024 at 06:37:20PM -0400, Melanie Plageman wrote:
> On Sun, Mar 24, 2024 at 5:59 PM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
> >
> > BTW when you say "up to 'Make table_scan_bitmap_next_block() async
> > friendly'" do you mean including that patch, or that this is the first
> > patch that is not one of the independently useful patches.
>
> I think the code is easier to understand with "Make
> table_scan_bitmap_next_block() async friendly". Prior to that commit,
> table_scan_bitmap_next_block() could return false even when the bitmap
> has more blocks and expects the caller to handle this and invoke it
> again. I think that interface is very confusing. The downside of the
> code in that state is that the code for prefetching is still in the
> BitmapHeapNext() code and the code for getting the current block is in
> the heap AM-specific code. I took a stab at fixing this in v9's 0013,
> but the outcome wasn't very attractive.
>
> What I will do tomorrow is reorder and group the commits such that all
> of the commits that are useful independent of streaming read are first
> (I think 0014 and 0015 are independently valuable but they are on top
> of some things that are only useful to streaming read because they are
> more recently requested changes). I think I can actually do a bit of
> simplification in terms of how many commits there are and what is in
> each. Just to be clear, v9 is still reviewable. I am just going to go
> back and change what is included in each commit.
So, attached v10 does not include the new version of streaming read API.
I focused instead on the refactoring patches commit regrouping I
mentioned here.
I realized "Remove table_scan_bitmap_next_block()" can't easily be moved
down below "Push BitmapHeapScan prefetch code into heapam.c" because we
have to do BitmapAdjustPrefetchTarget() and
BitmapAdjustPrefetchIterator() on either side of getting the next block
(via table_scan_bitmap_next_block()).
"Push BitmapHeapScan prefetch code into heapam.c" isn't very nice
because it adds a lot of bitmapheapscan specific members to
TableScanDescData and HeapScanDescData. I thought about wrapping all of
those members in some kind of BitmapHeapScanTableState struct -- but I
don't like that because the members are spread out across
HeapScanDescData and TableScanDescData so not all of them would go in
BitmapHeapScanTableState. I could move the ones I put in
HeapScanDescData back into TableScanDescData and then wrap that in a
BitmapHeapScanTableState. I haven't done that in this version.
I did manage to move "Unify parallel and serial BitmapHeapScan iterator
interfaces" down below the line of patches which are only useful if
the streaming read user also goes in.
In attached v10, all patches up to and including "Unify parallel and
serial BitmapHeapScan iterator interfaces" (0010) are proposed for
master with or without the streaming read API.
0010 does add additional indirection and thus pointer dereferencing for
accessing the iterators, which doesn't feel good. But, it does simplify
the code.
Perhaps it is worth renaming the existing TableScanDescData->rs_parallel
(a ParallelTableScanDescData) to something like rs_seq_parallel. It is
only for sequential scans and scans of tables when building indexes but
the comments say it is for parallel scans in general. There is a similar
member in HeapScanDescData called rs_parallelworkerdata.
- Melanie
Attachments:
[text/x-diff] v10-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/2-v10-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 62f6c877d130325748bf18724fc8166932605091 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v10 01/17] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index e57a0b7ea3..e78d793f69 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..93fdcd226b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v10-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/3-v10-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From c6ed55265735c2d2a47da1fc6339d3cc1db954e4 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v10 02/17] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226b..c64530674b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v10-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (15.0K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/4-v10-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From 844cfbb2c2700da1fd8a290776523190e9c73ef9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v10 03/17] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index cc67dd813d..6b2863391f 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,6 +948,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1036,6 +1038,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1055,6 +1063,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2b7c702642..7fdccaf613 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2236,6 +2255,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674b..83d9db8f39 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 368c570a0f..cef54e2d5d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 65834caeb1..1bc5f7c057 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -952,9 +959,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1774c56ae3..6871db9b21 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,10 +1785,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1808,9 +1805,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v10-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.2K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/5-v10-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From 8075018e79311547895034ef25be5294aee9e9fb Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v10 04/17] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39..5df3b5ca46 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v10-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.3K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/6-v10-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 370a50ea551d440476160e083a20ac356bb35c0a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v10 05/17] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46..404de0595e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v10-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch (4.4K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/7-v10-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch)
download | inline diff:
From 1538728ba1424299360e5845afbbf51ecebf5947 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v10 06/17] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 7fdccaf613..849cac3947 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2242,6 +2243,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595e..c95e3412da 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1bc5f7c057..b9ba4f9fb3 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -804,6 +804,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -819,7 +822,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1988,14 +1992,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2006,7 +2012,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v10-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (2.9K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/8-v10-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From b03464c30573a28e1f2e5e0915e75dc26a823a3d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v10 07/17] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da..49938c9ed4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v10-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/9-v10-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From 1c32c668a87649990c072dbdde69f14d8ba6b2ce Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v10 08/17] Remove table_scan_bitmap_next_tuple parameter
tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 849cac3947..cf4387f443 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2250,7 +2250,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed4..282dcb9791 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b9ba4f9fb3..bcf1497f67 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -795,10 +795,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -829,15 +826,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2025,7 +2017,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2037,7 +2028,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v10-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/10-v10-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 9f6b3738ac4f6648d830168d8659aa49510baecc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v10 09/17] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index cf4387f443..2ad785e511 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,18 +2114,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2144,16 +2177,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2245,7 +2269,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb9791..7e73583fe5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno > node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index bcf1497f67..a820cc8c99 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -788,19 +789,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -819,8 +815,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -957,9 +953,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1019,6 +1019,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1029,6 +1044,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1981,19 +2011,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2003,8 +2031,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6871db9b21..8688bc5ab0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,8 +1783,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1793,9 +1791,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,8 +1803,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1813,9 +1811,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v10-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.0K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/11-v10-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From 64d69f1249731b740c510cd86163517a8b1320ec Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v10 10/17] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext(). Now, on the first scan, the the
iterator is created after the scan descriptor is created.
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 163 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 29 +---
src/include/executor/nodeBitmapHeapscan.h | 10 ++
src/include/nodes/execnodes.h | 8 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2ad785e511..c76849a98e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2133,10 +2133,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7e73583fe5..fe471a8a0c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,6 +56,56 @@ static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ dsa_pointer shared_area,
+ dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(shared_area))
+ result->parallel = tbm_attach_shared_iterate(personal_area, shared_area);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -97,43 +147,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
+ if (init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -154,21 +184,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +216,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ dsa);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ node->pf_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
+ dsa);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -280,7 +311,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ node->pf_iterator &&
node->pfblockno > node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -321,12 +352,11 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
@@ -335,7 +365,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -348,8 +378,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -371,7 +399,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -431,23 +459,22 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (prefetch_iterator)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
node->prefetch_pages++;
@@ -475,8 +502,6 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
if (prefetch_iterator)
{
while (1)
@@ -500,12 +525,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
@@ -572,18 +597,17 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator)
+ {
+ bhs_end_iterate(node->pf_iterator);
+ node->pf_iterator = NULL;
+ }
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -628,12 +652,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (node->pf_iterator)
+ bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -671,11 +693,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
+ scanstate->pf_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..fb22f305bf 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,12 +20,12 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
+struct BitmapHeapIterator;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -44,8 +44,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ struct BitmapHeapIterator *rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a820cc8c99..29387166c1 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -957,8 +957,7 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
+ result->rs_bhs_iterator = NULL;
return result;
}
@@ -1021,17 +1020,8 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1046,17 +1036,8 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caa..cb56d20dc6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,5 +28,15 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8688bc5ab0..52cedd1b35 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1778,6 +1778,8 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+struct BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1786,12 +1788,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1806,12 +1807,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 4679660837..6d5cb0bdaa 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v10-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.2K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/12-v10-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From 8776aa053a556d61303f9f6d4af6ce7bb6732121 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v10 11/17] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c76849a98e..d85fee1e50 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2264,7 +2265,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index fe471a8a0c..076e1ff674 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -119,7 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -298,14 +297,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 29387166c1..edc54ecffe 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -797,8 +797,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -815,8 +815,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1994,15 +1996,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2013,7 +2017,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v10-0012-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/13-v10-0012-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From c7e25c55056880848dea5f23ef62b859475f8973 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v10 12/17] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fc..1dc4c99bf9 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639b..432fae5296 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v10-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch (20.7K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/14-v10-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch)
download | inline diff:
From 99f7c9224046b021bc82a9f3d48048fd6916a29b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v10 13/17] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 30 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++-------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/executor/nodeBitmapHeapscan.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
8 files changed, 113 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb..3aa457a29e 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544..033d525339 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d85fee1e50..105137396b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2122,7 +2122,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2134,9 +2134,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(scan->rs_bhs_iterator);
+ bhs_iterate(scan->rs_bhs_iterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2151,11 +2151,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2163,19 +2163,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2204,7 +2204,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2213,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2265,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 076e1ff674..78f79aafff 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -77,15 +77,16 @@ bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_ar
return result;
}
-TBMIterateResult *
-bhs_iterate(BitmapHeapIterator *iterator)
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result)
{
Assert(iterator);
+ Assert(result);
if (iterator->serial)
- return tbm_iterate(iterator->serial);
+ tbm_iterate(iterator->serial, result);
else
- return tbm_shared_iterate(iterator->parallel);
+ tbm_shared_iterate(iterator->parallel, result);
}
void
@@ -348,7 +349,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -360,8 +361,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -394,8 +395,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -462,10 +463,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -473,7 +476,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -482,13 +485,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -501,7 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -520,8 +523,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -529,17 +532,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf9..309a44bdb8 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae..3b432263bb 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index cb56d20dc6..3c330f86e6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -34,7 +34,7 @@ typedef struct BitmapHeapIterator
struct TBMSharedIterator *parallel;
} BitmapHeapIterator;
-extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result);
extern void bhs_end_iterate(BitmapHeapIterator *iterator);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae5296..f000c1af28 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v10-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (31.5K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/15-v10-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From 4d5dcb8783d92a2e2a37ea37bbf346545cd09337 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v10 14/17] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 262 +++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 341 ++--------------------
src/include/access/heapam.h | 17 ++
src/include/access/relscan.h | 8 +
src/include/access/tableam.h | 26 +-
src/include/nodes/execnodes.h | 14 -
7 files changed, 355 insertions(+), 339 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 6b2863391f..3d92fb5135 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,8 +948,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1032,6 +1040,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1044,6 +1058,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1069,6 +1089,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 105137396b..867061325c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,6 +55,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2112,6 +2115,73 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2130,6 +2200,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2270,6 +2342,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->rs_pf_bhs_iterator &&
+ hscan->pfblockno > hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2280,6 +2364,154 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+
+ if (pstate == NULL)
+ {
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2305,6 +2537,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 78f79aafff..187b288e68 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
dsa_pointer shared_area,
@@ -122,7 +118,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -142,7 +137,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
@@ -154,7 +149,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -169,23 +173,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
@@ -216,19 +213,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
- node->pf_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
- dsa);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
+ scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
+ scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
+ dsa);
}
#endif /* USE_PREFETCH */
@@ -243,36 +240,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -296,23 +263,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->pf_iterator &&
- node->pfblockno > node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,219 +289,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -594,22 +334,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator)
- {
- bhs_end_iterate(node->pf_iterator);
- node->pf_iterator = NULL;
- }
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -648,14 +378,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->pf_iterator)
- bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -688,17 +414,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->pf_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -738,13 +458,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -828,7 +541,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index cef54e2d5d..29fdd55893 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -82,6 +82,23 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used in serial BHS
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fb22f305bf..7938b741d6 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@
struct ParallelTableScanDescData;
struct BitmapHeapIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,6 +46,13 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
+ struct BitmapHeapIterator *rs_pf_bhs_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+ /* used to validate BHS prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index edc54ecffe..284ea3d864 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -800,17 +800,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -960,6 +949,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
+ result->rs_pf_bhs_iterator = NULL;
+ result->prefetch_maximum = 0;
+ result->bm_parallel = NULL;
return result;
}
@@ -1024,6 +1016,12 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1040,6 +1038,12 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 52cedd1b35..60916bf0d0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,18 +1785,11 @@ struct BitmapHeapIterator;
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1804,18 +1797,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v10-0015-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/16-v10-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From feec27e011ef4f1f15e65afca2564192b85b17c1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v10 15/17] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 3d92fb5135..8de1a11164 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -319,6 +319,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 867061325c..6af1791faa 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2109,12 +2109,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2148,8 +2142,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2512,30 +2506,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -2917,7 +2924,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index ce637a5a5d..1d6b03d1ca 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -92,8 +92,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 187b288e68..2f9387e51a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -230,44 +230,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
#endif /* USE_PREFETCH */
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 284ea3d864..44d0885d9e 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -788,36 +788,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -1998,44 +1982,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2046,7 +1999,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v10-0016-v7-Streaming-Read-API.patch (56.1K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/17-v10-0016-v7-Streaming-Read-API.patch)
download | inline diff:
From d5f6f3f2e80e9f85d054dfa99b186b24571930da Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:59:42 -0400
Subject: [PATCH v10 16/17] v7 Streaming Read API
---
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 659 +++++++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 642 +++++++++++++++-------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/include/storage/bufmgr.h | 45 ++
src/include/storage/streaming_read.h | 52 ++
src/tools/pgindent/typedefs.list | 3 +
10 files changed, 1227 insertions(+), 210 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca2..eec03f6f2b 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 0000000000..bcab44c802
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 0000000000..39aef2a84a
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 0000000000..d5c29b750d
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,659 @@
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ */
+typedef struct PgStreamingReadRange
+{
+ bool need_wait;
+ bool advice_issued;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index;
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+ ReadBuffersOperation operation;
+} PgStreamingReadRange;
+
+/*
+ * Streaming read object.
+ */
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int next_tail_buffer;
+ int distance;
+ bool started;
+ bool finished;
+ bool advice_enabled;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ /* Sometimes we need to buffer one block for flow control. */
+ BlockNumber unget_blocknum;
+ void *unget_per_buffer_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /*
+ * Make sure our bmr's smgr and persistent are populated. The caller
+ * asserts that the storage manager will remain valid.
+ */
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+
+ /*
+ * Choose a maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least
+ * MAX_BUFFER_PER_TRANSFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+ pgsr->callback = next_block_cb;
+ pgsr->bmr = bmr;
+ pgsr->forknum = forknum;
+
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out doing full-sized
+ * reads.
+ */
+ if (flags & PGSR_FLAG_FULL)
+ pgsr->distance = Min(MAX_BUFFERS_PER_TRANSFER, pgsr->max_pinned_buffers);
+ else
+ pgsr->distance = 1;
+
+ /*
+ * Space for the callback to store extra data along with each block. Note
+ * that we need one more than max_pinned_buffers, so we can return a
+ * pointer to a slot that can't be overwritten until the next call.
+ */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * size);
+
+ return pgsr;
+}
+
+/*
+ * Find the per-buffer data index for the Nth block of a range.
+ */
+static int
+get_per_buffer_data_index(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ int result;
+
+ /*
+ * Find slot in the circular buffer of per-buffer data, without using the
+ * expensive % operator.
+ */
+ result = range->per_buffer_data_index + n;
+ while (result >= pgsr->size)
+ result -= pgsr->size;
+ Assert(result == (range->per_buffer_data_index + n) % pgsr->size);
+
+ return result;
+}
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static void *
+get_per_buffer_data_by_index(PgStreamingRead *pgsr, int per_buffer_data_index)
+{
+ return (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * per_buffer_data_index;
+}
+
+/*
+ * Return a pointer to the per-buffer data for the Nth block of a range.
+ */
+static void *
+get_per_buffer_data(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ return get_per_buffer_data_by_index(pgsr,
+ get_per_buffer_data_index(pgsr,
+ range,
+ n));
+}
+
+/*
+ * Start reading the head range, and create a new head range. The new head
+ * range is returned. It may not be empty, if StartReadBuffers() couldn't
+ * start the entire range; in that case the returned range contains the
+ * remaining portion of the range.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_start_head_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+ PgStreamingReadRange *new_head_range;
+ int nblocks_pinned;
+ int flags;
+
+ /* Caller should make sure we never exceed max_ios. */
+ Assert((pgsr->ios_in_progress < pgsr->max_ios) ||
+ (pgsr->ios_in_progress == 0 && pgsr->max_ios == 0));
+
+ /* Should only call if the head range has some blocks to read. */
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If advice hasn't been suppressed, and this system supports it, this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (pgsr->advice_enabled &&
+ pgsr->max_ios > 0 &&
+ pgsr->started &&
+ head_range->blocknum != pgsr->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* Suppress advice on the first call, because it's too late to benefit. */
+ if (!pgsr->started)
+ pgsr->started = true;
+
+ /* We shouldn't be trying to pin more buffers that we're allowed to. */
+ Assert(pgsr->pinned_buffers + head_range->nblocks <= pgsr->max_pinned_buffers);
+
+ /* Start reading as many blocks as we can from the head range. */
+ nblocks_pinned = head_range->nblocks;
+ head_range->need_wait =
+ StartReadBuffers(pgsr->bmr,
+ head_range->buffers,
+ pgsr->forknum,
+ head_range->blocknum,
+ &nblocks_pinned,
+ pgsr->strategy,
+ flags,
+ &head_range->operation);
+
+ Assert(pgsr->pinned_buffers <= pgsr->max_pinned_buffers);
+
+ if (head_range->need_wait && (flags & READ_BUFFERS_ISSUE_ADVICE))
+ {
+ /*
+ * Since we've issued advice, we count an I/O in progress until we
+ * call WaitReadBuffers().
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ Assert(pgsr->ios_in_progress <= pgsr->max_ios);
+ }
+
+ /*
+ * StartReadBuffers() might have pinned fewer blocks than we asked it to,
+ * but always at least one.
+ */
+ Assert(nblocks_pinned <= head_range->nblocks);
+ Assert(nblocks_pinned >= 1);
+ pgsr->pinned_buffers += nblocks_pinned;
+
+ /*
+ * Remember where the next block would be after that, so we can detect
+ * sequential access next time.
+ */
+ pgsr->seq_blocknum = head_range->blocknum + nblocks_pinned;
+
+ /*
+ * Create a new head range. There must be space, because we have enough
+ * elements for every range to hold just one block, up to the pin limit.
+ */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ new_head_range = &pgsr->ranges[pgsr->head];
+ new_head_range->nblocks = 0;
+ new_head_range->advice_issued = false;
+
+ /*
+ * If we didn't manage to start the whole read above, we split the range,
+ * moving the remainder into the new head range.
+ */
+ if (nblocks_pinned < head_range->nblocks)
+ {
+ int nblocks_remaining = head_range->nblocks - nblocks_pinned;
+
+ head_range->nblocks = nblocks_pinned;
+
+ new_head_range->blocknum = head_range->blocknum + nblocks_pinned;
+ new_head_range->nblocks = nblocks_remaining;
+ }
+
+ /* The new range has per-buffer data starting after the previous range. */
+ new_head_range->per_buffer_data_index =
+ get_per_buffer_data_index(pgsr, head_range, nblocks_pinned);
+
+ return new_head_range;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow pg_streaming_unget_block() to work.
+ */
+static BlockNumber
+pg_streaming_get_block(PgStreamingRead *pgsr, void *per_buffer_data)
+{
+ BlockNumber result;
+
+ if (unlikely(pgsr->unget_blocknum != InvalidBlockNumber))
+ {
+ /*
+ * If we had to unget a block, now it is time to return that one
+ * again.
+ */
+ result = pgsr->unget_blocknum;
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+ /*
+ * The same per_buffer_data element must have been used, and still
+ * contains whatever data the callback wrote into it. So we just
+ * sanity-check that we were called with the value that
+ * pg_streaming_unget_block() pushed back.
+ */
+ Assert(per_buffer_data == pgsr->unget_per_buffer_data);
+ }
+ else
+ {
+ /* Use the installed callback directly. */
+ result = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ }
+
+ return result;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later. This *must* be called with the
+ * last value returned by pg_streaming_get_block().
+ */
+static void
+pg_streaming_unget_block(PgStreamingRead *pgsr, BlockNumber blocknum, void *per_buffer_data)
+{
+ Assert(pgsr->unget_blocknum == InvalidBlockNumber);
+ pgsr->unget_blocknum = blocknum;
+ pgsr->unget_per_buffer_data = per_buffer_data;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *range;
+
+ /* If we're finished, don't look ahead. */
+ if (pgsr->finished)
+ return;
+
+ /*
+ * We we've already started the maximum allowed number of I/Os, don't look
+ * ahead. There is a special case for max_ios == 0.
+ */
+ if (pgsr->max_ios > 0 && pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /* Can't pin any more buffers. */
+ if (pgsr->pinned_buffers == pgsr->distance)
+ return;
+
+ /*
+ * Keep trying to add new blocks to the end of the head range while doing
+ * so wouldn't exceed the distance limit.
+ */
+ range = &pgsr->ranges[pgsr->head];
+ while (pgsr->pinned_buffers + range->nblocks < pgsr->distance)
+ {
+ BlockNumber blocknum;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ if (range->nblocks == lengthof(range->buffers))
+ {
+ /* Start as much of it as we can. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /* If we're now at the I/O limit, stop here. */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * That might have only been partially started, but always
+ * processes at least one so that'll do for now.
+ */
+ Assert(range->nblocks < lengthof(range->buffers));
+ }
+
+ /* Find per-buffer data slot for the next block. */
+ per_buffer_data = get_per_buffer_data(pgsr, range, range->nblocks);
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pg_streaming_get_block(pgsr, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ pgsr->finished = true;
+ break;
+ }
+
+ /*
+ * Is there a head range that we cannot extend, because the requested
+ * block is not consecutive?
+ */
+ if (range->nblocks > 0 &&
+ range->blocknum + range->nblocks != blocknum)
+ {
+ /* Yes. Start it, so we can begin building a new one. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * It's possible that it was only partially started, and we have a
+ * new range with the remainder. Keep starting I/Os until we get
+ * it all out of the way, or we hit the I/O limit.
+ */
+ while (range->nblocks > 0 && pgsr->ios_in_progress < pgsr->max_ios)
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * We do have to worry about I/O capacity running out if the head
+ * range was split. In that case we have to 'unget' the block
+ * returned by the callback.
+ */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ {
+ pg_streaming_unget_block(pgsr, blocknum, per_buffer_data);
+ return;
+ }
+ }
+
+ /* If we have a new, empty range, initialize the start block. */
+ if (range->nblocks == 0)
+ range->blocknum = blocknum;
+
+ /* This block extends the range by one. */
+ Assert(range->blocknum + range->nblocks == blocknum);
+ range->nblocks++;
+ };
+
+ /*
+ * Normally we don't start the head range, preferring to give it a chance
+ * to grow to full size once more buffers have been consumed. In cases
+ * where that can't possibly happen, we might as well start the read
+ * immediately.
+ */
+ if ((range->nblocks > 0 && pgsr->finished) ||
+ (range->nblocks == pgsr->distance))
+ pg_streaming_read_start_head_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ PgStreamingReadRange *tail_range;
+
+ for (;;)
+ {
+ if (pgsr->tail != pgsr->head)
+ {
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to wait for a ReadBuffers operation to finish before
+ * returning the buffers in this range?
+ */
+ if (tail_range->need_wait)
+ {
+ int distance;
+
+ Assert(pgsr->next_tail_buffer == 0);
+ WaitReadBuffers(&tail_range->operation);
+ tail_range->need_wait = false;
+
+ /*
+ * We don't really know if the kernel generated a physical I/O
+ * when we issued advice, let alone when it finished, but it
+ * has certainly finished now because we've performed the
+ * read.
+ */
+ if (tail_range->advice_issued)
+ {
+
+ Assert(pgsr->ios_in_progress > 0);
+ pgsr->ios_in_progress--;
+
+ /*
+ * Look-ahead distance ramps up rapidly if we're issuing
+ * advice, so we can search for new more I/Os to start.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ else
+ {
+ /*
+ * There is no point in increasing look-ahead distance if
+ * we've already reached the full I/O size, since we're
+ * not issuing advice. Extra distance would only pin more
+ * buffers for no benefit.
+ */
+ if (pgsr->distance > MAX_BUFFERS_PER_TRANSFER)
+ {
+ /*
+ * Look-ahead distance gradually decays to full I/O
+ * size.
+ */
+ pgsr->distance--;
+ }
+ else
+ {
+ /*
+ * Look-ahead distance ramps up rapidly, but not more
+ * that the full I/O size.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, MAX_BUFFERS_PER_TRANSFER);
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ }
+ }
+ else if (pgsr->next_tail_buffer == 0)
+ {
+ /* No I/O necessary. Look-ahead distance gradually decays. */
+ if (pgsr->distance > 1)
+ pgsr->distance--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(pgsr, tail_range, buffer_index);
+
+ /* We may be able to get another I/O started. */
+ pg_streaming_read_look_ahead(pgsr);
+
+ return buffer;
+ }
+
+ /* Advance tail to next range. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+ else
+ {
+ /*
+ * If tail crashed into head, and head is not empty, then it is
+ * time to start that range. Otherwise, force a look-ahead, to
+ * kick start the stream.
+ */
+ Assert(pgsr->tail == pgsr->head);
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ {
+ pg_streaming_read_start_head_range(pgsr);
+ }
+ else
+ {
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* Finished? */
+ if (pgsr->tail == pgsr->head &&
+ pgsr->ranges[pgsr->head].nblocks == 0)
+ break;
+ }
+ }
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ pgsr->finished = true;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(pgsr->pinned_buffers == 0);
+ Assert(pgsr->ios_in_progress == 0);
+
+ /* Release memory. */
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c..d0e9c7deff 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -471,10 +476,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +504,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +785,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +797,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +819,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +990,68 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
+/*
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
+ */
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* ReadBuffer_common -- common logic for all ReadBuffer variants
*
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+ BufferAccessStrategy strategy)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int nblocks;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1070,405 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ nblocks = 1;
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
+ else
+ flags = 0;
+ if (StartReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ &nblocks,
+ strategy,
+ flags,
+ &operation))
+ WaitReadBuffers(&operation);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ ZeroBuffer(buffer, mode);
+
+ return buffer;
+}
+static Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested.
+ *
+ * If false is returned, no I/O is necessary and WaitReadBuffers() is not
+ * necessary. If true is returned, one I/O has been started, and
+ * WaitReadBuffers() must be called with the same operation object before the
+ * buffers are accessed. Along with the operation object, the caller-supplied
+ * array of buffers must remain valid until WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+bool
+StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation)
+{
+ int actual_nblocks = *nblocks;
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ operation->bmr = bmr;
+ operation->forknum = forkNum;
+ operation->blocknum = blockNum;
+ operation->buffers = buffers;
+ operation->nblocks = actual_nblocks;
+ operation->strategy = strategy;
+ operation->flags = flags;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ operation->io_buffers_len = 0;
- /*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
- */
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
- else
+ for (int i = 0; i < actual_nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ bool found;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ buffers[i] = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum + i,
+ strategy,
+ &found);
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = operation->nblocks = *nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ operation->io_buffers_len++;
+ }
+ }
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (operation->io_buffers_len > 0)
+ {
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
- {
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
+ /*
+ * In theory we should only do this if PrepareReadBuffers() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(bmr.smgr, forkNum, blockNum, operation->io_buffers_len);
}
+
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
}
+ else
+ {
+ return false;
+ }
+}
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
{
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ /*
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
+ */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+ bmr = operation->bmr;
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
-
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
}
else
{
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
}
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ /*
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
+ */
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
+ else
+ pgBufferUsage.shared_blks_read += nblocks;
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ for (int i = 0; i < nblocks; ++i)
+ {
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- return BufferDescriptorGetBuffer(bufHdr);
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
+
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
+ {
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
+ {
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
+ }
+ else
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
+
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
+
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
+
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
+
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for StartReadBuffers. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1476,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1535,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1603,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1638,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +1994,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2259,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2282,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2372,7 +2597,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2611,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3679,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5184,9 +5414,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5435,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a5..985a2c7049 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca2..739d13293f 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d335..b57f71f97e 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,42 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+/*
+ * Private state used by StartReadBuffers() and WaitReadBuffers(). Declared
+ * in public header only to allow inclusion in other structs, but contents
+ * should not be accessed.
+ */
+struct ReadBuffersOperation
+{
+ /* Parameters passed in to StartReadBuffers(). */
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ int nblocks;
+ BufferAccessStrategy strategy;
+ int flags;
+
+ /* Range of buffers, if we need to perform a read. */
+ int io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +292,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 0000000000..c4d3892bb2
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,52 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define PGSR_FLAG_FULL 0x04
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 6d5cb0bdaa..4558c2ecfc 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2116,6 +2116,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
@@ -2288,6 +2290,7 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
--
2.40.1
[text/x-diff] v10-0017-BitmapHeapScan-uses-streaming-read-API.patch (26.3K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/18-v10-0017-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From fb7740a535532673e50b87014a0b77e565c46af8 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v10 17/17] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
ci-os-only:
---
src/backend/access/heap/heapam.c | 96 ++++--
src/backend/access/heap/heapam_handler.c | 347 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 43 +--
src/include/access/heapam.h | 21 +-
src/include/access/relscan.h | 6 -
src/include/access/tableam.h | 14 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 114 insertions(+), 422 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 8de1a11164..5421b552d9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -108,6 +108,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -330,6 +332,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -950,16 +968,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1042,12 +1053,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1060,12 +1065,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1091,12 +1090,6 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1114,6 +1107,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10025,3 +10021,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_bhs_iterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 6af1791faa..fe9ee5976f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,9 +55,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2109,147 +2106,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_bhs_iterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2270,7 +2188,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2279,9 +2197,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2331,23 +2249,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_bhs_iterator &&
- hscan->pfblockno > hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2358,153 +2264,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2539,41 +2298,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2f9387e51a..f2662ea542 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -131,14 +131,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -149,15 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,13 +157,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -213,22 +190,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- {
- scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
- scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
- dsa);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
}
@@ -525,14 +492,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -563,11 +526,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 29fdd55893..1b8ce82c9e 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -82,23 +86,6 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /*
- * These fields only used for prefetching in bitmap table scans
- */
-
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used in serial BHS
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
-
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 7938b741d6..02893bf99b 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -46,13 +46,7 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
- struct BitmapHeapIterator *rs_pf_bhs_iterator;
-
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate BHS prefetch and current block stay in sync */
- BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 44d0885d9e..b1b09bbac2 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -933,8 +933,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
- result->rs_pf_bhs_iterator = NULL;
- result->prefetch_maximum = 0;
result->bm_parallel = NULL;
return result;
}
@@ -1000,12 +998,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1022,12 +1014,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 60916bf0d0..430668f597 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1758,11 +1758,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1770,10 +1766,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-25 16:07 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-27 19:37 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-28 05:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 2 replies; 248+ messages in thread
From: Melanie Plageman @ 2024-03-27 19:37 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Mon, Mar 25, 2024 at 12:07:09PM -0400, Melanie Plageman wrote:
> On Sun, Mar 24, 2024 at 06:37:20PM -0400, Melanie Plageman wrote:
> > On Sun, Mar 24, 2024 at 5:59 PM Tomas Vondra
> > <tomas.vondra@enterprisedb.com> wrote:
> > >
> > > BTW when you say "up to 'Make table_scan_bitmap_next_block() async
> > > friendly'" do you mean including that patch, or that this is the first
> > > patch that is not one of the independently useful patches.
> >
> > I think the code is easier to understand with "Make
> > table_scan_bitmap_next_block() async friendly". Prior to that commit,
> > table_scan_bitmap_next_block() could return false even when the bitmap
> > has more blocks and expects the caller to handle this and invoke it
> > again. I think that interface is very confusing. The downside of the
> > code in that state is that the code for prefetching is still in the
> > BitmapHeapNext() code and the code for getting the current block is in
> > the heap AM-specific code. I took a stab at fixing this in v9's 0013,
> > but the outcome wasn't very attractive.
> >
> > What I will do tomorrow is reorder and group the commits such that all
> > of the commits that are useful independent of streaming read are first
> > (I think 0014 and 0015 are independently valuable but they are on top
> > of some things that are only useful to streaming read because they are
> > more recently requested changes). I think I can actually do a bit of
> > simplification in terms of how many commits there are and what is in
> > each. Just to be clear, v9 is still reviewable. I am just going to go
> > back and change what is included in each commit.
>
> So, attached v10 does not include the new version of streaming read API.
> I focused instead on the refactoring patches commit regrouping I
> mentioned here.
Attached v11 has the updated Read Stream API Thomas sent this morning
[1]. No other changes.
- Melanie
[1] https://www.postgresql.org/message-id/CA%2BhUKGJTwrS7F%3DuJPx3SeigMiQiW%2BLJaOkjGyZdCntwyMR%3DuAw%40...
Attachments:
[text/x-diff] v11-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240327193750.3mlcmzqondpj27xe@liskov/2-v11-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From c8bdc1f2143adb2d9a5c9f8f69c249b307287a1a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v11 01/17] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 8d3675be95..a254e3175b 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..93fdcd226b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v11-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/3-v11-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 6e41737489dbeb07bc9f0cf9c595c11115cb985b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v11 02/17] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226b..c64530674b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v11-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (15.0K, ../../20240327193750.3mlcmzqondpj27xe@liskov/4-v11-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From 523d1a454c77ccb8734a16b02338145deacd0a60 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v11 03/17] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 2f6527df0d..ed3a3607b7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,6 +948,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1036,6 +1038,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1055,6 +1063,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 6abfe36dec..5ba8cb3657 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2182,6 +2183,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2294,6 +2313,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674b..83d9db8f39 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index f112245373..c7a538221a 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index fc0e702715..ae758857bd 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -963,9 +970,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1774c56ae3..6871db9b21 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,10 +1785,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1808,9 +1805,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v11-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/5-v11-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From 54afa2d047bd10d451c7120ec33e476974534219 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v11 04/17] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39..5df3b5ca46 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v11-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.3K, ../../20240327193750.3mlcmzqondpj27xe@liskov/6-v11-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 227a1ea3d6ceeecc57538a3a4ecb4839fd769445 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v11 05/17] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46..404de0595e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v11-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch (4.4K, ../../20240327193750.3mlcmzqondpj27xe@liskov/7-v11-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch)
download | inline diff:
From 42ad33d1f16aeb9f763a896f8677254292f9b4c2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v11 06/17] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 5ba8cb3657..ce47a158ae 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2172,7 +2172,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2300,6 +2301,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595e..c95e3412da 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index ae758857bd..5c2a7b7422 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -815,6 +815,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -830,7 +833,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2023,14 +2027,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2041,7 +2047,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v11-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (2.9K, ../../20240327193750.3mlcmzqondpj27xe@liskov/8-v11-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From 8fec5f876636383e802dbb10c107892bb71a4d8d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v11 07/17] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da..49938c9ed4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v11-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240327193750.3mlcmzqondpj27xe@liskov/9-v11-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From 836b7922f3d2c85472525d22304436409f06fd90 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v11 08/17] Remove table_scan_bitmap_next_tuple parameter
tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ce47a158ae..ddcdbbaf7e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2308,7 +2308,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed4..282dcb9791 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5c2a7b7422..9c7b8bf162 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -806,10 +806,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -840,15 +837,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2060,7 +2052,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2072,7 +2063,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v11-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240327193750.3mlcmzqondpj27xe@liskov/10-v11-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 9affa3100a127c747608e3cd8726696aabe93530 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v11 09/17] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ddcdbbaf7e..196f69e30e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2172,18 +2172,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2202,16 +2235,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2303,7 +2327,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb9791..7e73583fe5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno > node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 9c7b8bf162..68478d16b2 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -799,19 +800,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -830,8 +826,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -968,9 +964,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1030,6 +1030,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1040,6 +1055,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -2016,19 +2046,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2038,8 +2066,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6871db9b21..8688bc5ab0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,8 +1783,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1793,9 +1791,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,8 +1803,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1813,9 +1811,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v11-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.0K, ../../20240327193750.3mlcmzqondpj27xe@liskov/11-v11-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From 3f3bb4559e8a9862fb6c56c8de0c2e4abc69e5c2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v11 10/17] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext(). Now, on the first scan, the the
iterator is created after the scan descriptor is created.
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 163 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 29 +---
src/include/executor/nodeBitmapHeapscan.h | 10 ++
src/include/nodes/execnodes.h | 8 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 196f69e30e..2d9e0e1a9f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2191,10 +2191,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7e73583fe5..fe471a8a0c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,6 +56,56 @@ static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ dsa_pointer shared_area,
+ dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(shared_area))
+ result->parallel = tbm_attach_shared_iterate(personal_area, shared_area);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -97,43 +147,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
+ if (init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -154,21 +184,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +216,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ dsa);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ node->pf_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
+ dsa);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -280,7 +311,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ node->pf_iterator &&
node->pfblockno > node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -321,12 +352,11 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
@@ -335,7 +365,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -348,8 +378,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -371,7 +399,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -431,23 +459,22 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (prefetch_iterator)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
node->prefetch_pages++;
@@ -475,8 +502,6 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
if (prefetch_iterator)
{
while (1)
@@ -500,12 +525,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
@@ -572,18 +597,17 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator)
+ {
+ bhs_end_iterate(node->pf_iterator);
+ node->pf_iterator = NULL;
+ }
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -628,12 +652,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (node->pf_iterator)
+ bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -671,11 +693,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
+ scanstate->pf_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..fb22f305bf 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,12 +20,12 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
+struct BitmapHeapIterator;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -44,8 +44,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ struct BitmapHeapIterator *rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 68478d16b2..459e123e92 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -968,8 +968,7 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
+ result->rs_bhs_iterator = NULL;
return result;
}
@@ -1032,17 +1031,8 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1057,17 +1047,8 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caa..cb56d20dc6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,5 +28,15 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8688bc5ab0..52cedd1b35 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1778,6 +1778,8 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+struct BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1786,12 +1788,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1806,12 +1807,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index cfa9d5aaea..a6562d19a6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v11-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/12-v11-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From 5621e637fd815e6c7fd5ee492678c943cdac82ea Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v11 11/17] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2d9e0e1a9f..81a7488007 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2172,7 +2172,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2322,7 +2323,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index fe471a8a0c..076e1ff674 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -119,7 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -298,14 +297,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 459e123e92..bb2b79717c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -808,8 +808,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -826,8 +826,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2029,15 +2031,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2048,7 +2052,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v11-0012-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240327193750.3mlcmzqondpj27xe@liskov/13-v11-0012-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From f457af280f8d84c43e4f6cf5a0cbd76bb06f5f19 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v11 12/17] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fc..1dc4c99bf9 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639b..432fae5296 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v11-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch (20.7K, ../../20240327193750.3mlcmzqondpj27xe@liskov/14-v11-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch)
download | inline diff:
From 7fa873c6bc40682f58193d8bc00848d5e2f5cb16 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v11 13/17] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 30 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++-------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/executor/nodeBitmapHeapscan.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
8 files changed, 113 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb..3aa457a29e 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544..033d525339 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 81a7488007..f7e4d1094d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2180,7 +2180,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2192,9 +2192,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(scan->rs_bhs_iterator);
+ bhs_iterate(scan->rs_bhs_iterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2209,11 +2209,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2221,19 +2221,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2262,7 +2262,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2271,9 +2271,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2323,7 +2323,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 076e1ff674..78f79aafff 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -77,15 +77,16 @@ bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_ar
return result;
}
-TBMIterateResult *
-bhs_iterate(BitmapHeapIterator *iterator)
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result)
{
Assert(iterator);
+ Assert(result);
if (iterator->serial)
- return tbm_iterate(iterator->serial);
+ tbm_iterate(iterator->serial, result);
else
- return tbm_shared_iterate(iterator->parallel);
+ tbm_shared_iterate(iterator->parallel, result);
}
void
@@ -348,7 +349,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -360,8 +361,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -394,8 +395,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -462,10 +463,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -473,7 +476,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -482,13 +485,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -501,7 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -520,8 +523,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -529,17 +532,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf9..309a44bdb8 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae..3b432263bb 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index cb56d20dc6..3c330f86e6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -34,7 +34,7 @@ typedef struct BitmapHeapIterator
struct TBMSharedIterator *parallel;
} BitmapHeapIterator;
-extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result);
extern void bhs_end_iterate(BitmapHeapIterator *iterator);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae5296..f000c1af28 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v11-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (31.5K, ../../20240327193750.3mlcmzqondpj27xe@liskov/15-v11-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From 5eb00d2b400eb58e532a128946a57f79aef6434d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v11 14/17] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 262 +++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 341 ++--------------------
src/include/access/heapam.h | 17 ++
src/include/access/relscan.h | 8 +
src/include/access/tableam.h | 26 +-
src/include/nodes/execnodes.h | 14 -
7 files changed, 355 insertions(+), 339 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index ed3a3607b7..614d715fc7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,8 +948,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1032,6 +1040,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1044,6 +1058,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1069,6 +1089,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index f7e4d1094d..68bbb6f88c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -61,6 +61,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2170,6 +2173,73 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2188,6 +2258,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2328,6 +2400,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->rs_pf_bhs_iterator &&
+ hscan->pfblockno > hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2338,6 +2422,154 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+
+ if (pstate == NULL)
+ {
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2363,6 +2595,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 78f79aafff..187b288e68 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
dsa_pointer shared_area,
@@ -122,7 +118,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -142,7 +137,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
@@ -154,7 +149,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -169,23 +173,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
@@ -216,19 +213,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
- node->pf_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
- dsa);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
+ scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
+ scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
+ dsa);
}
#endif /* USE_PREFETCH */
@@ -243,36 +240,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -296,23 +263,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->pf_iterator &&
- node->pfblockno > node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,219 +289,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -594,22 +334,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator)
- {
- bhs_end_iterate(node->pf_iterator);
- node->pf_iterator = NULL;
- }
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -648,14 +378,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->pf_iterator)
- bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -688,17 +414,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->pf_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -738,13 +458,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -828,7 +541,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index c7a538221a..4726d31403 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -82,6 +82,23 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used in serial BHS
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fb22f305bf..7938b741d6 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@
struct ParallelTableScanDescData;
struct BitmapHeapIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,6 +46,13 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
+ struct BitmapHeapIterator *rs_pf_bhs_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+ /* used to validate BHS prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index bb2b79717c..799ac013d4 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -811,17 +811,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -971,6 +960,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
+ result->rs_pf_bhs_iterator = NULL;
+ result->prefetch_maximum = 0;
+ result->bm_parallel = NULL;
return result;
}
@@ -1035,6 +1027,12 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1051,6 +1049,12 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 52cedd1b35..60916bf0d0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,18 +1785,11 @@ struct BitmapHeapIterator;
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1804,18 +1797,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v11-0015-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../20240327193750.3mlcmzqondpj27xe@liskov/16-v11-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 7ac2a79503d7110c19a7c335fd5ab5422166b6a6 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v11 15/17] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 614d715fc7..5fc052f019 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -319,6 +319,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 68bbb6f88c..80f210b405 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2167,12 +2167,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2206,8 +2200,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2570,30 +2564,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -2975,7 +2982,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index ce637a5a5d..1d6b03d1ca 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -92,8 +92,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 187b288e68..2f9387e51a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -230,44 +230,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
#endif /* USE_PREFETCH */
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 799ac013d4..5979ddee8b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -799,36 +799,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -2033,44 +2017,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2081,7 +2034,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v11-0016-v10-Read-Stream-API.patch (71.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/17-v11-0016-v10-Read-Stream-API.patch)
download | inline diff:
From 88dde25e1b9a9b340ae57161f098b6351d538f3c Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Mon, 26 Feb 2024 23:48:31 +1300
Subject: [PATCH v11 16/17] v10 Read Stream API
Part 1:
Provide vectored variant of ReadBuffer().
Break ReadBuffer() up into two steps: StartReadBuffers() and
WaitReadBuffers(). This has two advantages:
1. Multiple consecutive blocks can be read with one system call.
2. Advice (hints of future reads) can optionally be issued to the kernel.
The traditional ReadBuffer() function is now implemented in terms of
those functions, to avoid duplication. For now we still only read a
block at a time so there is no change to generated system calls yet, but
later commits will provide infrastructure to help build up larger calls.
Callers should respect the new GUC io_combine_limit, and the limit on
per-backend pins which is now exposed as a public interface.
With some more infrastructure in later work, StartReadBuffers() could
be extended to start real asynchronous I/O instead of advice.
Part 2:
Provide API for streaming relation data.
Introduce an abstraction where relation data can be accessed as a
stream of buffers, with an implementation that is more efficient than
the equivalent sequence of ReadBuffer() calls.
Client code supplies a callback that can say which block number is
wanted next, and then consumes individual buffers one at a time from the
stream. This division allows read_stream.c to build up large calls to
StartReadBuffers() up to io_combine_limit, and issue fadvise() advice
ahead of time in a systematic way when random access is detected.
This API is based on an idea from Andres Freund to pave the way for
asynchronous I/O in future work as required to support direct I/O. The
goal is to have an abstraction that insulates client code from future
changes to the I/O subsystem.
An extended API may be necessary in future for more complicated cases
(for example recovery, whose LsnReadQueue device in xlogprefetcher.c is
a distant cousin of this code that should eventually be replaced by it),
but this basic API is sufficient for many common usage patterns
involving predictable access to a single relation fork.
Author: Thomas Munro <thomas.munro@gmail.com>
Author: Andres Freund <andres@anarazel.de> (optimization tweaks)
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJkOiOCa+mag4BF+zHo7qo=o9CFheB8=g6uT5TUm2gkvA@mail.gmail.com
---
doc/src/sgml/config.sgml | 14 +
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/read_stream.c | 733 ++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 709 +++++++++++------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 14 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/storage/bufmgr.h | 41 +-
src/include/storage/read_stream.h | 62 ++
src/tools/pgindent/typedefs.list | 3 +
13 files changed, 1386 insertions(+), 227 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/read_stream.c
create mode 100644 src/include/storage/read_stream.h
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 5468637e2e..f3736000ad 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2719,6 +2719,20 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-io-combine-limit" xreflabel="io_combine_limit">
+ <term><varname>io_combine_limit</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>io_combine_limit</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Controls the largest I/O size in operations that combine I/O.
+ The default is 128kB.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-max-worker-processes" xreflabel="max_worker_processes">
<term><varname>max_worker_processes</varname> (<type>integer</type>)
<indexterm>
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca2..eec03f6f2b 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 0000000000..2f29a9ec4d
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ read_stream.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 0000000000..10e1aa3b20
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'read_stream.c',
+)
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
new file mode 100644
index 0000000000..4e293e0df6
--- /dev/null
+++ b/src/backend/storage/aio/read_stream.c
@@ -0,0 +1,733 @@
+/*-------------------------------------------------------------------------
+ *
+ * read_stream.c
+ * Mechanism for accessing buffered relation data with look-ahead
+ *
+ * Code that needs to access relation data typically pins blocks one at a
+ * time, often in a predictable order that might be sequential or data-driven.
+ * Calling the simple ReadBuffer() function for each block is inefficient,
+ * because blocks that are not yet in the buffer pool require I/O operations
+ * that are small and might stall waiting for storage. This mechanism looks
+ * into the future and calls StartReadBuffers() and WaitReadBuffers() to read
+ * neighboring blocks together and ahead of time, with an adaptive look-ahead
+ * distance.
+ *
+ * A user-provided callback generates a stream of block numbers that is used
+ * to form reads of up to io_combine_limit, by attempting to merge them with a
+ * pending read. When that isn't possible, the existing pending read is sent
+ * to StartReadBuffers() so that a new one can begin to form.
+ *
+ * The algorithm for controlling the look-ahead distance tries to classify the
+ * stream into three ideal behaviors:
+ *
+ * A) No I/O is necessary, because the requested blocks are fully cached
+ * already. There is no benefit to looking ahead more than one block, so
+ * distance is 1. This is the default initial assumption.
+ *
+ * B) I/O is necessary, but fadvise is undesirable because the access is
+ * sequential, or impossible because direct I/O is enabled or the system
+ * doesn't support advice. There is no benefit in looking ahead more than
+ * io_combine_limit, because in this case only goal is larger read system
+ * calls. Looking further ahead would pin many buffers and perform
+ * speculative work looking ahead for no benefit.
+ *
+ * C) I/O is necesssary, it appears random, and this system supports fadvise.
+ * We'll look further ahead in order to reach the configured level of I/O
+ * concurrency.
+ *
+ * The distance increases rapidly and decays slowly, so that it moves towards
+ * those levels as different I/O patterns are discovered. For example, a
+ * sequential scan of fully cached data doesn't bother looking ahead, but a
+ * sequential scan that hits a region of uncached blocks will start issuing
+ * increasingly wide read calls until it plateaus at io_combine_limit.
+ *
+ * The main data structure is a circular queue of buffers of size
+ * max_pinned_buffers plus some extra space for technical reasons, ready to be
+ * returned by read_stream_next_buffer(). Each buffer also has an optional
+ * variable sized object that is passed from the callback to the consumer of
+ * buffers.
+ *
+ * Parallel to the queue of buffers, there is a circular queue of in-progress
+ * I/Os that have been started with StartReadBuffers(), and for which
+ * WaitReadBuffers() must be called before returning the buffer.
+ *
+ * For example, if the callback return block numbers 10, 42, 43, 60 in
+ * successive calls, then these data structures might appear as follows:
+ *
+ * buffers buf/data ios
+ *
+ * +----+ +-----+ +--------+
+ * | | | | +----+ 42..44 | <- oldest_io_index
+ * +----+ +-----+ | +--------+
+ * oldest_buffer_index -> | 10 | | ? | | +--+ 60..60 |
+ * +----+ +-----+ | | +--------+
+ * | 42 | | ? |<-+ | | | <- next_io_index
+ * +----+ +-----+ | +--------+
+ * | 43 | | ? | | | |
+ * +----+ +-----+ | +--------+
+ * | 44 | | ? | | | |
+ * +----+ +-----+ | +--------+
+ * | 60 | | ? |<---+
+ * +----+ +-----+
+ * next_buffer_index -> | | | |
+ * +----+ +-----+
+ *
+ * In the example, 5 buffers are pinned, and the next buffer to be streamed to
+ * the client is block 10. Block 10 was a hit and has no associated I/O, but
+ * the range 42..44 requires an I/O wait before its buffers are returned, as
+ * does block 60.
+ *
+ *
+ * Portions Copyright (c) 2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/storage/aio/read_stream.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+#include "storage/read_stream.h"
+#include "utils/memdebug.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+typedef struct InProgressIO
+{
+ int16 buffer_index;
+ ReadBuffersOperation op;
+} InProgressIO;
+
+/*
+ * State for managing a stream of reads.
+ */
+struct ReadStream
+{
+ int16 max_ios;
+ int16 ios_in_progress;
+ int16 queue_size;
+ int16 max_pinned_buffers;
+ int16 pinned_buffers;
+ int16 distance;
+ bool advice_enabled;
+
+ /*
+ * Sometimes we need to be able to 'unget' a block number to resolve a
+ * flow control problem when I/Os are split.
+ */
+ BlockNumber unget_blocknum;
+ bool have_unget_blocknum;
+
+ /*
+ * The callback that will tell us which block numbers to read, and an
+ * opaque pointer that will be pass to it for its own purposes.
+ */
+ ReadStreamBlockNumberCB callback;
+ void *callback_private_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* The read operation we are currently preparing. */
+ BlockNumber pending_read_blocknum;
+ int16 pending_read_nblocks;
+
+ /* Space for buffers and optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Read operations that have been started but not waited for yet. */
+ InProgressIO *ios;
+ int16 oldest_io_index;
+ int16 next_io_index;
+
+ /* Circular queue of buffers. */
+ int16 oldest_buffer_index; /* Next pinned buffer to return */
+ int16 next_buffer_index; /* Index of next buffer to pin */
+ Buffer buffers[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static inline void *
+get_per_buffer_data(ReadStream *stream, int16 buffer_index)
+{
+ return (char *) stream->per_buffer_data +
+ stream->per_buffer_data_size * buffer_index;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow streaming_unget_block() to work.
+ */
+static inline BlockNumber
+read_stream_get_block(ReadStream *stream, void *per_buffer_data)
+{
+ if (!stream->have_unget_blocknum)
+ return stream->callback(stream,
+ stream->callback_private_data,
+ per_buffer_data);
+
+ /*
+ * You can only unget one block, and next_buffer_index can't change across
+ * a get, unget, get sequence, so the callback's per_buffer_data, if any,
+ * is still present in the correct slot. We just have to return the
+ * previous block number.
+ */
+ stream->have_unget_blocknum = false;
+ return stream->unget_blocknum;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later.
+ */
+static inline void
+read_stream_unget_block(ReadStream *stream, BlockNumber blocknum)
+{
+ Assert(!stream->have_unget_blocknum);
+ stream->have_unget_blocknum = true;
+ stream->unget_blocknum = blocknum;
+}
+
+static void
+read_stream_start_pending_read(ReadStream *stream, bool suppress_advice)
+{
+ bool need_wait;
+ int nblocks;
+ int flags;
+ int16 io_index;
+ int16 overflow;
+ int16 buffer_index;
+
+ /* This should only be called with a pending read. */
+ Assert(stream->pending_read_nblocks > 0);
+ Assert(stream->pending_read_nblocks <= io_combine_limit);
+
+ /* We had better not exceed the pin limit by starting this read. */
+ Assert(stream->pinned_buffers + stream->pending_read_nblocks <=
+ stream->max_pinned_buffers);
+
+ /* We had better not be overwriting an existing pinned buffer. */
+ if (stream->pinned_buffers > 0)
+ Assert(stream->next_buffer_index != stream->oldest_buffer_index);
+ else
+ Assert(stream->next_buffer_index == stream->oldest_buffer_index);
+
+ /*
+ * If advice hasn't been suppressed, this system supports it, and this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (!suppress_advice &&
+ stream->advice_enabled &&
+ stream->pending_read_blocknum != stream->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* We say how many blocks we want to read, but may be smaller on return. */
+ buffer_index = stream->next_buffer_index;
+ io_index = stream->next_io_index;
+ nblocks = stream->pending_read_nblocks;
+ need_wait = StartReadBuffers(&stream->ios[io_index].op,
+ &stream->buffers[buffer_index],
+ stream->pending_read_blocknum,
+ &nblocks,
+ flags);
+ stream->pinned_buffers += nblocks;
+
+ /* Remember whether we need to wait before returning this buffer. */
+ if (!need_wait)
+ {
+ /* Look-ahead distance decays, no I/O necessary (behavior A). */
+ if (stream->distance > 1)
+ stream->distance--;
+ }
+ else
+ {
+ /*
+ * Remember to call WaitReadBuffers() before returning head buffer.
+ * Look-ahead distance will be adjusted after waiting.
+ */
+ stream->ios[io_index].buffer_index = buffer_index;
+ if (++stream->next_io_index == stream->max_ios)
+ stream->next_io_index = 0;
+ Assert(stream->ios_in_progress < stream->max_ios);
+ stream->ios_in_progress++;
+ stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+ }
+
+ /*
+ * We gave a contiguous range of buffer space to StartReadBuffers(), but
+ * we want it to wrap around at queue_size. Slide overflowing buffers to
+ * the front of the array.
+ */
+ overflow = (buffer_index + nblocks) - stream->queue_size;
+ if (overflow > 0)
+ memmove(&stream->buffers[0],
+ &stream->buffers[stream->queue_size],
+ sizeof(stream->buffers[0]) * overflow);
+
+ /* Compute location of start of next read, without using % operator. */
+ buffer_index += nblocks;
+ if (buffer_index >= stream->queue_size)
+ buffer_index -= stream->queue_size;
+ Assert(buffer_index >= 0 && buffer_index < stream->queue_size);
+ stream->next_buffer_index = buffer_index;
+
+ /* Adjust the pending read to cover the remaining portion, if any. */
+ stream->pending_read_blocknum += nblocks;
+ stream->pending_read_nblocks -= nblocks;
+}
+
+static void
+read_stream_look_ahead(ReadStream *stream, bool suppress_advice)
+{
+ while (stream->ios_in_progress < stream->max_ios &&
+ stream->pinned_buffers + stream->pending_read_nblocks < stream->distance)
+ {
+ BlockNumber blocknum;
+ int16 buffer_index;
+ void *per_buffer_data;
+
+ if (stream->pending_read_nblocks == io_combine_limit)
+ {
+ read_stream_start_pending_read(stream, suppress_advice);
+ suppress_advice = false;
+ continue;
+ }
+
+ /*
+ * See which block the callback wants next in the stream. We need to
+ * compute the index of the Nth block of the pending read including
+ * wrap-around, but we don't want to use the expensive % operator.
+ */
+ buffer_index = stream->next_buffer_index + stream->pending_read_nblocks;
+ if (buffer_index >= stream->queue_size)
+ buffer_index -= stream->queue_size;
+ Assert(buffer_index >= 0 && buffer_index < stream->queue_size);
+ per_buffer_data = get_per_buffer_data(stream, buffer_index);
+ blocknum = read_stream_get_block(stream, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ stream->distance = 0;
+ break;
+ }
+
+ /* Can we merge it with the pending read? */
+ if (stream->pending_read_nblocks > 0 &&
+ stream->pending_read_blocknum + stream->pending_read_nblocks == blocknum)
+ {
+ stream->pending_read_nblocks++;
+ continue;
+ }
+
+ /* We have to start the pending read before we can build another. */
+ if (stream->pending_read_nblocks > 0)
+ {
+ read_stream_start_pending_read(stream, suppress_advice);
+ suppress_advice = false;
+ if (stream->ios_in_progress == stream->max_ios)
+ {
+ /* And we've hit the limit. Rewind, and stop here. */
+ read_stream_unget_block(stream, blocknum);
+ return;
+ }
+ }
+
+ /* This is the start of a new pending read. */
+ stream->pending_read_blocknum = blocknum;
+ stream->pending_read_nblocks = 1;
+ }
+
+ /*
+ * Normally we don't start the pending read just because we've hit a
+ * limit, preferring to give it another chance to grow to a larger size
+ * once more buffers have been consumed. However, in cases where that
+ * can't possibly happen, we might as well start the read immediately.
+ */
+ if (stream->pending_read_nblocks > 0 &&
+ (stream->distance == stream->pending_read_nblocks ||
+ stream->distance == 0) &&
+ stream->ios_in_progress < stream->max_ios)
+ read_stream_start_pending_read(stream, suppress_advice);
+}
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead. The callback should return block numbers or InvalidBlockNumber to
+ * signal end-of-stream, and if per_buffer_data_size is non-zero, it may also
+ * write extra data for each block into the space provided to it. It will
+ * also receive callback_private_data for its own purposes.
+ */
+ReadStream *
+read_stream_begin_relation(int flags,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ ReadStreamBlockNumberCB callback,
+ void *callback_private_data,
+ size_t per_buffer_data_size)
+{
+ ReadStream *stream;
+ size_t size;
+ int16 queue_size;
+ int16 max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /* Make sure our bmr's smgr and persistent are populated. */
+ if (bmr.smgr == NULL)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many I/Os we will allow to run at the same time. That
+ * currently means advice to the kernel to tell it that we will soon read.
+ * This number also affects how far we look ahead for opportunities to
+ * start more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & READ_STREAM_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+ max_ios = Min(max_ios, PG_INT16_MAX);
+
+ /*
+ * Choose the maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least io_combine_limit
+ * so that we can have a chance to build up a full io_combine_limit sized
+ * read, even when max_ios is zero. Be careful not to allow int16 to
+ * overflow (even though that's not possible with the current GUC range
+ * limits), allowing also for the spare entry and the overflow space.
+ */
+ max_pinned_buffers = Max(max_ios * 4, io_combine_limit);
+ max_pinned_buffers = Min(max_pinned_buffers,
+ PG_INT16_MAX - io_combine_limit - 1);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * We need one extra entry for buffers and per-buffer data, because users
+ * of per-buffer data have access to the object until the next call to
+ * read_stream_next_buffer(), so we need a gap between the head and tail
+ * of the queue so that we don't clobber it.
+ */
+ queue_size = max_pinned_buffers + 1;
+
+ /*
+ * Allocate the object, the buffers, the ios and per_data_data space in
+ * one big chunk. Though we have queue_size buffers, we want to be able
+ * to assume that all the buffers for a single read are contiguous (i.e.
+ * don't wrap around halfway through), so we allow temporary overflows of
+ * up to the maximum possible read size by allocating an extra
+ * io_combine_limit - 1 elements.
+ */
+ size = offsetof(ReadStream, buffers);
+ size += sizeof(Buffer) * (queue_size + io_combine_limit - 1);
+ size += sizeof(InProgressIO) * Max(1, max_ios);
+ size += per_buffer_data_size * queue_size;
+ size += MAXIMUM_ALIGNOF * 2;
+ stream = (ReadStream *) palloc(size);
+ memset(stream, 0, offsetof(ReadStream, buffers));
+ stream->ios = (InProgressIO *)
+ MAXALIGN(&stream->buffers[queue_size + io_combine_limit - 1]);
+ if (per_buffer_data_size > 0)
+ stream->per_buffer_data = (void *)
+ MAXALIGN(&stream->ios[Max(1, max_ios)]);
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. We can use it as long as
+ * direct I/O isn't enabled, the caller hasn't promised sequential access
+ * (overriding our detection heuristics), and max_ios hasn't been set to
+ * zero.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & READ_STREAM_SEQUENTIAL) == 0 &&
+ max_ios > 0)
+ stream->advice_enabled = true;
+#endif
+
+ /*
+ * For now, max_ios = 0 is interpreted as max_ios = 1 with advice disabled
+ * above. If we had real asynchronous I/O we might need a slightly
+ * different definition.
+ */
+ if (max_ios == 0)
+ max_ios = 1;
+
+ stream->max_ios = max_ios;
+ stream->per_buffer_data_size = per_buffer_data_size;
+ stream->max_pinned_buffers = max_pinned_buffers;
+ stream->queue_size = queue_size;
+
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+ stream->callback = callback;
+ stream->callback_private_data = callback_private_data;
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out assuming we'll be
+ * doing full io_combine_limit sized reads (behavior B).
+ */
+ if (flags & READ_STREAM_FULL)
+ stream->distance = Min(max_pinned_buffers, io_combine_limit);
+ else
+ stream->distance = 1;
+
+ /*
+ * Since we always currently always access the same relation, we can
+ * initialize parts of the ReadBuffersOperation objects and leave them
+ * that way, to avoid wasting CPU cycles writing to them for each read.
+ */
+ for (int i = 0; i < max_ios; ++i)
+ {
+ stream->ios[i].op.bmr = bmr;
+ stream->ios[i].op.forknum = forknum;
+ stream->ios[i].op.strategy = strategy;
+ }
+
+ return stream;
+}
+
+/*
+ * Pull one pinned buffer out of a stream created with
+ * read_stream_begin_buffered(). Each call returns successive blocks in the
+ * order specified by the callback. If per_buffer_data_size was set to a
+ * non-zero size, *per_buffer_data receives a pointer to the extra per-buffer
+ * data that the callback had a chance to populate, which remains valid until
+ * the next call to read_stream_next_buffer(). When the stream runs out of
+ * data, InvalidBuffer is returned. The caller may decide to end the stream
+ * early at any time by calling read_stream_end_buffered().
+ */
+Buffer
+read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
+{
+ Buffer buffer;
+ int16 oldest_buffer_index;
+
+ /*
+ * A fast path for all-cached scans (behavior A). This is the same as the
+ * usual algorithm, but it is specialized for no I/O and no per-buffer
+ * data, so we can skip the queue management code, stay in the same buffer
+ * slot and use singular StartReadBuffer().
+ */
+ if (likely(per_buffer_data == NULL &&
+ stream->ios_in_progress == 0 &&
+ stream->pinned_buffers == 1 &&
+ stream->distance == 1))
+ {
+ BlockNumber next_blocknum;
+
+ /*
+ * We have a pinned buffer that we need to serve up, but we also want
+ * to probe the next one before we return, just in case we need to
+ * start an I/O. We can re-use the same buffer slot, and an arbitrary
+ * I/O slot since they're all free.
+ */
+ oldest_buffer_index = stream->oldest_buffer_index;
+ Assert((oldest_buffer_index + 1) % stream->queue_size ==
+ stream->next_buffer_index);
+ buffer = stream->buffers[oldest_buffer_index];
+ Assert(buffer != InvalidBuffer);
+ Assert(stream->pending_read_nblocks <= 1);
+ if (unlikely(stream->pending_read_nblocks == 1))
+ {
+ next_blocknum = stream->pending_read_blocknum;
+ stream->pending_read_nblocks = 0;
+ }
+ else
+ next_blocknum = read_stream_get_block(stream, NULL);
+ if (unlikely(next_blocknum == InvalidBlockNumber))
+ {
+ /* End of stream. */
+ stream->distance = 0;
+ stream->next_buffer_index = oldest_buffer_index;
+ /* Pin transferred to caller. */
+ stream->pinned_buffers = 0;
+ return buffer;
+ }
+ /* Call the special single block version, which is marginally faster. */
+ if (unlikely(StartReadBuffer(&stream->ios[0].op,
+ &stream->buffers[oldest_buffer_index],
+ next_blocknum,
+ stream->advice_enabled ?
+ READ_BUFFERS_ISSUE_ADVICE : 0)))
+ {
+ /* I/O needed. We'll take the general path next time. */
+ stream->oldest_io_index = 0;
+ stream->next_io_index = stream->max_ios > 1 ? 1 : 0;
+ stream->ios_in_progress = 1;
+ stream->ios[0].buffer_index = oldest_buffer_index;
+ stream->seq_blocknum = next_blocknum + 1;
+ /* Increase look ahead distance (move towards behavior B/C). */
+ stream->distance = Min(2, stream->max_pinned_buffers);
+ }
+ /* Pin transferred to caller, got another one, no net change. */
+ Assert(stream->pinned_buffers == 1);
+ return buffer;
+ }
+
+ if (stream->pinned_buffers == 0)
+ {
+ Assert(stream->oldest_buffer_index == stream->next_buffer_index);
+
+ /* End of stream reached? */
+ if (stream->distance == 0)
+ return InvalidBuffer;
+
+ /*
+ * The usual order of operations is that we look ahead at the bottom
+ * of this function after potentially finishing an I/O and making
+ * space for more, but if we're just starting up we'll need to crank
+ * the handle to get started.
+ */
+ read_stream_look_ahead(stream, true);
+
+ /* End of stream reached? */
+ if (stream->pinned_buffers == 0)
+ {
+ Assert(stream->distance == 0);
+ return InvalidBuffer;
+ }
+ }
+
+ /* Grab the oldest pinned buffer and associated per-buffer data. */
+ Assert(stream->pinned_buffers > 0);
+ oldest_buffer_index = stream->oldest_buffer_index;
+ Assert(oldest_buffer_index >= 0 &&
+ oldest_buffer_index < stream->queue_size);
+ buffer = stream->buffers[oldest_buffer_index];
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(stream, oldest_buffer_index);
+
+ Assert(BufferIsValid(buffer));
+
+ /* Do we have to wait for an associated I/O first? */
+ if (stream->ios_in_progress > 0 &&
+ stream->ios[stream->oldest_io_index].buffer_index == oldest_buffer_index)
+ {
+ int16 io_index = stream->oldest_io_index;
+ int16 distance;
+
+ /* Sanity check that we still agree on the buffers. */
+ Assert(stream->ios[io_index].op.buffers ==
+ &stream->buffers[oldest_buffer_index]);
+
+ WaitReadBuffers(&stream->ios[io_index].op);
+
+ Assert(stream->ios_in_progress > 0);
+ stream->ios_in_progress--;
+ if (++stream->oldest_io_index == stream->max_ios)
+ stream->oldest_io_index = 0;
+
+ if (stream->ios[io_index].op.flags & READ_BUFFERS_ISSUE_ADVICE)
+ {
+ /* Distance ramps up fast (behavior C). */
+ distance = stream->distance * 2;
+ distance = Min(distance, stream->max_pinned_buffers);
+ stream->distance = distance;
+ }
+ else
+ {
+ /* No advice; move towards io_combine_limit (behavior B). */
+ if (stream->distance > io_combine_limit)
+ {
+ stream->distance--;
+ }
+ else
+ {
+ distance = stream->distance * 2;
+ distance = Min(distance, io_combine_limit);
+ distance = Min(distance, stream->max_pinned_buffers);
+ stream->distance = distance;
+ }
+ }
+ }
+
+#ifdef CLOBBER_FREED_MEMORY
+ /* Clobber old buffer and per-buffer data for debugging purposes. */
+ stream->buffers[oldest_buffer_index] = InvalidBuffer;
+
+ /*
+ * The caller will get access to the per-buffer data, until the next call.
+ * We wipe the one before, which is never occupied because queue_size
+ * allowed one extra element. This will hopefully trip up client code
+ * that is holding a dangling pointer to it.
+ */
+ if (stream->per_buffer_data)
+ wipe_mem(get_per_buffer_data(stream,
+ oldest_buffer_index == 0 ?
+ stream->queue_size - 1 :
+ oldest_buffer_index - 1),
+ stream->per_buffer_data_size);
+#endif
+
+ /* Pin transferred to caller. */
+ Assert(stream->pinned_buffers > 0);
+ stream->pinned_buffers--;
+
+ /* Advance oldest buffer, with wrap-around. */
+ stream->oldest_buffer_index++;
+ if (stream->oldest_buffer_index == stream->queue_size)
+ stream->oldest_buffer_index = 0;
+
+ /* Prepare for the next call. */
+ read_stream_look_ahead(stream, false);
+
+ return buffer;
+}
+
+/*
+ * Release stream resources.
+ */
+void
+read_stream_end(ReadStream *stream)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ stream->distance = 0;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(stream->pinned_buffers == 0);
+ Assert(stream->ios_in_progress == 0);
+
+ /* Release memory. */
+ pfree(stream);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c..577bcf6e5d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -160,6 +165,9 @@ int checkpoint_flush_after = DEFAULT_CHECKPOINT_FLUSH_AFTER;
int bgwriter_flush_after = DEFAULT_BGWRITER_FLUSH_AFTER;
int backend_flush_after = DEFAULT_BACKEND_FLUSH_AFTER;
+/* Limit on how many blocks should be handled in single I/O operations. */
+int io_combine_limit = DEFAULT_IO_COMBINE_LIMIT;
+
/* local state for LockBufferForCleanup */
static BufferDesc *PinCountWaitBuf = NULL;
@@ -471,10 +479,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +507,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +788,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +800,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +822,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +993,146 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
/*
- * ReadBuffer_common -- common logic for all ReadBuffer variants
- *
- * *hit is set to true if the request was satisfied from shared buffer cache.
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
*/
-static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
- BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
{
BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
+/*
+ * Pin a buffer for a given block. *foundPtr is set to true if the block was
+ * already present, or false if more work is required to either read it in or
+ * zero it.
+ */
+static inline Buffer
+PinBufferForBlock(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
IOContext io_context;
IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
- *hit = false;
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = bmr.relpersistence == RELPERSISTENCE_TEMP;
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(strategy);
+ io_object = IOOBJECT_RELATION;
+ }
+
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
+
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
+ {
+ VacuumPageHit++;
+ pgstat_count_io_op(io_object, io_context, IOOP_HIT);
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageHit;
+
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
+
+ return BufferDescriptorGetBuffer(bufHdr);
+}
+
+/*
+ * ReadBuffer_common -- common logic for all ReadBuffer variants
+ */
+static inline Buffer
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
+ BlockNumber blockNum, ReadBufferMode mode,
+ BufferAccessStrategy strategy)
+{
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1151,359 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
-
- if (isLocalBuf)
+ if (unlikely(mode == RBM_ZERO_AND_CLEANUP_LOCK ||
+ mode == RBM_ZERO_AND_LOCK))
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
- io_context = IOCONTEXT_NORMAL;
- io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
+ bool found;
+
+ if (bmr.smgr == NULL)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ buffer = PinBufferForBlock(bmr, forkNum, blockNum, strategy, &found);
+ ZeroBuffer(buffer, mode);
+ return buffer;
}
+
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
else
+ flags = 0;
+ operation.bmr = bmr;
+ operation.forknum = forkNum;
+ operation.strategy = strategy;
+ if (StartReadBuffer(&operation,
+ &buffer,
+ blockNum,
+ flags))
+ WaitReadBuffers(&operation);
+
+ return buffer;
+}
+
+/*
+ * Single block version of the StartReadBuffers(). This might save a few
+ * instructions when called from another translation unit, if the compiler
+ * inlines the code and specializes for nblocks == 1.
+ */
+bool
+StartReadBuffer(ReadBuffersOperation *operation,
+ Buffer *buffer,
+ BlockNumber blocknum,
+ int flags)
+{
+ int nblocks = 1;
+ bool result;
+
+ result = StartReadBuffers(operation, buffer, blocknum, &nblocks, flags);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ return result;
+}
+
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested. Caller sets some of the
+ * members of operation; see struct definition.
+ *
+ * If false is returned, no I/O is necessary. If true is returned, one I/O
+ * has been started, and WaitReadBuffers() must be called with the same
+ * operation object before the buffers are accessed. Along with the operation
+ * object, the caller-supplied array of buffers must remain valid until
+ * WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+inline bool
+StartReadBuffers(ReadBuffersOperation *operation,
+ Buffer *buffers,
+ BlockNumber blockNum,
+ int *nblocks,
+ int flags)
+{
+ int actual_nblocks = *nblocks;
+ int io_buffers_len = 0;
+
+ Assert(*nblocks > 0);
+ Assert(*nblocks <= MAX_IO_COMBINE_LIMIT);
+
+ if (!operation->bmr.smgr)
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
- io_context = IOContextForStrategy(strategy);
- io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
+ operation->bmr.smgr = RelationGetSmgr(operation->bmr.rel);
+ operation->bmr.relpersistence = operation->bmr.rel->rd_rel->relpersistence;
}
- /* At this point we do NOT hold any locks. */
-
- /* if it was already in the buffer pool, we're done */
- if (found)
+ for (int i = 0; i < actual_nblocks; ++i)
{
- /* Just need to update stats before we exit */
- *hit = true;
- VacuumPageHit++;
- pgstat_count_io_op(io_object, io_context, IOOP_HIT);
+ bool found;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageHit;
+ buffers[i] = PinBufferForBlock(operation->bmr,
+ operation->forknum,
+ blockNum + i,
+ operation->strategy,
+ &found);
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ io_buffers_len++;
+ }
+ }
+ *nblocks = actual_nblocks;
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
+ if (io_buffers_len > 0)
+ {
+ /* Populate information needed for I/O. */
+ operation->buffers = buffers;
+ operation->blocknum = blockNum;
+ operation->flags = flags;
+ operation->nblocks = actual_nblocks;
+ operation->io_buffers_len = io_buffers_len;
+
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
+ /*
+ * In theory we should only do this if PinBufferForBlock() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(operation->bmr.smgr,
+ operation->forknum,
+ blockNum,
+ operation->io_buffers_len);
}
- return BufferDescriptorGetBuffer(bufHdr);
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
+ }
+ else
+ {
+ return false;
}
+}
+
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
+ {
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
+ }
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
/*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
*/
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+
+ isLocalBuf = operation->bmr.relpersistence == RELPERSISTENCE_TEMP;
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
+ }
/*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
*/
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
else
+ pgBufferUsage.shared_blks_read += nblocks;
+
+ for (int i = 0; i < nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ int io_buffers_len;
+ Buffer io_buffers[MAX_IO_COMBINE_LIMIT];
+ void *io_pages[MAX_IO_COMBINE_LIMIT];
+ instr_time io_start;
+ BlockNumber io_first_block;
+
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PinBufferForBlock().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ operation->bmr.smgr->smgr_rlocator.locator.spcOid,
+ operation->bmr.smgr->smgr_rlocator.locator.dbOid,
+ operation->bmr.smgr->smgr_rlocator.locator.relNumber,
+ operation->bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(operation->bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
{
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
}
else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- }
- }
-
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
- {
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
- }
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
- if (isLocalBuf)
- {
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(operation->bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(operation->bmr.smgr->smgr_rlocator, forknum))));
+ }
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
- }
- else
- {
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
- }
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ operation->bmr.smgr->smgr_rlocator.locator.spcOid,
+ operation->bmr.smgr->smgr_rlocator.locator.dbOid,
+ operation->bmr.smgr->smgr_rlocator.locator.relNumber,
+ operation->bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
- return BufferDescriptorGetBuffer(bufHdr);
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for PinBufferForBlock. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1511,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1570,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1638,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1673,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +2029,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2294,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2317,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2193,7 +2453,7 @@ MarkBufferDirty(Buffer buffer)
uint32 old_buf_state;
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
{
@@ -2372,7 +2632,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2646,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3714,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -4560,7 +4825,7 @@ void
ReleaseBuffer(Buffer buffer)
{
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
UnpinLocalBuffer(buffer);
@@ -4627,7 +4892,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
Page page = BufferGetPage(buffer);
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
{
@@ -5184,9 +5449,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5470,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
@@ -5691,7 +5964,7 @@ ResOwnerReleaseBufferPin(Datum res)
/* Like ReleaseBuffer, but don't call ResourceOwnerForgetBuffer */
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
UnpinLocalBufferNoOwner(buffer);
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a5..985a2c7049 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca2..739d13293f 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index abd9029451..313e393262 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3112,6 +3112,20 @@ struct config_int ConfigureNamesInt[] =
NULL
},
+ {
+ {"io_combine_limit",
+ PGC_USERSET,
+ RESOURCES_ASYNCHRONOUS,
+ gettext_noop("Limit on the size of data reads and writes."),
+ NULL,
+ GUC_UNIT_BLOCKS
+ },
+ &io_combine_limit,
+ DEFAULT_IO_COMBINE_LIMIT,
+ 1, MAX_IO_COMBINE_LIMIT,
+ NULL, NULL, NULL
+ },
+
{
{"backend_flush_after", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 2244ee52f7..7fa6d5a64c 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -203,6 +203,7 @@
#backend_flush_after = 0 # measured in pages, 0 disables
#effective_io_concurrency = 1 # 1-1000; 0 disables prefetching
#maintenance_io_concurrency = 10 # 1-1000; 0 disables prefetching
+#io_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
#max_worker_processes = 8 # (change requires restart)
#max_parallel_workers_per_gather = 2 # limited by max_parallel_workers
#max_parallel_maintenance_workers = 2 # limited by max_parallel_workers
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d335..241f68c45e 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -133,6 +134,10 @@ extern PGDLLIMPORT bool track_io_timing;
extern PGDLLIMPORT int effective_io_concurrency;
extern PGDLLIMPORT int maintenance_io_concurrency;
+#define MAX_IO_COMBINE_LIMIT PG_IOV_MAX
+#define DEFAULT_IO_COMBINE_LIMIT Min(MAX_IO_COMBINE_LIMIT, (128 * 1024) / BLCKSZ)
+extern PGDLLIMPORT int io_combine_limit;
+
extern PGDLLIMPORT int checkpoint_flush_after;
extern PGDLLIMPORT int backend_flush_after;
extern PGDLLIMPORT int bgwriter_flush_after;
@@ -158,7 +163,6 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
-
/*
* prototypes for functions in bufmgr.c
*/
@@ -177,6 +181,38 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+struct ReadBuffersOperation
+{
+ /* The following members should be set by the caller. */
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+ BufferAccessStrategy strategy;
+
+ /* The following private members should not be accessed directly. */
+ Buffer *buffers;
+ BlockNumber blocknum;
+ int flags;
+ int16 nblocks;
+ int16 io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffer(ReadBuffersOperation *operation,
+ Buffer *buffer,
+ BlockNumber blocknum,
+ int flags);
+extern bool StartReadBuffers(ReadBuffersOperation *operation,
+ Buffer *buffers,
+ BlockNumber blocknum,
+ int *nblocks,
+ int flags);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +286,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
new file mode 100644
index 0000000000..9e5fa2acf1
--- /dev/null
+++ b/src/include/storage/read_stream.h
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * read_stream.h
+ * Mechanism for accessing buffered relation data with look-ahead
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/read_stream.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef READ_STREAM_H
+#define READ_STREAM_H
+
+#include "storage/bufmgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define READ_STREAM_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users, and thus should be governed by maintenance_io_concurrency
+ * instead of effective_io_concurrency. For example, VACUUM or CREATE INDEX.
+ */
+#define READ_STREAM_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define READ_STREAM_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define READ_STREAM_FULL 0x04
+
+struct ReadStream;
+typedef struct ReadStream ReadStream;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*ReadStreamBlockNumberCB) (ReadStream *stream,
+ void *callback_private_data,
+ void *per_buffer_data);
+
+extern ReadStream *read_stream_begin_relation(int flags,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ ReadStreamBlockNumberCB callback,
+ void *callback_private_data,
+ size_t per_buffer_data_size);
+extern Buffer read_stream_next_buffer(ReadStream *stream, void **per_buffer_private);
+extern void read_stream_end(ReadStream *stream);
+
+#endif /* READ_STREAM_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a6562d19a6..16bfdf3444 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1215,6 +1215,7 @@ InjectionPointCacheEntry
InjectionPointEntry
InjectionPointSharedState
InlineCodeBlock
+InProgressIO
InsertStmt
Instrumentation
Int128AggState
@@ -2287,11 +2288,13 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
ReadLocalXLogPageNoWaitPrivate
ReadReplicationSlotCmd
+ReadStream
ReassignOwnedStmt
RecheckForeignScan_function
RecordCacheArrayEntry
--
2.40.1
[text/x-diff] v11-0017-BitmapHeapScan-uses-read-stream-API.patch (26.3K, ../../20240327193750.3mlcmzqondpj27xe@liskov/18-v11-0017-BitmapHeapScan-uses-read-stream-API.patch)
download | inline diff:
From bc1f127ff17a27f5bd9ac6dff96576707bbe1855 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplageman@gmail.com>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v11 17/17] BitmapHeapScan uses read stream API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the read stream API prefetching. Heap table AM implements a read
stream callback which uses the iterator to get the next valid block that
needs to be fetched for the read stream API.
ci-os-only:
---
src/backend/access/heap/heapam.c | 96 ++++--
src/backend/access/heap/heapam_handler.c | 347 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 43 +--
src/include/access/heapam.h | 21 +-
src/include/access/relscan.h | 6 -
src/include/access/tableam.h | 14 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 114 insertions(+), 422 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 5fc052f019..f7b1012b26 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -108,6 +108,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_stream_read_next(ReadStream *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -330,6 +332,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
+
+ scan->rs_read_stream = read_stream_begin_relation(READ_STREAM_DEFAULT,
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_stream_read_next,
+ scan,
+ sizeof(TBMIterateResult));
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -950,16 +968,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
+ scan->rs_read_stream = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1042,12 +1053,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1060,12 +1065,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1091,12 +1090,6 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1114,6 +1107,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
+
pfree(scan);
}
@@ -10130,3 +10126,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_stream_read_next(ReadStream *pgsr, void *private_data,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) private_data;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_bhs_iterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 80f210b405..2e17a85d00 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -61,9 +61,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2167,147 +2164,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *per_buffer_data;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_read_stream);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_bhs_iterator, &tbmres);
+ hscan->rs_cbuf = read_stream_next_buffer(hscan->rs_read_stream, &per_buffer_data);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(per_buffer_data);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = per_buffer_data;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2328,7 +2246,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2337,9 +2255,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2389,23 +2307,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_bhs_iterator &&
- hscan->pfblockno > hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2416,153 +2322,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2597,41 +2356,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2f9387e51a..f2662ea542 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -131,14 +131,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -149,15 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,13 +157,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -213,22 +190,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- {
- scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
- scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
- dsa);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
}
@@ -525,14 +492,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -563,11 +526,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4726d31403..bb701cca08 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/read_stream.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ ReadStream *rs_read_stream;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -82,23 +86,6 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /*
- * These fields only used for prefetching in bitmap table scans
- */
-
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used in serial BHS
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
-
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 7938b741d6..02893bf99b 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -46,13 +46,7 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
- struct BitmapHeapIterator *rs_pf_bhs_iterator;
-
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate BHS prefetch and current block stay in sync */
- BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5979ddee8b..2b24d7441a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -944,8 +944,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
- result->rs_pf_bhs_iterator = NULL;
- result->prefetch_maximum = 0;
result->bm_parallel = NULL;
return result;
}
@@ -1011,12 +1009,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1033,12 +1025,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 60916bf0d0..430668f597 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1758,11 +1758,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1770,10 +1766,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-25 16:07 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
@ 2024-03-28 05:20 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-28 18:01 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
1 sibling, 1 reply; 248+ messages in thread
From: Thomas Munro @ 2024-03-28 05:20 UTC (permalink / raw)
To: Melanie Plageman <melanieplageman@gmail.com>; +Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Nazir Bilal Yavuz <byavuz81@gmail.com>
With the unexplained but apparently somewhat systematic regression
patterns on certain tests and settings, I wonder if they might be due
to read_stream.c trying to form larger reads, making it a bit lazier.
It tries to see what the next block will be before issuing the
fadvise. I think that means that with small I/O concurrency settings,
there might be contrived access patterns where it loses, and needs
effective_io_concurrency to be set one notch higher to keep up, or
something like that. One way to test that idea would be to run the
tests with io_combine_limit = 1 (meaning 1 block). It issues advise
eagerly when io_combine_limit is reached, so I suppose it should be
exactly as eager as master. The only difference then should be that
it automatically suppresses sequential fadvise calls.
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-25 16:07 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-28 05:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
@ 2024-03-28 18:01 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-28 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 248+ messages in thread
From: Tomas Vondra @ 2024-03-28 18:01 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; Melanie Plageman <melanieplageman@gmail.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Nazir Bilal Yavuz <byavuz81@gmail.com>
On 3/28/24 06:20, Thomas Munro wrote:
> With the unexplained but apparently somewhat systematic regression
> patterns on certain tests and settings, I wonder if they might be due
> to read_stream.c trying to form larger reads, making it a bit lazier.
> It tries to see what the next block will be before issuing the
> fadvise. I think that means that with small I/O concurrency settings,
> there might be contrived access patterns where it loses, and needs
> effective_io_concurrency to be set one notch higher to keep up, or
> something like that.
Yes, I think we've speculated this might be the root cause before, but
IIRC we didn't manage to verify it actually is the problem.
FWIW I don't think the tests use synthetic data, but I don't think it's
particularly contrived.
> One way to test that idea would be to run the
> tests with io_combine_limit = 1 (meaning 1 block). It issues advise
> eagerly when io_combine_limit is reached, so I suppose it should be
> exactly as eager as master. The only difference then should be that
> it automatically suppresses sequential fadvise calls.
Sure, I'll give that a try. What are some good values to test? Perhaps
32 and 1, i.e. the default and "no coalescing"?
If this turns out to be the problem, does that mean we would consider
using a more conservative default value? Is there some "auto tuning" we
could do? For example, could we reduce the value combine limit if we
start not finding buffers in memory, or something like that?
I recognize this may not be possible with buffered I/O, due to not
having any insight into page cache. And maybe it's misguided anyway,
because how would we know if the right response is to increase or reduce
the combine limit?
Anyway, doesn't the combine limit work against the idea that
effective_io_concurrency is "prefetch distance"? With eic=32 I'd expect
we issue prefetch 32 pages ahead, i.e. if we prefetch page X, we should
then process 32 pages before we actually need X (and we expect the page
to already be in memory, thanks to the gap). But with the combine limit
set to 32, is this still true?
I've tried going through read_stream_* to determine how this will
behave, but read_stream_look_ahead/read_stream_start_pending_read does
not make this very clear. I'll have to experiment with some tracing.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-25 16:07 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-28 05:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <thomas.munro@gmail.com>
2024-03-28 18:01 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-28 21:19 ` Thomas Munro <thomas.munro@gmail.com>
0 siblings, 0 replies; 248+ messages in thread
From: Thomas Munro @ 2024-03-28 21:19 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Fri, Mar 29, 2024 at 7:01 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
> On 3/28/24 06:20, Thomas Munro wrote:
> > With the unexplained but apparently somewhat systematic regression
> > patterns on certain tests and settings, I wonder if they might be due
> > to read_stream.c trying to form larger reads, making it a bit lazier.
> > It tries to see what the next block will be before issuing the
> > fadvise. I think that means that with small I/O concurrency settings,
> > there might be contrived access patterns where it loses, and needs
> > effective_io_concurrency to be set one notch higher to keep up, or
> > something like that.
>
> Yes, I think we've speculated this might be the root cause before, but
> IIRC we didn't manage to verify it actually is the problem.
Another factor could be the bug in master that allows it to get out of
sync -- can it allow *more* concurrency than it intended to? Or fewer
hints, but somehow that goes faster because of the
stepping-on-kernel-toes problem?
> > One way to test that idea would be to run the
> > tests with io_combine_limit = 1 (meaning 1 block). It issues advise
> > eagerly when io_combine_limit is reached, so I suppose it should be
> > exactly as eager as master. The only difference then should be that
> > it automatically suppresses sequential fadvise calls.
>
> Sure, I'll give that a try. What are some good values to test? Perhaps
> 32 and 1, i.e. the default and "no coalescing"?
Thanks! Yeah. The default is actually 16, computed backwards from
128kB. (Explanation: POSIX requires 16 as minimum IOV_MAX, ie number
of vectors acceptable to writev/readv and related functions, though
actual acceptable number is usually much higher, and it also seems to
be a conservative acceptable number for hardware scatter/gather lists
in various protocols, ie if doing direct I/O, the transfer won't be
chopped up into more than one physical I/O command because the disk
and DMA engine can handle it as a single I/O in theory at least.
Actual limit on random SSDs might be more like 33, a weird number but
that's what I'm seeing; Mr Axboe wrote a nice short article[1] to get
some starting points for terminology on that topic on Linux. Also,
just anecdotally, returns seem to diminish after that with huge
transfers of buffered I/O so it seems like an OK number if you have to
pick one; but IDK, YMMV, subject for future research as direct I/O
grows in relevance, hence GUC.)
> If this turns out to be the problem, does that mean we would consider
> using a more conservative default value? Is there some "auto tuning" we
> could do? For example, could we reduce the value combine limit if we
> start not finding buffers in memory, or something like that?
Hmm, not sure... I like that number for seq scans. I also like
auto-tuning. But it seems to me that *if* the problem is that we're
not allowing ourselves as many concurrent I/Os as master BHS because
we're waiting to see if the next block is consecutive, that might
indicate that the distance needs to be higher so that we can have a
better chance to see the 'edge' (the non-contiguous next block) and
start the I/O, not that the io_combine_limit needs to be lower. But I
could be way off, given the fuzziness on this problem so far...
> Anyway, doesn't the combine limit work against the idea that
> effective_io_concurrency is "prefetch distance"? With eic=32 I'd expect
> we issue prefetch 32 pages ahead, i.e. if we prefetch page X, we should
> then process 32 pages before we actually need X (and we expect the page
> to already be in memory, thanks to the gap). But with the combine limit
> set to 32, is this still true?
Hmm. It's different. (1) Master BHS has prefetch_maximum, which is
indeed directly taken from the eic setting, while read_stream.c is
prepared to look much ahead further than that (potentially as far as
max_pinned_buffers) if it's been useful recently, to find
opportunities to coalesce and start I/O. (2) Master BHS has
prefetch_target to control the look-ahead window, which starts small
and ramps up until it hits prefetch_maximum, while read_stream.c has
distance which goes up and down according to a more complex algorithm
described at the top.
> I've tried going through read_stream_* to determine how this will
> behave, but read_stream_look_ahead/read_stream_start_pending_read does
> not make this very clear. I'll have to experiment with some tracing.
I'm going to try to set up something like your experiment here too,
and figure out some way to visualise or trace what's going on...
The differences come from (1) multi-block I/Os, requiring two separate
numbers: how many blocks ahead we're looking, and how many I/Os are
running, and (2) being more aggressive about trying to reach the
desired I/O level. Let me try to describe the approach again.
"distance" is the size of a window that we're searching for
opportunities to start I/Os. read_stream_look_ahead() will keep
looking ahead until we already have max_ios I/Os running, or we hit
the end of that window. That's the two conditions in the while loop
at the top:
while (stream->ios_in_progress < stream->max_ios &&
stream->pinned_buffers + stream->pending_read_nblocks <
stream->distance)
If that window is not large enough, we won't be able to find enough
I/Os to reach max_ios. So, every time we finish up starting a random
(non-sequential) I/O, we increase the distance, widening the window
until it can hopefully reach the I/O goal. (I call that behaviour C
in the comments and code.) In other words, master BHS can only find
opportunities to start I/Os in a smaller window, and can only reach
the full I/O concurrency target if they are right next to each other
in that window, but read_stream.c will look much further ahead, but
only if that has recently proven to be useful.
If we find I/Os that need doing, but they're all sequential, the
window size moves towards io_combine_limit, because we know that
issuing advice won't help, so there is no point in making the window
wider than one maximum-sized I/O. For example, sequential scans or
bitmap heapscans with lots of consecutive page bits fall into this
pattern. (Behaviour B in the code comments.) This is a pattern that
master BHS doesn't have anything like.
If we find that we don't need to do any I/O, we slowly move the window
size towards 1 (also the initial value) as there is no point in doing
anything special as it can't help. In contrast, master BHS never
shrinks its prefetch_target, it only goes up until it hits eic.
[1] https://kernel.dk/when-2mb-turns-into-512k.pdf
^ permalink raw reply [nested|flat] 248+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 19:42 ` Re: BitmapHeapScan streaming read user and prelim refactoring Andres Freund <andres@anarazel.de>
2024-03-18 12:10 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Re: BitmapHeapScan streaming read user and prelim refactoring Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-25 16:07 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
@ 2024-03-29 01:12 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <tomas.vondra@enterprisedb.com>
0 siblings, 1 reply; 248+ messages in thread
From: Thomas Munro @ 2024-03-29 01:12 UTC (permalink / raw)
To: Tomas Vondra <tomas.vondra@enterprisedb.com>; +Cc: Melanie Plageman <melanieplageman@gmail.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Andres Freund <andres@anarazel.de>; pgsql-hackers; Nazir Bilal Yavuz <byavuz81@gmail.com>
On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
> I think there's some sort of bug, triggering this assert in heapam
>
> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
Thanks for the repro. I can't seem to reproduce it (still trying) but
I assume this is with Melanie's v11 patch set which had
v11-0016-v10-Read-Stream-API.patch.
Would you mind removing that commit and instead applying the v13
stream_read.c patches[1]? v10 stream_read.c was a little confused
about random I/O combining, which I fixed with a small adjustment to
the conditions for the "if" statement right at the end of
read_stream_look_ahead(). Sorry about that. The fixed version, with
eic=4, with your test query using WHERE a < a, ends its scan with:
...
posix_fadvise(32,0x28aee000,0x4000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0@4\M-5:\0\0\^D\0\M-x\^A"...,40960,0x28acc000) = 40960 (0xa000)
posix_fadvise(32,0x28af4000,0x4000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0\^XC\M-6:\0\0\^D\0\M-x"...,32768,0x28ad8000) = 32768 (0x8000)
posix_fadvise(32,0x28afc000,0x4000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0\M-XQ\M-7:\0\0\^D\0\M-x"...,24576,0x28ae4000) = 24576 (0x6000)
posix_fadvise(32,0x28b02000,0x8000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0\M^@3\M-8:\0\0\^D\0\M-x"...,16384,0x28aee000) = 16384 (0x4000)
pread(32,"\0\0\0\0\M-`\M-:\M-8:\0\0\^D\0"...,16384,0x28af4000) = 16384 (0x4000)
pread(32,"\0\0\0\0po\M-9:\0\0\^D\0\M-x\^A"...,16384,0x28afc000) = 16384 (0x4000)
pread(32,"\0\0\0\0\M-P\M-v\M-9:\0\0\^D\0"...,32768,0x28b02000) = 32768 (0x8000)
In other words it's able to coalesce, but v10 was a bit b0rked in that
respect and wouldn't do as well at that. Then if you set
io_combine_limit = 1, it looks more like master, eg lots of little
reads, but not as many fadvises as master because of sequential
access:
...
posix_fadvise(32,0x28af4000,0x2000,POSIX_FADV_WILLNEED) = 0 (0x0) -+
pread(32,...,8192,0x28ae8000) = 8192 (0x2000) |
pread(32,...,8192,0x28aee000) = 8192 (0x2000) |
posix_fadvise(32,0x28afc000,0x2000,POSIX_FADV_WILLNEED) = 0 (0x0) ---+
pread(32,...,8192,0x28af0000) = 8192 (0x2000) | |
pread(32,...,8192,0x28af4000) = 8192 (0x2000) <--------------------+ |
posix_fadvise(32,0x28b02000,0x2000,POSIX_FADV_WILLNEED) = 0 (0x0) -----+
pread(32,...,8192,0x28af6000) = 8192 (0x2000) | |
pread(32,...,8192,0x28afc000) = 8192 (0x2000) <----------------------+ |
pread(32,...,8192,0x28afe000) = 8192 (0x2000) }-- no advice |
pread(32,...,8192,0x28b02000) = 8192 (0x2000) <------------------------+
pread(32,...,8192,0x28b04000) = 8192 (0x2000) }
pread(32,...,8192,0x28b06000) = 8192 (0x2000) }-- no advice
pread(32,...,8192,0x28b08000) = 8192 (0x2000) }
It becomes slightly less eager to start I/Os as soon as
io_combine_limit > 1, because when it has hit max_ios, if ... <thinks>
yeah if the average block that it can combine is bigger than 4, an
arbitrary number from:
max_pinned_buffers = Max(max_ios * 4, io_combine_limit);
.... then it can run out of look ahead window before it can reach
max_ios (aka eic), so that's a kind of arbitrary/bogus I/O depth
constraint, which is another way of saying what I was saying earlier:
maybe it just needs more distance. So let's see the average combined
I/O length in your test query... for me it works out to 27,169 bytes.
But I think there must be times when it runs out of window due to
clustering. So you could also try increasing that 4->8 to see what
happens to performance.
[1] https://www.postgresql.org/message-id/CA%2BhUKG%2B5UofvseJWv6YqKmuc_%3Drguc7VqKcNEG1eawKh3MzHXQ%40ma...
^ permalink raw reply [nested|flat] 248+ messages in thread
165 further messages in this thread omitted from this page (render size limit).
Use the per-message pages, or the t.mbox.gz byte stream, for the complete thread.
end of thread, other threads:[~2025-04-17 14:56 UTC | newest]
Thread overview: 248+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-02-13 23:11 BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 04:34 ` Mark Dilger <mark.dilger@enterprisedb.com>
2024-02-14 14:47 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-14 16:41 ` Mark Dilger <mark.dilger@enterprisedb.com>
2024-02-14 19:42 ` Andres Freund <andres@anarazel.de>
2024-02-16 03:31 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-16 17:35 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 01:50 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-27 14:22 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 13:22 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 14:38 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 14:56 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 19:23 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 20:06 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-28 23:17 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-28 23:40 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 12:54 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 21:19 ` Melanie Plageman <melanieplageman@gmail.com>
2024-02-29 22:44 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-02-29 23:44 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 01:18 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 14:05 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-01 16:51 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 17:08 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 15:05 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:28 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:41 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 22:52 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-13 22:38 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-14 14:17 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 20:58 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-02 23:59 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-03 00:15 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 00:29 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-01 19:31 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:11 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-02 22:51 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-02 23:39 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-03 14:36 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-13 13:34 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-13 15:39 ` Dilip Kumar <dilipbalaut@gmail.com>
2024-03-13 15:55 ` Robert Haas <robertmhaas@gmail.com>
2024-03-14 04:54 ` Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 10:37 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 10:55 ` Dilip Kumar <dilipbalaut@gmail.com>
2024-03-14 13:32 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 18:16 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-14 21:26 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 21:39 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-15 21:14 ` Andres Freund <andres@anarazel.de>
2024-03-15 22:42 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-16 19:12 ` Andres Freund <andres@anarazel.de>
2024-03-16 20:25 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-17 16:38 ` Andres Freund <andres@anarazel.de>
2024-03-17 19:36 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 11:34 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-17 19:21 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 14:47 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-18 15:55 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-19 20:34 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2025-01-30 23:10 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-21 14:55 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-14 12:34 ` Robert Haas <robertmhaas@gmail.com>
2024-03-14 13:00 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-14 13:20 ` Robert Haas <robertmhaas@gmail.com>
2024-03-13 18:14 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-18 12:10 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-18 15:19 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-19 12:33 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-03-23 00:22 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-23 00:26 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 12:36 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 17:38 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 18:22 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 20:12 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-24 21:59 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-24 22:37 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-25 16:07 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-27 19:37 ` Melanie Plageman <melanieplageman@gmail.com>
2024-03-28 05:20 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-28 18:01 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-28 21:19 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-28 21:43 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-29 01:12 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-29 11:05 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-29 11:17 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-29 13:36 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-29 15:52 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-29 21:39 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-29 22:03 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-29 23:34 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-30 00:09 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-29 23:40 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-29 23:56 ` Thomas Munro <thomas.munro@gmail.com>
2024-03-31 15:45 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-03 22:57 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-04 14:35 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-05 08:06 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-05 23:53 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-06 00:51 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-06 13:40 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-06 14:57 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-06 16:04 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-06 21:34 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-07 00:27 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-07 04:17 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-07 11:37 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-07 13:11 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-07 14:10 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-07 14:38 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-07 14:24 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-07 14:41 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-07 14:54 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-18 07:10 ` Michael Paquier <michael@paquier.xyz>
2024-04-18 09:39 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-22 17:01 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-23 16:05 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-23 22:43 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-04-24 20:46 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-25 23:03 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-25 23:28 ` Tom Lane <tgl@sss.pgh.pa.us>
2024-04-25 23:57 ` Tom Lane <tgl@sss.pgh.pa.us>
2024-04-26 13:04 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-30 12:07 ` Daniel Gustafsson <daniel@yesql.se>
2024-05-02 21:31 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-05-02 21:37 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-05-10 19:48 ` Melanie Plageman <melanieplageman@gmail.com>
2024-05-11 19:18 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-05-13 14:05 ` Melanie Plageman <melanieplageman@gmail.com>
2024-05-14 06:18 ` Michael Paquier <michael@paquier.xyz>
2024-05-14 17:42 ` Melanie Plageman <melanieplageman@gmail.com>
2024-05-14 18:33 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-05-14 18:40 ` Melanie Plageman <melanieplageman@gmail.com>
2024-05-14 18:44 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-05-14 19:11 ` Melanie Plageman <melanieplageman@gmail.com>
2024-05-14 20:05 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2024-05-14 20:09 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2024-05-14 21:19 ` Melanie Plageman <melanieplageman@gmail.com>
2024-05-15 14:24 ` Robert Haas <robertmhaas@gmail.com>
2024-05-15 15:14 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2024-04-12 03:18 ` Richard Guo <guofenglinux@gmail.com>
2024-06-14 23:56 ` Melanie Plageman <melanieplageman@gmail.com>
2024-06-17 21:22 ` Melanie Plageman <melanieplageman@gmail.com>
2024-06-18 22:02 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-06-19 15:55 ` Melanie Plageman <melanieplageman@gmail.com>
2024-06-19 16:38 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-06-19 18:13 ` Melanie Plageman <melanieplageman@gmail.com>
2024-09-27 20:13 ` Melanie Plageman <melanieplageman@gmail.com>
2024-10-08 20:56 ` Thomas Munro <thomas.munro@gmail.com>
2024-10-14 22:25 ` Melanie Plageman <melanieplageman@gmail.com>
2024-08-26 13:37 ` Heikki Linnakangas <hlinnaka@iki.fi>
2024-10-15 00:34 ` Melanie Plageman <melanieplageman@gmail.com>
2024-10-16 15:09 ` Tomas Vondra <tomas@vondra.me>
2024-10-18 20:48 ` Melanie Plageman <melanieplageman@gmail.com>
2024-12-09 06:22 ` Dilip Kumar <dilipbalaut@gmail.com>
2025-01-22 21:24 ` Melanie Plageman <melanieplageman@gmail.com>
2025-01-29 21:38 ` Melanie Plageman <melanieplageman@gmail.com>
2025-01-30 08:02 ` Nazir Bilal Yavuz <byavuz81@gmail.com>
2025-01-30 20:36 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-09 14:27 ` Tomas Vondra <tomas@vondra.me>
2025-02-10 18:02 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-10 18:11 ` Tomas Vondra <tomas@vondra.me>
2025-02-10 21:24 ` Robert Haas <robertmhaas@gmail.com>
2025-02-10 21:41 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-12 19:08 ` Robert Haas <robertmhaas@gmail.com>
2025-02-12 20:07 ` Tomas Vondra <tomas@vondra.me>
2025-02-12 20:57 ` Robert Haas <robertmhaas@gmail.com>
2025-02-14 09:04 ` Jakub Wartak <jakub.wartak@enterprisedb.com>
2025-02-14 17:14 ` Andres Freund <andres@anarazel.de>
2025-02-14 17:36 ` Tomas Vondra <tomas@vondra.me>
2025-02-14 18:16 ` Andres Freund <andres@anarazel.de>
2025-02-19 13:28 ` Jakub Wartak <jakub.wartak@enterprisedb.com>
2025-02-20 16:51 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-10 22:06 ` Andres Freund <andres@anarazel.de>
2025-02-10 21:22 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-10 22:31 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-13 15:46 ` Tomas Vondra <tomas@vondra.me>
2025-02-13 16:01 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-13 16:28 ` Tomas Vondra <tomas@vondra.me>
2025-02-13 16:52 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-13 22:50 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-13 22:53 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-19 21:14 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-20 22:17 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-21 00:32 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-21 22:00 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-24 16:18 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-24 22:07 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-10 19:06 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-10 22:11 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-10 23:45 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-12 00:07 ` Andres Freund <andres@anarazel.de>
2025-03-12 20:01 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-13 09:46 ` Jakub Wartak <jakub.wartak@enterprisedb.com>
2025-03-13 20:34 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-17 07:43 ` Jakub Wartak <jakub.wartak@enterprisedb.com>
2025-03-17 18:52 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-17 18:54 ` Andres Freund <andres@anarazel.de>
2025-03-17 21:46 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-18 10:51 ` Jakub Wartak <jakub.wartak@enterprisedb.com>
2025-03-13 21:41 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-15 14:43 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-15 15:20 ` Andres Freund <andres@anarazel.de>
2025-03-22 20:00 ` Alexander Lakhin <exclusion@gmail.com>
2025-03-22 20:14 ` Andres Freund <andres@anarazel.de>
2025-03-22 20:42 ` Andres Freund <andres@anarazel.de>
2025-03-22 21:04 ` Andres Freund <andres@anarazel.de>
2025-03-23 17:27 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-24 16:14 ` Melanie Plageman <melanieplageman@gmail.com>
2025-03-24 20:44 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-13 20:32 ` Andres Freund <andres@anarazel.de>
2025-02-19 23:12 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-20 16:10 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-14 17:18 ` Tomas Vondra <tomas@vondra.me>
2025-02-14 17:31 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-14 17:50 ` Tomas Vondra <tomas@vondra.me>
2025-02-16 01:15 ` Tomas Vondra <tomas@vondra.me>
2025-02-16 12:29 ` Tomas Vondra <tomas@vondra.me>
2025-02-20 16:36 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-14 18:44 ` Andres Freund <andres@anarazel.de>
2025-02-13 00:40 ` Melanie Plageman <melanieplageman@gmail.com>
2025-02-13 04:15 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-13 12:47 ` Tomas Vondra <tomas@vondra.me>
2025-02-13 12:08 ` Tomas Vondra <tomas@vondra.me>
2025-02-13 14:35 ` Melanie Plageman <melanieplageman@gmail.com>
2025-04-08 02:33 ` Thomas Munro <thomas.munro@gmail.com>
2025-04-09 01:46 ` James Hunter <james.hunter.pg@gmail.com>
2025-04-10 06:00 ` Thomas Munro <thomas.munro@gmail.com>
2025-04-10 17:50 ` James Hunter <james.hunter.pg@gmail.com>
2025-04-11 03:14 ` Thomas Munro <thomas.munro@gmail.com>
2025-04-14 16:58 ` James Hunter <james.hunter.pg@gmail.com>
2025-04-14 17:20 ` Andres Freund <andres@anarazel.de>
2025-04-15 17:58 ` James Hunter <james.hunter.pg@gmail.com>
2025-04-14 17:44 ` Robert Haas <robertmhaas@gmail.com>
2025-04-16 04:30 ` Thomas Munro <thomas.munro@gmail.com>
2025-04-17 14:56 ` Robert Haas <robertmhaas@gmail.com>
2024-12-19 09:15 ` Richard Guo <guofenglinux@gmail.com>
2024-12-19 15:12 ` Richard Guo <guofenglinux@gmail.com>
2024-12-19 15:23 ` Melanie Plageman <melanieplageman@gmail.com>
2024-12-19 16:58 ` Melanie Plageman <melanieplageman@gmail.com>
2024-06-19 16:51 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2024-06-19 18:20 ` Melanie Plageman <melanieplageman@gmail.com>
2024-08-26 14:24 ` Robert Haas <robertmhaas@gmail.com>
2024-08-26 14:49 ` Tom Lane <tgl@sss.pgh.pa.us>
2024-09-27 18:58 ` Melanie Plageman <melanieplageman@gmail.com>
2024-04-06 14:59 ` Tom Lane <tgl@sss.pgh.pa.us>
2024-04-06 14:40 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
2024-03-20 18:13 ` Tomas Vondra <tomas.vondra@enterprisedb.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox