public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 1/9] Pass all scan keys to BRIN consistent function at once
21+ messages / 4 participants
[nested] [flat]
* [PATCH 1/9] Pass all scan keys to BRIN consistent function at once
@ 2020-09-12 13:07 Tomas Vondra <[email protected]>
0 siblings, 0 replies; 21+ messages in thread
From: Tomas Vondra @ 2020-09-12 13:07 UTC (permalink / raw)
Passing all scan keys to the BRIN consistent function at once may allow
elimination of additional ranges, which would be impossible when only
passing individual scan keys.
The code continues to support both the original (one scan key at a time)
and new (all scan keys at once) approaches, depending on whether the
consistent function accepts three or four arguments.
This modifies the existing BRIN opclasses (minmax, inclusion) although
those don't really benefit from this change. The primary purpose of this
is to allow more advanced opclases in the future.
Author: Tomas Vondra <[email protected]>
Reviewed-by: Alvaro Herrera <[email protected]>
Reviewed-by: Mark Dilger <[email protected]>
Reviewed-by: Alexander Korotkov <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/backend/access/brin/brin.c | 150 +++++++++++++++-----
src/backend/access/brin/brin_inclusion.c | 170 +++++++++++++++--------
src/backend/access/brin/brin_minmax.c | 121 +++++++++++-----
src/backend/access/brin/brin_validate.c | 4 +-
src/include/catalog/pg_proc.dat | 4 +-
5 files changed, 324 insertions(+), 125 deletions(-)
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index 27ba596c6e..dc187153aa 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -390,6 +390,9 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
BrinMemTuple *dtup;
BrinTuple *btup = NULL;
Size btupsz = 0;
+ ScanKey **keys;
+ int *nkeys;
+ int keyno;
opaque = (BrinOpaque *) scan->opaque;
bdesc = opaque->bo_bdesc;
@@ -411,6 +414,61 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
*/
consistentFn = palloc0(sizeof(FmgrInfo) * bdesc->bd_tupdesc->natts);
+ /*
+ * Make room for per-attribute lists of scan keys that we'll pass to the
+ * consistent support procedure.
+ */
+ keys = palloc0(sizeof(ScanKey *) * bdesc->bd_tupdesc->natts);
+ nkeys = palloc0(sizeof(int) * bdesc->bd_tupdesc->natts);
+
+ /*
+ * Preprocess the scan keys - split them into per-attribute arrays.
+ */
+ for (keyno = 0; keyno < scan->numberOfKeys; keyno++)
+ {
+ ScanKey key = &scan->keyData[keyno];
+ AttrNumber keyattno = key->sk_attno;
+
+ /*
+ * The collation of the scan key must match the collation
+ * used in the index column (but only if the search is not
+ * IS NULL/ IS NOT NULL). Otherwise we shouldn't be using
+ * this index ...
+ */
+ Assert((key->sk_flags & SK_ISNULL) ||
+ (key->sk_collation ==
+ TupleDescAttr(bdesc->bd_tupdesc,
+ keyattno - 1)->attcollation));
+
+ /* First time we see this index attribute, so init as needed. */
+ if (!keys[keyattno-1])
+ {
+ FmgrInfo *tmp;
+
+ /*
+ * This is a bit of an overkill - we don't know how many
+ * scan keys are there for this attribute, so we simply
+ * allocate the largest number possible. This may waste
+ * a bit of memory, but we only expect small number of
+ * scan keys in general, so this should be negligible,
+ * and it's cheaper than having to repalloc repeatedly.
+ */
+ keys[keyattno - 1] = palloc0(sizeof(ScanKey) * scan->numberOfKeys);
+
+ /* First time this column, so look up consistent function */
+ Assert(consistentFn[keyattno - 1].fn_oid == InvalidOid);
+
+ tmp = index_getprocinfo(idxRel, keyattno,
+ BRIN_PROCNUM_CONSISTENT);
+ fmgr_info_copy(&consistentFn[keyattno - 1], tmp,
+ CurrentMemoryContext);
+ }
+
+ /* Add key to the per-attribute array. */
+ keys[keyattno - 1][nkeys[keyattno - 1]] = key;
+ nkeys[keyattno - 1]++;
+ }
+
/* allocate an initial in-memory tuple, out of the per-range memcxt */
dtup = brin_new_memtuple(bdesc);
@@ -471,7 +529,7 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
}
else
{
- int keyno;
+ int attno;
/*
* Compare scan keys with summary values stored for the range.
@@ -481,51 +539,75 @@ bringetbitmap(IndexScanDesc scan, TIDBitmap *tbm)
* no keys.
*/
addrange = true;
- for (keyno = 0; keyno < scan->numberOfKeys; keyno++)
+ for (attno = 1; attno <= bdesc->bd_tupdesc->natts; attno++)
{
- ScanKey key = &scan->keyData[keyno];
- AttrNumber keyattno = key->sk_attno;
- BrinValues *bval = &dtup->bt_columns[keyattno - 1];
+ BrinValues *bval;
Datum add;
- /*
- * The collation of the scan key must match the collation
- * used in the index column (but only if the search is not
- * IS NULL/ IS NOT NULL). Otherwise we shouldn't be using
- * this index ...
- */
- Assert((key->sk_flags & SK_ISNULL) ||
- (key->sk_collation ==
- TupleDescAttr(bdesc->bd_tupdesc,
- keyattno - 1)->attcollation));
+ /* skip attributes without any san keys */
+ if (nkeys[attno - 1] == 0)
+ continue;
- /* First time this column? look up consistent function */
- if (consistentFn[keyattno - 1].fn_oid == InvalidOid)
- {
- FmgrInfo *tmp;
+ bval = &dtup->bt_columns[attno - 1];
- tmp = index_getprocinfo(idxRel, keyattno,
- BRIN_PROCNUM_CONSISTENT);
- fmgr_info_copy(&consistentFn[keyattno - 1], tmp,
- CurrentMemoryContext);
- }
+ Assert((nkeys[attno - 1] > 0) &&
+ (nkeys[attno - 1] <= scan->numberOfKeys));
/*
* Check whether the scan key is consistent with the page
* range values; if so, have the pages in the range added
* to the output bitmap.
*
- * When there are multiple scan keys, failure to meet the
- * criteria for a single one of them is enough to discard
- * the range as a whole, so break out of the loop as soon
- * as a false return value is obtained.
+ * The opclass may or may not support processing of multiple
+ * scan keys. We can determine that based on the number of
+ * arguments - functions with extra parameter (number of scan
+ * keys) do support this, otherwise we have to simply pass the
+ * scan keys one by one,
*/
- add = FunctionCall3Coll(&consistentFn[keyattno - 1],
- key->sk_collation,
- PointerGetDatum(bdesc),
- PointerGetDatum(bval),
- PointerGetDatum(key));
- addrange = DatumGetBool(add);
+ if (consistentFn[attno - 1].fn_nargs >= 4)
+ {
+ Oid collation;
+
+ /*
+ * Collation from the first key (has to be the same for
+ * all keys for the same attribue).
+ */
+ collation = keys[attno - 1][0]->sk_collation;
+
+ /* Check all keys at once */
+ add = FunctionCall4Coll(&consistentFn[attno - 1],
+ collation,
+ PointerGetDatum(bdesc),
+ PointerGetDatum(bval),
+ PointerGetDatum(keys[attno - 1]),
+ Int32GetDatum(nkeys[attno - 1]));
+ addrange = DatumGetBool(add);
+ }
+ else
+ {
+ /*
+ * Check keys one by one
+ *
+ * When there are multiple scan keys, failure to meet the
+ * criteria for a single one of them is enough to discard
+ * the range as a whole, so break out of the loop as soon
+ * as a false return value is obtained.
+ */
+ int keyno;
+
+ for (keyno = 0; keyno < nkeys[attno - 1]; keyno++)
+ {
+ add = FunctionCall3Coll(&consistentFn[attno - 1],
+ keys[attno - 1][keyno]->sk_collation,
+ PointerGetDatum(bdesc),
+ PointerGetDatum(bval),
+ PointerGetDatum(keys[attno - 1][keyno]));
+ addrange = DatumGetBool(add);
+ if (!addrange)
+ break;
+ }
+ }
+
if (!addrange)
break;
}
diff --git a/src/backend/access/brin/brin_inclusion.c b/src/backend/access/brin/brin_inclusion.c
index 12e5bddd1f..215bc794d3 100644
--- a/src/backend/access/brin/brin_inclusion.c
+++ b/src/backend/access/brin/brin_inclusion.c
@@ -85,6 +85,8 @@ static FmgrInfo *inclusion_get_procinfo(BrinDesc *bdesc, uint16 attno,
uint16 procnum);
static FmgrInfo *inclusion_get_strategy_procinfo(BrinDesc *bdesc, uint16 attno,
Oid subtype, uint16 strategynum);
+static bool inclusion_consistent_key(BrinDesc *bdesc, BrinValues *column,
+ ScanKey key, Oid colloid);
/*
@@ -258,53 +260,109 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
{
BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0);
BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1);
- ScanKey key = (ScanKey) PG_GETARG_POINTER(2);
- Oid colloid = PG_GET_COLLATION(),
- subtype;
- Datum unionval;
- AttrNumber attno;
- Datum query;
- FmgrInfo *finfo;
- Datum result;
-
- Assert(key->sk_attno == column->bv_attno);
+ ScanKey *keys = (ScanKey *) PG_GETARG_POINTER(2);
+ int nkeys = PG_GETARG_INT32(3);
+ Oid colloid = PG_GET_COLLATION();
+ int keyno;
+ bool regular_keys = false;
- /* Handle IS NULL/IS NOT NULL tests. */
- if (key->sk_flags & SK_ISNULL)
+ /*
+ * First check if there are any IS NULL scan keys, and if we're
+ * violating them. In that case we can terminate early, without
+ * inspecting the ranges.
+ */
+ for (keyno = 0; keyno < nkeys; keyno++)
{
- if (key->sk_flags & SK_SEARCHNULL)
+ ScanKey key = keys[keyno];
+
+ Assert(key->sk_attno == column->bv_attno);
+
+ /* handle IS NULL/IS NOT NULL tests */
+ if (key->sk_flags & SK_ISNULL)
{
- if (column->bv_allnulls || column->bv_hasnulls)
- PG_RETURN_BOOL(true);
- PG_RETURN_BOOL(false);
- }
+ if (key->sk_flags & SK_SEARCHNULL)
+ {
+ if (column->bv_allnulls || column->bv_hasnulls)
+ continue; /* this key is fine, continue */
- /*
- * For IS NOT NULL, we can only skip ranges that are known to have
- * only nulls.
- */
- if (key->sk_flags & SK_SEARCHNOTNULL)
- PG_RETURN_BOOL(!column->bv_allnulls);
+ PG_RETURN_BOOL(false);
+ }
- /*
- * Neither IS NULL nor IS NOT NULL was used; assume all indexable
- * operators are strict and return false.
- */
- PG_RETURN_BOOL(false);
+ /*
+ * For IS NOT NULL, we can only skip ranges that are known to have
+ * only nulls.
+ */
+ if (key->sk_flags & SK_SEARCHNOTNULL)
+ {
+ if (column->bv_allnulls)
+ PG_RETURN_BOOL(false);
+
+ continue;
+ }
+
+ /*
+ * Neither IS NULL nor IS NOT NULL was used; assume all indexable
+ * operators are strict and return false.
+ */
+ PG_RETURN_BOOL(false);
+ }
+ else
+ /* note we have regular (non-NULL) scan keys */
+ regular_keys = true;
}
- /* If it is all nulls, it cannot possibly be consistent. */
- if (column->bv_allnulls)
+ /*
+ * If the page range is all nulls, it cannot possibly be consistent if
+ * there are some regular scan keys.
+ */
+ if (column->bv_allnulls && regular_keys)
PG_RETURN_BOOL(false);
+ /* If there are no regular keys, the page range is considered consistent. */
+ if (!regular_keys)
+ PG_RETURN_BOOL(true);
+
/* It has to be checked, if it contains elements that are not mergeable. */
if (DatumGetBool(column->bv_values[INCLUSION_UNMERGEABLE]))
PG_RETURN_BOOL(true);
- attno = key->sk_attno;
- subtype = key->sk_subtype;
- query = key->sk_argument;
- unionval = column->bv_values[INCLUSION_UNION];
+ /* Check that the range is consistent with all scan keys. */
+ for (keyno = 0; keyno < nkeys; keyno++)
+ {
+ ScanKey key = keys[keyno];
+
+ /* ignore IS NULL/IS NOT NULL tests handled above */
+ if (key->sk_flags & SK_ISNULL)
+ continue;
+
+ /*
+ * When there are multiple scan keys, failure to meet the
+ * criteria for a single one of them is enough to discard
+ * the range as a whole, so break out of the loop as soon
+ * as a false return value is obtained.
+ */
+ if (!inclusion_consistent_key(bdesc, column, key, colloid))
+ PG_RETURN_BOOL(false);
+ }
+
+ PG_RETURN_BOOL(true);
+}
+
+/*
+ * inclusion_consistent_key
+ * Determine if the range is consistent with a single scan key.
+ */
+static bool
+inclusion_consistent_key(BrinDesc *bdesc, BrinValues *column, ScanKey key,
+ Oid colloid)
+{
+ FmgrInfo *finfo;
+ AttrNumber attno = key->sk_attno;
+ Oid subtype = key->sk_subtype;
+ Datum query = key->sk_argument;
+ Datum unionval = column->bv_values[INCLUSION_UNION];
+ Datum result;
+
switch (key->sk_strategy)
{
/*
@@ -324,49 +382,49 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTOverRightStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
case RTOverLeftStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTRightStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
case RTOverRightStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTLeftStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
case RTRightStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTOverLeftStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
case RTBelowStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTOverAboveStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
case RTOverBelowStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTAboveStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
case RTOverAboveStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTBelowStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
case RTAboveStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTOverBelowStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
/*
* Overlap and contains strategies
@@ -384,7 +442,7 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
key->sk_strategy);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_DATUM(result);
+ return DatumGetBool(result);
/*
* Contained by strategies
@@ -404,9 +462,9 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
RTOverlapStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
if (DatumGetBool(result))
- PG_RETURN_BOOL(true);
+ return true;
- PG_RETURN_DATUM(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
+ return DatumGetBool(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
/*
* Adjacent strategy
@@ -423,12 +481,12 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
RTOverlapStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
if (DatumGetBool(result))
- PG_RETURN_BOOL(true);
+ return true;
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
- RTAdjacentStrategyNumber);
+ RTAdjacentStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_DATUM(result);
+ return DatumGetBool(result);
/*
* Basic comparison strategies
@@ -458,9 +516,9 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
RTRightStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
if (!DatumGetBool(result))
- PG_RETURN_BOOL(true);
+ return true;
- PG_RETURN_DATUM(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
+ return DatumGetBool(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
case RTSameStrategyNumber:
case RTEqualStrategyNumber:
@@ -468,30 +526,30 @@ brin_inclusion_consistent(PG_FUNCTION_ARGS)
RTContainsStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
if (DatumGetBool(result))
- PG_RETURN_BOOL(true);
+ return true;
- PG_RETURN_DATUM(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
+ return DatumGetBool(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
case RTGreaterEqualStrategyNumber:
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTLeftStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
if (!DatumGetBool(result))
- PG_RETURN_BOOL(true);
+ return true;
- PG_RETURN_DATUM(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
+ return DatumGetBool(column->bv_values[INCLUSION_CONTAINS_EMPTY]);
case RTGreaterStrategyNumber:
/* no need to check for empty elements */
finfo = inclusion_get_strategy_procinfo(bdesc, attno, subtype,
RTLeftStrategyNumber);
result = FunctionCall2Coll(finfo, colloid, unionval, query);
- PG_RETURN_BOOL(!DatumGetBool(result));
+ return !DatumGetBool(result);
default:
/* shouldn't happen */
elog(ERROR, "invalid strategy number %d", key->sk_strategy);
- PG_RETURN_BOOL(false);
+ return false;
}
}
diff --git a/src/backend/access/brin/brin_minmax.c b/src/backend/access/brin/brin_minmax.c
index 2ffbd9bf0d..12878ff3a0 100644
--- a/src/backend/access/brin/brin_minmax.c
+++ b/src/backend/access/brin/brin_minmax.c
@@ -30,6 +30,8 @@ typedef struct MinmaxOpaque
static FmgrInfo *minmax_get_strategy_procinfo(BrinDesc *bdesc, uint16 attno,
Oid subtype, uint16 strategynum);
+static bool minmax_consistent_key(BrinDesc *bdesc, BrinValues *column,
+ ScanKey key, Oid colloid);
Datum
@@ -146,47 +148,104 @@ brin_minmax_consistent(PG_FUNCTION_ARGS)
{
BrinDesc *bdesc = (BrinDesc *) PG_GETARG_POINTER(0);
BrinValues *column = (BrinValues *) PG_GETARG_POINTER(1);
- ScanKey key = (ScanKey) PG_GETARG_POINTER(2);
- Oid colloid = PG_GET_COLLATION(),
- subtype;
- AttrNumber attno;
- Datum value;
- Datum matches;
- FmgrInfo *finfo;
-
- Assert(key->sk_attno == column->bv_attno);
+ ScanKey *keys = (ScanKey *) PG_GETARG_POINTER(2);
+ int nkeys = PG_GETARG_INT32(3);
+ Oid colloid = PG_GET_COLLATION();
+ int keyno;
+ bool regular_keys = false;
- /* handle IS NULL/IS NOT NULL tests */
- if (key->sk_flags & SK_ISNULL)
+ /*
+ * First check if there are any IS NULL scan keys, and if we're
+ * violating them. In that case we can terminate early, without
+ * inspecting the ranges.
+ */
+ for (keyno = 0; keyno < nkeys; keyno++)
{
- if (key->sk_flags & SK_SEARCHNULL)
+ ScanKey key = keys[keyno];
+
+ Assert(key->sk_attno == column->bv_attno);
+
+ /* handle IS NULL/IS NOT NULL tests */
+ if (key->sk_flags & SK_ISNULL)
{
- if (column->bv_allnulls || column->bv_hasnulls)
- PG_RETURN_BOOL(true);
+ if (key->sk_flags & SK_SEARCHNULL)
+ {
+ if (column->bv_allnulls || column->bv_hasnulls)
+ continue; /* this key is fine, continue */
+
+ PG_RETURN_BOOL(false);
+ }
+
+ /*
+ * For IS NOT NULL, we can only skip ranges that are known to have
+ * only nulls.
+ */
+ if (key->sk_flags & SK_SEARCHNOTNULL)
+ {
+ if (column->bv_allnulls)
+ PG_RETURN_BOOL(false);
+
+ continue;
+ }
+
+ /*
+ * Neither IS NULL nor IS NOT NULL was used; assume all indexable
+ * operators are strict and return false.
+ */
PG_RETURN_BOOL(false);
}
+ else
+ /* note we have regular (non-NULL) scan keys */
+ regular_keys = true;
+ }
- /*
- * For IS NOT NULL, we can only skip ranges that are known to have
- * only nulls.
- */
- if (key->sk_flags & SK_SEARCHNOTNULL)
- PG_RETURN_BOOL(!column->bv_allnulls);
+ /*
+ * If the page range is all nulls, it cannot possibly be consistent if
+ * there are some regular scan keys.
+ */
+ if (column->bv_allnulls && regular_keys)
+ PG_RETURN_BOOL(false);
+
+ /* If there are no regular keys, the page range is considered consistent. */
+ if (!regular_keys)
+ PG_RETURN_BOOL(true);
- /*
- * Neither IS NULL nor IS NOT NULL was used; assume all indexable
- * operators are strict and return false.
+ /* Check that the range is consistent with all scan keys. */
+ for (keyno = 0; keyno < nkeys; keyno++)
+ {
+ ScanKey key = keys[keyno];
+
+ /* ignore IS NULL/IS NOT NULL tests handled above */
+ if (key->sk_flags & SK_ISNULL)
+ continue;
+
+ /*
+ * When there are multiple scan keys, failure to meet the
+ * criteria for a single one of them is enough to discard
+ * the range as a whole, so break out of the loop as soon
+ * as a false return value is obtained.
*/
- PG_RETURN_BOOL(false);
+ if (!minmax_consistent_key(bdesc, column, key, colloid))
+ PG_RETURN_DATUM(false);;
}
- /* if the range is all empty, it cannot possibly be consistent */
- if (column->bv_allnulls)
- PG_RETURN_BOOL(false);
+ PG_RETURN_DATUM(true);
+}
+
+/*
+ * minmax_consistent_key
+ * Determine if the range is consistent with a single scan key.
+ */
+static bool
+minmax_consistent_key(BrinDesc *bdesc, BrinValues *column, ScanKey key,
+ Oid colloid)
+{
+ FmgrInfo *finfo;
+ AttrNumber attno = key->sk_attno;
+ Oid subtype = key->sk_subtype;
+ Datum value = key->sk_argument;
+ Datum matches;
- attno = key->sk_attno;
- subtype = key->sk_subtype;
- value = key->sk_argument;
switch (key->sk_strategy)
{
case BTLessStrategyNumber:
@@ -229,7 +288,7 @@ brin_minmax_consistent(PG_FUNCTION_ARGS)
break;
}
- PG_RETURN_DATUM(matches);
+ return DatumGetBool(matches);
}
/*
diff --git a/src/backend/access/brin/brin_validate.c b/src/backend/access/brin/brin_validate.c
index 6d4253c05e..11835d85cd 100644
--- a/src/backend/access/brin/brin_validate.c
+++ b/src/backend/access/brin/brin_validate.c
@@ -97,8 +97,8 @@ brinvalidate(Oid opclassoid)
break;
case BRIN_PROCNUM_CONSISTENT:
ok = check_amproc_signature(procform->amproc, BOOLOID, true,
- 3, 3, INTERNALOID, INTERNALOID,
- INTERNALOID);
+ 3, 4, INTERNALOID, INTERNALOID,
+ INTERNALOID, INT4OID);
break;
case BRIN_PROCNUM_UNION:
ok = check_amproc_signature(procform->amproc, BOOLOID, true,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1487710d59..33841e14f2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8191,7 +8191,7 @@
prosrc => 'brin_minmax_add_value' },
{ oid => '3385', descr => 'BRIN minmax support',
proname => 'brin_minmax_consistent', prorettype => 'bool',
- proargtypes => 'internal internal internal',
+ proargtypes => 'internal internal internal int4',
prosrc => 'brin_minmax_consistent' },
{ oid => '3386', descr => 'BRIN minmax support',
proname => 'brin_minmax_union', prorettype => 'bool',
@@ -8207,7 +8207,7 @@
prosrc => 'brin_inclusion_add_value' },
{ oid => '4107', descr => 'BRIN inclusion support',
proname => 'brin_inclusion_consistent', prorettype => 'bool',
- proargtypes => 'internal internal internal',
+ proargtypes => 'internal internal internal int4',
prosrc => 'brin_inclusion_consistent' },
{ oid => '4108', descr => 'BRIN inclusion support',
proname => 'brin_inclusion_union', prorettype => 'bool',
--
2.26.2
--------------22A4B241170149838D4D1F8F
Content-Type: text/x-patch; charset=UTF-8;
name="0002-Move-IS-NOT-NULL-handling-from-BRIN-support-20210215.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="0002-Move-IS-NOT-NULL-handling-from-BRIN-support-20210215.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
@ 2024-03-25 16:07 Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Melanie Plageman @ 2024-03-25 16:07 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Thomas Munro <[email protected]>; Nazir Bilal Yavuz <[email protected]>
On Sun, Mar 24, 2024 at 06:37:20PM -0400, Melanie Plageman wrote:
> On Sun, Mar 24, 2024 at 5:59 PM Tomas Vondra
> <[email protected]> wrote:
> >
> > BTW when you say "up to 'Make table_scan_bitmap_next_block() async
> > friendly'" do you mean including that patch, or that this is the first
> > patch that is not one of the independently useful patches.
>
> I think the code is easier to understand with "Make
> table_scan_bitmap_next_block() async friendly". Prior to that commit,
> table_scan_bitmap_next_block() could return false even when the bitmap
> has more blocks and expects the caller to handle this and invoke it
> again. I think that interface is very confusing. The downside of the
> code in that state is that the code for prefetching is still in the
> BitmapHeapNext() code and the code for getting the current block is in
> the heap AM-specific code. I took a stab at fixing this in v9's 0013,
> but the outcome wasn't very attractive.
>
> What I will do tomorrow is reorder and group the commits such that all
> of the commits that are useful independent of streaming read are first
> (I think 0014 and 0015 are independently valuable but they are on top
> of some things that are only useful to streaming read because they are
> more recently requested changes). I think I can actually do a bit of
> simplification in terms of how many commits there are and what is in
> each. Just to be clear, v9 is still reviewable. I am just going to go
> back and change what is included in each commit.
So, attached v10 does not include the new version of streaming read API.
I focused instead on the refactoring patches commit regrouping I
mentioned here.
I realized "Remove table_scan_bitmap_next_block()" can't easily be moved
down below "Push BitmapHeapScan prefetch code into heapam.c" because we
have to do BitmapAdjustPrefetchTarget() and
BitmapAdjustPrefetchIterator() on either side of getting the next block
(via table_scan_bitmap_next_block()).
"Push BitmapHeapScan prefetch code into heapam.c" isn't very nice
because it adds a lot of bitmapheapscan specific members to
TableScanDescData and HeapScanDescData. I thought about wrapping all of
those members in some kind of BitmapHeapScanTableState struct -- but I
don't like that because the members are spread out across
HeapScanDescData and TableScanDescData so not all of them would go in
BitmapHeapScanTableState. I could move the ones I put in
HeapScanDescData back into TableScanDescData and then wrap that in a
BitmapHeapScanTableState. I haven't done that in this version.
I did manage to move "Unify parallel and serial BitmapHeapScan iterator
interfaces" down below the line of patches which are only useful if
the streaming read user also goes in.
In attached v10, all patches up to and including "Unify parallel and
serial BitmapHeapScan iterator interfaces" (0010) are proposed for
master with or without the streaming read API.
0010 does add additional indirection and thus pointer dereferencing for
accessing the iterators, which doesn't feel good. But, it does simplify
the code.
Perhaps it is worth renaming the existing TableScanDescData->rs_parallel
(a ParallelTableScanDescData) to something like rs_seq_parallel. It is
only for sequential scans and scans of tables when building indexes but
the comments say it is for parallel scans in general. There is a similar
member in HeapScanDescData called rs_parallelworkerdata.
- Melanie
Attachments:
[text/x-diff] v10-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/2-v10-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 62f6c877d130325748bf18724fc8166932605091 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v10 01/17] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index e57a0b7ea3..e78d793f69 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..93fdcd226b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v10-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/3-v10-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From c6ed55265735c2d2a47da1fc6339d3cc1db954e4 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v10 02/17] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226b..c64530674b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v10-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (15.0K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/4-v10-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From 844cfbb2c2700da1fd8a290776523190e9c73ef9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v10 03/17] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index cc67dd813d..6b2863391f 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,6 +948,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1036,6 +1038,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1055,6 +1063,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2b7c702642..7fdccaf613 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2124,6 +2125,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2236,6 +2255,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674b..83d9db8f39 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 368c570a0f..cef54e2d5d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 65834caeb1..1bc5f7c057 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -952,9 +959,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1774c56ae3..6871db9b21 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,10 +1785,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1808,9 +1805,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v10-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.2K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/5-v10-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From 8075018e79311547895034ef25be5294aee9e9fb Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v10 04/17] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39..5df3b5ca46 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v10-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.3K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/6-v10-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 370a50ea551d440476160e083a20ac356bb35c0a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v10 05/17] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46..404de0595e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v10-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch (4.4K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/7-v10-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch)
download | inline diff:
From 1538728ba1424299360e5845afbbf51ecebf5947 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v10 06/17] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 7fdccaf613..849cac3947 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2242,6 +2243,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595e..c95e3412da 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1bc5f7c057..b9ba4f9fb3 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -804,6 +804,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -819,7 +822,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1988,14 +1992,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2006,7 +2012,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v10-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (2.9K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/8-v10-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From b03464c30573a28e1f2e5e0915e75dc26a823a3d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v10 07/17] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da..49938c9ed4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v10-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/9-v10-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From 1c32c668a87649990c072dbdde69f14d8ba6b2ce Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v10 08/17] Remove table_scan_bitmap_next_tuple parameter
tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 849cac3947..cf4387f443 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2250,7 +2250,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed4..282dcb9791 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index b9ba4f9fb3..bcf1497f67 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -795,10 +795,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -829,15 +826,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2025,7 +2017,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2037,7 +2028,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v10-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/10-v10-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 9f6b3738ac4f6648d830168d8659aa49510baecc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v10 09/17] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index cf4387f443..2ad785e511 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,18 +2114,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2144,16 +2177,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2245,7 +2269,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb9791..7e73583fe5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno > node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index bcf1497f67..a820cc8c99 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -788,19 +789,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -819,8 +815,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -957,9 +953,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1019,6 +1019,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1029,6 +1044,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1981,19 +2011,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2003,8 +2031,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6871db9b21..8688bc5ab0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,8 +1783,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1793,9 +1791,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,8 +1803,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1813,9 +1811,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v10-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.0K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/11-v10-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From 64d69f1249731b740c510cd86163517a8b1320ec Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v10 10/17] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext(). Now, on the first scan, the the
iterator is created after the scan descriptor is created.
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 163 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 29 +---
src/include/executor/nodeBitmapHeapscan.h | 10 ++
src/include/nodes/execnodes.h | 8 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2ad785e511..c76849a98e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2133,10 +2133,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7e73583fe5..fe471a8a0c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,6 +56,56 @@ static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ dsa_pointer shared_area,
+ dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(shared_area))
+ result->parallel = tbm_attach_shared_iterate(personal_area, shared_area);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -97,43 +147,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
+ if (init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -154,21 +184,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +216,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ dsa);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ node->pf_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
+ dsa);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -280,7 +311,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ node->pf_iterator &&
node->pfblockno > node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -321,12 +352,11 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
@@ -335,7 +365,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -348,8 +378,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -371,7 +399,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -431,23 +459,22 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (prefetch_iterator)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
node->prefetch_pages++;
@@ -475,8 +502,6 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
if (prefetch_iterator)
{
while (1)
@@ -500,12 +525,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
@@ -572,18 +597,17 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator)
+ {
+ bhs_end_iterate(node->pf_iterator);
+ node->pf_iterator = NULL;
+ }
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -628,12 +652,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (node->pf_iterator)
+ bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -671,11 +693,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
+ scanstate->pf_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..fb22f305bf 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,12 +20,12 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
+struct BitmapHeapIterator;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -44,8 +44,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ struct BitmapHeapIterator *rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a820cc8c99..29387166c1 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -957,8 +957,7 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
+ result->rs_bhs_iterator = NULL;
return result;
}
@@ -1021,17 +1020,8 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1046,17 +1036,8 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caa..cb56d20dc6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,5 +28,15 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8688bc5ab0..52cedd1b35 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1778,6 +1778,8 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+struct BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1786,12 +1788,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1806,12 +1807,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 4679660837..6d5cb0bdaa 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v10-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.2K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/12-v10-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From 8776aa053a556d61303f9f6d4af6ce7bb6732121 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v10 11/17] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c76849a98e..d85fee1e50 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2114,7 +2114,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2264,7 +2265,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index fe471a8a0c..076e1ff674 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -119,7 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -298,14 +297,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 29387166c1..edc54ecffe 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -797,8 +797,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -815,8 +815,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1994,15 +1996,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2013,7 +2017,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v10-0012-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/13-v10-0012-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From c7e25c55056880848dea5f23ef62b859475f8973 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v10 12/17] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fc..1dc4c99bf9 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639b..432fae5296 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v10-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch (20.7K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/14-v10-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch)
download | inline diff:
From 99f7c9224046b021bc82a9f3d48048fd6916a29b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v10 13/17] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 30 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++-------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/executor/nodeBitmapHeapscan.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
8 files changed, 113 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb..3aa457a29e 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544..033d525339 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d85fee1e50..105137396b 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2122,7 +2122,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2134,9 +2134,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(scan->rs_bhs_iterator);
+ bhs_iterate(scan->rs_bhs_iterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2151,11 +2151,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2163,19 +2163,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2204,7 +2204,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2213,9 +2213,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2265,7 +2265,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 076e1ff674..78f79aafff 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -77,15 +77,16 @@ bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_ar
return result;
}
-TBMIterateResult *
-bhs_iterate(BitmapHeapIterator *iterator)
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result)
{
Assert(iterator);
+ Assert(result);
if (iterator->serial)
- return tbm_iterate(iterator->serial);
+ tbm_iterate(iterator->serial, result);
else
- return tbm_shared_iterate(iterator->parallel);
+ tbm_shared_iterate(iterator->parallel, result);
}
void
@@ -348,7 +349,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -360,8 +361,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -394,8 +395,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -462,10 +463,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -473,7 +476,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -482,13 +485,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -501,7 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -520,8 +523,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -529,17 +532,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf9..309a44bdb8 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae..3b432263bb 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index cb56d20dc6..3c330f86e6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -34,7 +34,7 @@ typedef struct BitmapHeapIterator
struct TBMSharedIterator *parallel;
} BitmapHeapIterator;
-extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result);
extern void bhs_end_iterate(BitmapHeapIterator *iterator);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae5296..f000c1af28 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v10-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (31.5K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/15-v10-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From 4d5dcb8783d92a2e2a37ea37bbf346545cd09337 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v10 14/17] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 262 +++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 341 ++--------------------
src/include/access/heapam.h | 17 ++
src/include/access/relscan.h | 8 +
src/include/access/tableam.h | 26 +-
src/include/nodes/execnodes.h | 14 -
7 files changed, 355 insertions(+), 339 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 6b2863391f..3d92fb5135 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,8 +948,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1032,6 +1040,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1044,6 +1058,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1069,6 +1089,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 105137396b..867061325c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,6 +55,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2112,6 +2115,73 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2130,6 +2200,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2270,6 +2342,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->rs_pf_bhs_iterator &&
+ hscan->pfblockno > hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2280,6 +2364,154 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+
+ if (pstate == NULL)
+ {
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2305,6 +2537,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 78f79aafff..187b288e68 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
dsa_pointer shared_area,
@@ -122,7 +118,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -142,7 +137,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
@@ -154,7 +149,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -169,23 +173,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
@@ -216,19 +213,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
- node->pf_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
- dsa);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
+ scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
+ scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
+ dsa);
}
#endif /* USE_PREFETCH */
@@ -243,36 +240,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -296,23 +263,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->pf_iterator &&
- node->pfblockno > node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,219 +289,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -594,22 +334,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator)
- {
- bhs_end_iterate(node->pf_iterator);
- node->pf_iterator = NULL;
- }
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -648,14 +378,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->pf_iterator)
- bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -688,17 +414,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->pf_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -738,13 +458,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -828,7 +541,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index cef54e2d5d..29fdd55893 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -82,6 +82,23 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used in serial BHS
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fb22f305bf..7938b741d6 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@
struct ParallelTableScanDescData;
struct BitmapHeapIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,6 +46,13 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
+ struct BitmapHeapIterator *rs_pf_bhs_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+ /* used to validate BHS prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index edc54ecffe..284ea3d864 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -800,17 +800,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -960,6 +949,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
+ result->rs_pf_bhs_iterator = NULL;
+ result->prefetch_maximum = 0;
+ result->bm_parallel = NULL;
return result;
}
@@ -1024,6 +1016,12 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1040,6 +1038,12 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 52cedd1b35..60916bf0d0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,18 +1785,11 @@ struct BitmapHeapIterator;
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1804,18 +1797,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v10-0015-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/16-v10-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From feec27e011ef4f1f15e65afca2564192b85b17c1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v10 15/17] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 3d92fb5135..8de1a11164 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -319,6 +319,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 867061325c..6af1791faa 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2109,12 +2109,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2148,8 +2142,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2512,30 +2506,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -2917,7 +2924,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index ce637a5a5d..1d6b03d1ca 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -92,8 +92,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 187b288e68..2f9387e51a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -230,44 +230,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
#endif /* USE_PREFETCH */
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 284ea3d864..44d0885d9e 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -788,36 +788,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -1998,44 +1982,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2046,7 +1999,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v10-0016-v7-Streaming-Read-API.patch (56.1K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/17-v10-0016-v7-Streaming-Read-API.patch)
download | inline diff:
From d5f6f3f2e80e9f85d054dfa99b186b24571930da Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 14 Mar 2024 12:59:42 -0400
Subject: [PATCH v10 16/17] v7 Streaming Read API
---
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/streaming_read.c | 659 +++++++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 642 +++++++++++++++-------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/include/storage/bufmgr.h | 45 ++
src/include/storage/streaming_read.h | 52 ++
src/tools/pgindent/typedefs.list | 3 +
10 files changed, 1227 insertions(+), 210 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/streaming_read.c
create mode 100644 src/include/storage/streaming_read.h
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca2..eec03f6f2b 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 0000000000..bcab44c802
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ streaming_read.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 0000000000..39aef2a84a
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'streaming_read.c',
+)
diff --git a/src/backend/storage/aio/streaming_read.c b/src/backend/storage/aio/streaming_read.c
new file mode 100644
index 0000000000..d5c29b750d
--- /dev/null
+++ b/src/backend/storage/aio/streaming_read.c
@@ -0,0 +1,659 @@
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/streaming_read.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+/*
+ * Element type for PgStreamingRead's circular array of block ranges.
+ */
+typedef struct PgStreamingReadRange
+{
+ bool need_wait;
+ bool advice_issued;
+ BlockNumber blocknum;
+ int nblocks;
+ int per_buffer_data_index;
+ Buffer buffers[MAX_BUFFERS_PER_TRANSFER];
+ ReadBuffersOperation operation;
+} PgStreamingReadRange;
+
+/*
+ * Streaming read object.
+ */
+struct PgStreamingRead
+{
+ int max_ios;
+ int ios_in_progress;
+ int max_pinned_buffers;
+ int pinned_buffers;
+ int next_tail_buffer;
+ int distance;
+ bool started;
+ bool finished;
+ bool advice_enabled;
+ void *pgsr_private;
+ PgStreamingReadBufferCB callback;
+
+ BufferAccessStrategy strategy;
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+
+ /* Sometimes we need to buffer one block for flow control. */
+ BlockNumber unget_blocknum;
+ void *unget_per_buffer_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* Space for optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Circular buffer of ranges. */
+ int size;
+ int head;
+ int tail;
+ PgStreamingReadRange ranges[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead.
+ */
+PgStreamingRead *
+pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_data_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb)
+{
+ PgStreamingRead *pgsr;
+ int size;
+ int max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /*
+ * Make sure our bmr's smgr and persistent are populated. The caller
+ * asserts that the storage manager will remain valid.
+ */
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many assumed I/Os we will allow to run concurrently. That
+ * is, advice to the kernel to tell it that we will soon read. This
+ * number also affects how far we look ahead for opportunities to start
+ * more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & PGSR_FLAG_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+
+ /*
+ * Choose a maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least
+ * MAX_BUFFER_PER_TRANSFER so that we can have a chance to build up a full
+ * sized read, even when max_ios is zero.
+ */
+ max_pinned_buffers = Max(max_ios * 4, MAX_BUFFERS_PER_TRANSFER);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * pgsr->ranges is a circular buffer. When it is empty, head == tail.
+ * When it is full, there is an empty element between head and tail. Head
+ * can also be empty (nblocks == 0), therefore we need two extra elements
+ * for non-occupied ranges, on top of max_pinned_buffers to allow for the
+ * maxmimum possible number of occupied ranges of the smallest possible
+ * size of one.
+ */
+ size = max_pinned_buffers + 2;
+
+ pgsr = (PgStreamingRead *)
+ palloc0(offsetof(PgStreamingRead, ranges) +
+ sizeof(pgsr->ranges[0]) * size);
+
+ pgsr->max_ios = max_ios;
+ pgsr->per_buffer_data_size = per_buffer_data_size;
+ pgsr->max_pinned_buffers = max_pinned_buffers;
+ pgsr->pgsr_private = pgsr_private;
+ pgsr->strategy = strategy;
+ pgsr->size = size;
+
+ pgsr->callback = next_block_cb;
+ pgsr->bmr = bmr;
+ pgsr->forknum = forknum;
+
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. As long as direct I/O isn't
+ * enabled, and the caller hasn't promised sequential access, we can use
+ * it.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & PGSR_FLAG_SEQUENTIAL) == 0)
+ pgsr->advice_enabled = true;
+#endif
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out doing full-sized
+ * reads.
+ */
+ if (flags & PGSR_FLAG_FULL)
+ pgsr->distance = Min(MAX_BUFFERS_PER_TRANSFER, pgsr->max_pinned_buffers);
+ else
+ pgsr->distance = 1;
+
+ /*
+ * Space for the callback to store extra data along with each block. Note
+ * that we need one more than max_pinned_buffers, so we can return a
+ * pointer to a slot that can't be overwritten until the next call.
+ */
+ if (per_buffer_data_size)
+ pgsr->per_buffer_data = palloc(per_buffer_data_size * size);
+
+ return pgsr;
+}
+
+/*
+ * Find the per-buffer data index for the Nth block of a range.
+ */
+static int
+get_per_buffer_data_index(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ int result;
+
+ /*
+ * Find slot in the circular buffer of per-buffer data, without using the
+ * expensive % operator.
+ */
+ result = range->per_buffer_data_index + n;
+ while (result >= pgsr->size)
+ result -= pgsr->size;
+ Assert(result == (range->per_buffer_data_index + n) % pgsr->size);
+
+ return result;
+}
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static void *
+get_per_buffer_data_by_index(PgStreamingRead *pgsr, int per_buffer_data_index)
+{
+ return (char *) pgsr->per_buffer_data +
+ pgsr->per_buffer_data_size * per_buffer_data_index;
+}
+
+/*
+ * Return a pointer to the per-buffer data for the Nth block of a range.
+ */
+static void *
+get_per_buffer_data(PgStreamingRead *pgsr, PgStreamingReadRange *range, int n)
+{
+ return get_per_buffer_data_by_index(pgsr,
+ get_per_buffer_data_index(pgsr,
+ range,
+ n));
+}
+
+/*
+ * Start reading the head range, and create a new head range. The new head
+ * range is returned. It may not be empty, if StartReadBuffers() couldn't
+ * start the entire range; in that case the returned range contains the
+ * remaining portion of the range.
+ */
+static PgStreamingReadRange *
+pg_streaming_read_start_head_range(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *head_range;
+ PgStreamingReadRange *new_head_range;
+ int nblocks_pinned;
+ int flags;
+
+ /* Caller should make sure we never exceed max_ios. */
+ Assert((pgsr->ios_in_progress < pgsr->max_ios) ||
+ (pgsr->ios_in_progress == 0 && pgsr->max_ios == 0));
+
+ /* Should only call if the head range has some blocks to read. */
+ head_range = &pgsr->ranges[pgsr->head];
+ Assert(head_range->nblocks > 0);
+
+ /*
+ * If advice hasn't been suppressed, and this system supports it, this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (pgsr->advice_enabled &&
+ pgsr->max_ios > 0 &&
+ pgsr->started &&
+ head_range->blocknum != pgsr->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* Suppress advice on the first call, because it's too late to benefit. */
+ if (!pgsr->started)
+ pgsr->started = true;
+
+ /* We shouldn't be trying to pin more buffers that we're allowed to. */
+ Assert(pgsr->pinned_buffers + head_range->nblocks <= pgsr->max_pinned_buffers);
+
+ /* Start reading as many blocks as we can from the head range. */
+ nblocks_pinned = head_range->nblocks;
+ head_range->need_wait =
+ StartReadBuffers(pgsr->bmr,
+ head_range->buffers,
+ pgsr->forknum,
+ head_range->blocknum,
+ &nblocks_pinned,
+ pgsr->strategy,
+ flags,
+ &head_range->operation);
+
+ Assert(pgsr->pinned_buffers <= pgsr->max_pinned_buffers);
+
+ if (head_range->need_wait && (flags & READ_BUFFERS_ISSUE_ADVICE))
+ {
+ /*
+ * Since we've issued advice, we count an I/O in progress until we
+ * call WaitReadBuffers().
+ */
+ head_range->advice_issued = true;
+ pgsr->ios_in_progress++;
+ Assert(pgsr->ios_in_progress <= pgsr->max_ios);
+ }
+
+ /*
+ * StartReadBuffers() might have pinned fewer blocks than we asked it to,
+ * but always at least one.
+ */
+ Assert(nblocks_pinned <= head_range->nblocks);
+ Assert(nblocks_pinned >= 1);
+ pgsr->pinned_buffers += nblocks_pinned;
+
+ /*
+ * Remember where the next block would be after that, so we can detect
+ * sequential access next time.
+ */
+ pgsr->seq_blocknum = head_range->blocknum + nblocks_pinned;
+
+ /*
+ * Create a new head range. There must be space, because we have enough
+ * elements for every range to hold just one block, up to the pin limit.
+ */
+ Assert(pgsr->size > pgsr->max_pinned_buffers);
+ Assert((pgsr->head + 1) % pgsr->size != pgsr->tail);
+ if (++pgsr->head == pgsr->size)
+ pgsr->head = 0;
+ new_head_range = &pgsr->ranges[pgsr->head];
+ new_head_range->nblocks = 0;
+ new_head_range->advice_issued = false;
+
+ /*
+ * If we didn't manage to start the whole read above, we split the range,
+ * moving the remainder into the new head range.
+ */
+ if (nblocks_pinned < head_range->nblocks)
+ {
+ int nblocks_remaining = head_range->nblocks - nblocks_pinned;
+
+ head_range->nblocks = nblocks_pinned;
+
+ new_head_range->blocknum = head_range->blocknum + nblocks_pinned;
+ new_head_range->nblocks = nblocks_remaining;
+ }
+
+ /* The new range has per-buffer data starting after the previous range. */
+ new_head_range->per_buffer_data_index =
+ get_per_buffer_data_index(pgsr, head_range, nblocks_pinned);
+
+ return new_head_range;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow pg_streaming_unget_block() to work.
+ */
+static BlockNumber
+pg_streaming_get_block(PgStreamingRead *pgsr, void *per_buffer_data)
+{
+ BlockNumber result;
+
+ if (unlikely(pgsr->unget_blocknum != InvalidBlockNumber))
+ {
+ /*
+ * If we had to unget a block, now it is time to return that one
+ * again.
+ */
+ result = pgsr->unget_blocknum;
+ pgsr->unget_blocknum = InvalidBlockNumber;
+
+ /*
+ * The same per_buffer_data element must have been used, and still
+ * contains whatever data the callback wrote into it. So we just
+ * sanity-check that we were called with the value that
+ * pg_streaming_unget_block() pushed back.
+ */
+ Assert(per_buffer_data == pgsr->unget_per_buffer_data);
+ }
+ else
+ {
+ /* Use the installed callback directly. */
+ result = pgsr->callback(pgsr, pgsr->pgsr_private, per_buffer_data);
+ }
+
+ return result;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later. This *must* be called with the
+ * last value returned by pg_streaming_get_block().
+ */
+static void
+pg_streaming_unget_block(PgStreamingRead *pgsr, BlockNumber blocknum, void *per_buffer_data)
+{
+ Assert(pgsr->unget_blocknum == InvalidBlockNumber);
+ pgsr->unget_blocknum = blocknum;
+ pgsr->unget_per_buffer_data = per_buffer_data;
+}
+
+static void
+pg_streaming_read_look_ahead(PgStreamingRead *pgsr)
+{
+ PgStreamingReadRange *range;
+
+ /* If we're finished, don't look ahead. */
+ if (pgsr->finished)
+ return;
+
+ /*
+ * We we've already started the maximum allowed number of I/Os, don't look
+ * ahead. There is a special case for max_ios == 0.
+ */
+ if (pgsr->max_ios > 0 && pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /* Can't pin any more buffers. */
+ if (pgsr->pinned_buffers == pgsr->distance)
+ return;
+
+ /*
+ * Keep trying to add new blocks to the end of the head range while doing
+ * so wouldn't exceed the distance limit.
+ */
+ range = &pgsr->ranges[pgsr->head];
+ while (pgsr->pinned_buffers + range->nblocks < pgsr->distance)
+ {
+ BlockNumber blocknum;
+ void *per_buffer_data;
+
+ /* Do we have a full-sized range? */
+ if (range->nblocks == lengthof(range->buffers))
+ {
+ /* Start as much of it as we can. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /* If we're now at the I/O limit, stop here. */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ return;
+
+ /*
+ * That might have only been partially started, but always
+ * processes at least one so that'll do for now.
+ */
+ Assert(range->nblocks < lengthof(range->buffers));
+ }
+
+ /* Find per-buffer data slot for the next block. */
+ per_buffer_data = get_per_buffer_data(pgsr, range, range->nblocks);
+
+ /* Find out which block the callback wants to read next. */
+ blocknum = pg_streaming_get_block(pgsr, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ pgsr->finished = true;
+ break;
+ }
+
+ /*
+ * Is there a head range that we cannot extend, because the requested
+ * block is not consecutive?
+ */
+ if (range->nblocks > 0 &&
+ range->blocknum + range->nblocks != blocknum)
+ {
+ /* Yes. Start it, so we can begin building a new one. */
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * It's possible that it was only partially started, and we have a
+ * new range with the remainder. Keep starting I/Os until we get
+ * it all out of the way, or we hit the I/O limit.
+ */
+ while (range->nblocks > 0 && pgsr->ios_in_progress < pgsr->max_ios)
+ range = pg_streaming_read_start_head_range(pgsr);
+
+ /*
+ * We do have to worry about I/O capacity running out if the head
+ * range was split. In that case we have to 'unget' the block
+ * returned by the callback.
+ */
+ if (pgsr->ios_in_progress == pgsr->max_ios)
+ {
+ pg_streaming_unget_block(pgsr, blocknum, per_buffer_data);
+ return;
+ }
+ }
+
+ /* If we have a new, empty range, initialize the start block. */
+ if (range->nblocks == 0)
+ range->blocknum = blocknum;
+
+ /* This block extends the range by one. */
+ Assert(range->blocknum + range->nblocks == blocknum);
+ range->nblocks++;
+ };
+
+ /*
+ * Normally we don't start the head range, preferring to give it a chance
+ * to grow to full size once more buffers have been consumed. In cases
+ * where that can't possibly happen, we might as well start the read
+ * immediately.
+ */
+ if ((range->nblocks > 0 && pgsr->finished) ||
+ (range->nblocks == pgsr->distance))
+ pg_streaming_read_start_head_range(pgsr);
+}
+
+Buffer
+pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_data)
+{
+ PgStreamingReadRange *tail_range;
+
+ for (;;)
+ {
+ if (pgsr->tail != pgsr->head)
+ {
+ tail_range = &pgsr->ranges[pgsr->tail];
+
+ /*
+ * Do we need to wait for a ReadBuffers operation to finish before
+ * returning the buffers in this range?
+ */
+ if (tail_range->need_wait)
+ {
+ int distance;
+
+ Assert(pgsr->next_tail_buffer == 0);
+ WaitReadBuffers(&tail_range->operation);
+ tail_range->need_wait = false;
+
+ /*
+ * We don't really know if the kernel generated a physical I/O
+ * when we issued advice, let alone when it finished, but it
+ * has certainly finished now because we've performed the
+ * read.
+ */
+ if (tail_range->advice_issued)
+ {
+
+ Assert(pgsr->ios_in_progress > 0);
+ pgsr->ios_in_progress--;
+
+ /*
+ * Look-ahead distance ramps up rapidly if we're issuing
+ * advice, so we can search for new more I/Os to start.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ else
+ {
+ /*
+ * There is no point in increasing look-ahead distance if
+ * we've already reached the full I/O size, since we're
+ * not issuing advice. Extra distance would only pin more
+ * buffers for no benefit.
+ */
+ if (pgsr->distance > MAX_BUFFERS_PER_TRANSFER)
+ {
+ /*
+ * Look-ahead distance gradually decays to full I/O
+ * size.
+ */
+ pgsr->distance--;
+ }
+ else
+ {
+ /*
+ * Look-ahead distance ramps up rapidly, but not more
+ * that the full I/O size.
+ */
+ distance = pgsr->distance * 2;
+ distance = Min(distance, MAX_BUFFERS_PER_TRANSFER);
+ distance = Min(distance, pgsr->max_pinned_buffers);
+ pgsr->distance = distance;
+ }
+ }
+ }
+ else if (pgsr->next_tail_buffer == 0)
+ {
+ /* No I/O necessary. Look-ahead distance gradually decays. */
+ if (pgsr->distance > 1)
+ pgsr->distance--;
+ }
+
+ /* Are there more buffers available in this range? */
+ if (pgsr->next_tail_buffer < tail_range->nblocks)
+ {
+ int buffer_index;
+ Buffer buffer;
+
+ buffer_index = pgsr->next_tail_buffer++;
+ buffer = tail_range->buffers[buffer_index];
+
+ Assert(BufferIsValid(buffer));
+
+ /* We are giving away ownership of this pinned buffer. */
+ Assert(pgsr->pinned_buffers > 0);
+ pgsr->pinned_buffers--;
+
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(pgsr, tail_range, buffer_index);
+
+ /* We may be able to get another I/O started. */
+ pg_streaming_read_look_ahead(pgsr);
+
+ return buffer;
+ }
+
+ /* Advance tail to next range. */
+ if (++pgsr->tail == pgsr->size)
+ pgsr->tail = 0;
+ pgsr->next_tail_buffer = 0;
+ }
+ else
+ {
+ /*
+ * If tail crashed into head, and head is not empty, then it is
+ * time to start that range. Otherwise, force a look-ahead, to
+ * kick start the stream.
+ */
+ Assert(pgsr->tail == pgsr->head);
+ if (pgsr->ranges[pgsr->head].nblocks > 0)
+ {
+ pg_streaming_read_start_head_range(pgsr);
+ }
+ else
+ {
+ pg_streaming_read_look_ahead(pgsr);
+
+ /* Finished? */
+ if (pgsr->tail == pgsr->head &&
+ pgsr->ranges[pgsr->head].nblocks == 0)
+ break;
+ }
+ }
+ }
+
+ Assert(pgsr->pinned_buffers == 0);
+
+ return InvalidBuffer;
+}
+
+void
+pg_streaming_read_free(PgStreamingRead *pgsr)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ pgsr->finished = true;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = pg_streaming_read_buffer_get_next(pgsr, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(pgsr->pinned_buffers == 0);
+ Assert(pgsr->ios_in_progress == 0);
+
+ /* Release memory. */
+ if (pgsr->per_buffer_data)
+ pfree(pgsr->per_buffer_data);
+
+ pfree(pgsr);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c..d0e9c7deff 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -471,10 +476,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +504,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +785,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +797,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +819,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +990,68 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
+/*
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
+ */
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
+{
+ BufferDesc *bufHdr;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
/*
* ReadBuffer_common -- common logic for all ReadBuffer variants
*
* *hit is set to true if the request was satisfied from shared buffer cache.
*/
static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+ BufferAccessStrategy strategy)
{
- BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
- IOContext io_context;
- IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
-
- *hit = false;
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int nblocks;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1070,405 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
+ nblocks = 1;
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
+ else
+ flags = 0;
+ if (StartReadBuffers(bmr,
+ &buffer,
+ forkNum,
+ blockNum,
+ &nblocks,
+ strategy,
+ flags,
+ &operation))
+ WaitReadBuffers(&operation);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ if (mode == RBM_ZERO_AND_CLEANUP_LOCK || mode == RBM_ZERO_AND_LOCK)
+ ZeroBuffer(buffer, mode);
+
+ return buffer;
+}
+static Buffer
+PrepareReadBuffer(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
io_context = IOCONTEXT_NORMAL;
io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
}
else
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
io_context = IOContextForStrategy(strategy);
io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
}
- /* At this point we do NOT hold any locks. */
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
- /* if it was already in the buffer pool, we're done */
- if (found)
+ ResourceOwnerEnlarge(CurrentResourceOwner);
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
{
- /* Just need to update stats before we exit */
- *hit = true;
VacuumPageHit++;
pgstat_count_io_op(io_object, io_context, IOOP_HIT);
-
if (VacuumCostActive)
VacuumCostBalance += VacuumCostPageHit;
TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
- {
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
- }
+ return BufferDescriptorGetBuffer(bufHdr);
+}
- return BufferDescriptorGetBuffer(bufHdr);
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested.
+ *
+ * If false is returned, no I/O is necessary and WaitReadBuffers() is not
+ * necessary. If true is returned, one I/O has been started, and
+ * WaitReadBuffers() must be called with the same operation object before the
+ * buffers are accessed. Along with the operation object, the caller-supplied
+ * array of buffers must remain valid until WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+bool
+StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation)
+{
+ int actual_nblocks = *nblocks;
+
+ if (bmr.rel)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
}
- /*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
- */
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ operation->bmr = bmr;
+ operation->forknum = forkNum;
+ operation->blocknum = blockNum;
+ operation->buffers = buffers;
+ operation->nblocks = actual_nblocks;
+ operation->strategy = strategy;
+ operation->flags = flags;
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ operation->io_buffers_len = 0;
- /*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
- */
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
- else
+ for (int i = 0; i < actual_nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ bool found;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ buffers[i] = PrepareReadBuffer(bmr,
+ forkNum,
+ blockNum + i,
+ strategy,
+ &found);
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = operation->nblocks = *nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ operation->io_buffers_len++;
+ }
+ }
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ if (operation->io_buffers_len > 0)
+ {
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
- {
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
- }
- else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
+ /*
+ * In theory we should only do this if PrepareReadBuffers() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(bmr.smgr, forkNum, blockNum, operation->io_buffers_len);
}
+
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
}
+ else
+ {
+ return false;
+ }
+}
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
{
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
}
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
+
+ /*
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
+ */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+ bmr = operation->bmr;
+
+ isLocalBuf = SmgrIsTemp(bmr.smgr);
if (isLocalBuf)
{
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
-
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
}
else
{
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
}
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ /*
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
+ */
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
+ else
+ pgBufferUsage.shared_blks_read += nblocks;
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ for (int i = 0; i < nblocks; ++i)
+ {
+ int io_buffers_len;
+ Buffer io_buffers[MAX_BUFFERS_PER_TRANSFER];
+ void *io_pages[MAX_BUFFERS_PER_TRANSFER];
+ instr_time io_start;
+ BlockNumber io_first_block;
- return BufferDescriptorGetBuffer(bufHdr);
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PrepareReadBuffer().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
+
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
+
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
+
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
+ {
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
+ {
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
+ }
+ else
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
+
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(bmr.smgr->smgr_rlocator, forknum))));
+ }
+
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
+
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
+
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for StartReadBuffers. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1476,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1535,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1603,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1638,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +1994,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2259,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2282,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2372,7 +2597,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2611,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3679,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5184,9 +5414,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5435,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a5..985a2c7049 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca2..739d13293f 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d335..b57f71f97e 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -158,6 +159,11 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
+/*
+ * Maximum number of buffers for multi-buffer I/O functions. This is set to
+ * allow 128kB transfers, unless BLCKSZ and IOV_MAX imply a a smaller maximum.
+ */
+#define MAX_BUFFERS_PER_TRANSFER Min(PG_IOV_MAX, (128 * 1024) / BLCKSZ)
/*
* prototypes for functions in bufmgr.c
@@ -177,6 +183,42 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+/*
+ * Private state used by StartReadBuffers() and WaitReadBuffers(). Declared
+ * in public header only to allow inclusion in other structs, but contents
+ * should not be accessed.
+ */
+struct ReadBuffersOperation
+{
+ /* Parameters passed in to StartReadBuffers(). */
+ BufferManagerRelation bmr;
+ Buffer *buffers;
+ ForkNumber forknum;
+ BlockNumber blocknum;
+ int nblocks;
+ BufferAccessStrategy strategy;
+ int flags;
+
+ /* Range of buffers, if we need to perform a read. */
+ int io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffers(BufferManagerRelation bmr,
+ Buffer *buffers,
+ ForkNumber forknum,
+ BlockNumber blocknum,
+ int *nblocks,
+ BufferAccessStrategy strategy,
+ int flags,
+ ReadBuffersOperation *operation);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +292,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/streaming_read.h b/src/include/storage/streaming_read.h
new file mode 100644
index 0000000000..c4d3892bb2
--- /dev/null
+++ b/src/include/storage/streaming_read.h
@@ -0,0 +1,52 @@
+#ifndef STREAMING_READ_H
+#define STREAMING_READ_H
+
+#include "storage/bufmgr.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define PGSR_FLAG_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users.
+ */
+#define PGSR_FLAG_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define PGSR_FLAG_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define PGSR_FLAG_FULL 0x04
+
+struct PgStreamingRead;
+typedef struct PgStreamingRead PgStreamingRead;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*PgStreamingReadBufferCB) (PgStreamingRead *pgsr,
+ void *pgsr_private,
+ void *per_buffer_private);
+
+extern PgStreamingRead *pg_streaming_read_buffer_alloc(int flags,
+ void *pgsr_private,
+ size_t per_buffer_private_size,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ PgStreamingReadBufferCB next_block_cb);
+
+extern void pg_streaming_read_prefetch(PgStreamingRead *pgsr);
+extern Buffer pg_streaming_read_buffer_get_next(PgStreamingRead *pgsr, void **per_buffer_private);
+extern void pg_streaming_read_free(PgStreamingRead *pgsr);
+
+#endif
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 6d5cb0bdaa..4558c2ecfc 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2116,6 +2116,8 @@ PgStat_TableCounts
PgStat_TableStatus
PgStat_TableXactStatus
PgStat_WalStats
+PgStreamingRead
+PgStreamingReadRange
PgXmlErrorContext
PgXmlStrictness
Pg_finfo_record
@@ -2288,6 +2290,7 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
--
2.40.1
[text/x-diff] v10-0017-BitmapHeapScan-uses-streaming-read-API.patch (26.3K, ../../20240325160709.qjo6txzu6zjxzkqy@liskov/18-v10-0017-BitmapHeapScan-uses-streaming-read-API.patch)
download | inline diff:
From fb7740a535532673e50b87014a0b77e565c46af8 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v10 17/17] BitmapHeapScan uses streaming read API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the streaming read API prefetching. Heap table AM implements a
streaming read callback which uses the iterator to get the next valid
block that needs to be fetched for the streaming read API.
ci-os-only:
---
src/backend/access/heap/heapam.c | 96 ++++--
src/backend/access/heap/heapam_handler.c | 347 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 43 +--
src/include/access/heapam.h | 21 +-
src/include/access/relscan.h | 6 -
src/include/access/tableam.h | 14 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 114 insertions(+), 422 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 8de1a11164..5421b552d9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -108,6 +108,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -330,6 +332,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
+ scan->rs_pgsr = pg_streaming_read_buffer_alloc(PGSR_FLAG_DEFAULT,
+ scan,
+ sizeof(TBMIterateResult),
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_pgsr_next);
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -950,16 +968,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
+ scan->rs_pgsr = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1042,12 +1053,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1060,12 +1065,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1091,12 +1090,6 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1114,6 +1107,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_pgsr)
+ pg_streaming_read_free(scan->rs_pgsr);
+
pfree(scan);
}
@@ -10025,3 +10021,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_pgsr_next(PgStreamingRead *pgsr, void *pgsr_private,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) pgsr_private;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_bhs_iterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 6af1791faa..fe9ee5976f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -55,9 +55,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2109,147 +2106,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *io_private;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_pgsr);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_bhs_iterator, &tbmres);
+ hscan->rs_cbuf = pg_streaming_read_buffer_get_next(hscan->rs_pgsr, &io_private);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(io_private);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = io_private;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2270,7 +2188,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2279,9 +2197,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2331,23 +2249,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_bhs_iterator &&
- hscan->pfblockno > hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2358,153 +2264,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2539,41 +2298,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2f9387e51a..f2662ea542 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -131,14 +131,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -149,15 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,13 +157,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -213,22 +190,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- {
- scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
- scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
- dsa);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
}
@@ -525,14 +492,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -563,11 +526,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 29fdd55893..1b8ce82c9e 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/streaming_read.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ PgStreamingRead *rs_pgsr;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -82,23 +86,6 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /*
- * These fields only used for prefetching in bitmap table scans
- */
-
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used in serial BHS
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
-
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 7938b741d6..02893bf99b 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -46,13 +46,7 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
- struct BitmapHeapIterator *rs_pf_bhs_iterator;
-
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate BHS prefetch and current block stay in sync */
- BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 44d0885d9e..b1b09bbac2 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -933,8 +933,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
- result->rs_pf_bhs_iterator = NULL;
- result->prefetch_maximum = 0;
result->bm_parallel = NULL;
return result;
}
@@ -1000,12 +998,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1022,12 +1014,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 60916bf0d0..430668f597 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1758,11 +1758,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1770,10 +1766,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
@ 2024-03-27 19:37 ` Melanie Plageman <[email protected]>
2024-03-28 05:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
0 siblings, 2 replies; 21+ messages in thread
From: Melanie Plageman @ 2024-03-27 19:37 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Thomas Munro <[email protected]>; Nazir Bilal Yavuz <[email protected]>
On Mon, Mar 25, 2024 at 12:07:09PM -0400, Melanie Plageman wrote:
> On Sun, Mar 24, 2024 at 06:37:20PM -0400, Melanie Plageman wrote:
> > On Sun, Mar 24, 2024 at 5:59 PM Tomas Vondra
> > <[email protected]> wrote:
> > >
> > > BTW when you say "up to 'Make table_scan_bitmap_next_block() async
> > > friendly'" do you mean including that patch, or that this is the first
> > > patch that is not one of the independently useful patches.
> >
> > I think the code is easier to understand with "Make
> > table_scan_bitmap_next_block() async friendly". Prior to that commit,
> > table_scan_bitmap_next_block() could return false even when the bitmap
> > has more blocks and expects the caller to handle this and invoke it
> > again. I think that interface is very confusing. The downside of the
> > code in that state is that the code for prefetching is still in the
> > BitmapHeapNext() code and the code for getting the current block is in
> > the heap AM-specific code. I took a stab at fixing this in v9's 0013,
> > but the outcome wasn't very attractive.
> >
> > What I will do tomorrow is reorder and group the commits such that all
> > of the commits that are useful independent of streaming read are first
> > (I think 0014 and 0015 are independently valuable but they are on top
> > of some things that are only useful to streaming read because they are
> > more recently requested changes). I think I can actually do a bit of
> > simplification in terms of how many commits there are and what is in
> > each. Just to be clear, v9 is still reviewable. I am just going to go
> > back and change what is included in each commit.
>
> So, attached v10 does not include the new version of streaming read API.
> I focused instead on the refactoring patches commit regrouping I
> mentioned here.
Attached v11 has the updated Read Stream API Thomas sent this morning
[1]. No other changes.
- Melanie
[1] https://www.postgresql.org/message-id/CA%2BhUKGJTwrS7F%3DuJPx3SeigMiQiW%2BLJaOkjGyZdCntwyMR%3DuAw%40...
Attachments:
[text/x-diff] v11-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240327193750.3mlcmzqondpj27xe@liskov/2-v11-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From c8bdc1f2143adb2d9a5c9f8f69c249b307287a1a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v11 01/17] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 8d3675be95..a254e3175b 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..93fdcd226b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v11-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/3-v11-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 6e41737489dbeb07bc9f0cf9c595c11115cb985b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v11 02/17] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226b..c64530674b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v11-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (15.0K, ../../20240327193750.3mlcmzqondpj27xe@liskov/4-v11-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From 523d1a454c77ccb8734a16b02338145deacd0a60 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v11 03/17] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 2f6527df0d..ed3a3607b7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,6 +948,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1036,6 +1038,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1055,6 +1063,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 6abfe36dec..5ba8cb3657 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2182,6 +2183,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2294,6 +2313,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674b..83d9db8f39 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index f112245373..c7a538221a 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index fc0e702715..ae758857bd 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -62,6 +62,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -963,9 +970,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1774c56ae3..6871db9b21 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,10 +1785,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1808,9 +1805,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v11-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/5-v11-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From 54afa2d047bd10d451c7120ec33e476974534219 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v11 04/17] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39..5df3b5ca46 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v11-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.3K, ../../20240327193750.3mlcmzqondpj27xe@liskov/6-v11-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 227a1ea3d6ceeecc57538a3a4ecb4839fd769445 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v11 05/17] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46..404de0595e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v11-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch (4.4K, ../../20240327193750.3mlcmzqondpj27xe@liskov/7-v11-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch)
download | inline diff:
From 42ad33d1f16aeb9f763a896f8677254292f9b4c2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v11 06/17] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 5ba8cb3657..ce47a158ae 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2172,7 +2172,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2300,6 +2301,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595e..c95e3412da 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index ae758857bd..5c2a7b7422 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -815,6 +815,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -830,7 +833,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2023,14 +2027,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2041,7 +2047,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v11-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (2.9K, ../../20240327193750.3mlcmzqondpj27xe@liskov/8-v11-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From 8fec5f876636383e802dbb10c107892bb71a4d8d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v11 07/17] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da..49938c9ed4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v11-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240327193750.3mlcmzqondpj27xe@liskov/9-v11-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From 836b7922f3d2c85472525d22304436409f06fd90 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v11 08/17] Remove table_scan_bitmap_next_tuple parameter
tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ce47a158ae..ddcdbbaf7e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2308,7 +2308,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed4..282dcb9791 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5c2a7b7422..9c7b8bf162 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -806,10 +806,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -840,15 +837,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2060,7 +2052,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2072,7 +2063,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v11-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.9K, ../../20240327193750.3mlcmzqondpj27xe@liskov/10-v11-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 9affa3100a127c747608e3cd8726696aabe93530 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v11 09/17] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ddcdbbaf7e..196f69e30e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2172,18 +2172,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2202,16 +2235,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2303,7 +2327,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb9791..7e73583fe5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno > node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 9c7b8bf162..68478d16b2 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -21,6 +21,7 @@
#include "access/sdir.h"
#include "access/xact.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -799,19 +800,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -830,8 +826,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -968,9 +964,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1030,6 +1030,21 @@ table_beginscan_analyze(Relation rel)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1040,6 +1055,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -2016,19 +2046,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2038,8 +2066,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 6871db9b21..8688bc5ab0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1783,8 +1783,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1793,9 +1791,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1803,8 +1803,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1813,9 +1811,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v11-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.0K, ../../20240327193750.3mlcmzqondpj27xe@liskov/11-v11-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From 3f3bb4559e8a9862fb6c56c8de0c2e4abc69e5c2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v11 10/17] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext(). Now, on the first scan, the the
iterator is created after the scan descriptor is created.
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 163 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 29 +---
src/include/executor/nodeBitmapHeapscan.h | 10 ++
src/include/nodes/execnodes.h | 8 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 196f69e30e..2d9e0e1a9f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2191,10 +2191,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7e73583fe5..fe471a8a0c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,6 +56,56 @@ static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ dsa_pointer shared_area,
+ dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(shared_area))
+ result->parallel = tbm_attach_shared_iterate(personal_area, shared_area);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -97,43 +147,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
+ if (init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -154,21 +184,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +216,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ dsa);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ node->pf_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
+ dsa);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -280,7 +311,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ node->pf_iterator &&
node->pfblockno > node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -321,12 +352,11 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
@@ -335,7 +365,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -348,8 +378,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -371,7 +399,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -431,23 +459,22 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (prefetch_iterator)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
node->prefetch_pages++;
@@ -475,8 +502,6 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
if (prefetch_iterator)
{
while (1)
@@ -500,12 +525,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
@@ -572,18 +597,17 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator)
+ {
+ bhs_end_iterate(node->pf_iterator);
+ node->pf_iterator = NULL;
+ }
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -628,12 +652,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (node->pf_iterator)
+ bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -671,11 +693,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
+ scanstate->pf_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..fb22f305bf 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,12 +20,12 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
+struct BitmapHeapIterator;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -44,8 +44,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ struct BitmapHeapIterator *rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 68478d16b2..459e123e92 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -968,8 +968,7 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
+ result->rs_bhs_iterator = NULL;
return result;
}
@@ -1032,17 +1031,8 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1057,17 +1047,8 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caa..cb56d20dc6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,5 +28,15 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8688bc5ab0..52cedd1b35 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1778,6 +1778,8 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+struct BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1786,12 +1788,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1806,12 +1807,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index cfa9d5aaea..a6562d19a6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v11-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/12-v11-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From 5621e637fd815e6c7fd5ee492678c943cdac82ea Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v11 11/17] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2d9e0e1a9f..81a7488007 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2172,7 +2172,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2322,7 +2323,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index fe471a8a0c..076e1ff674 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -119,7 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -298,14 +297,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 459e123e92..bb2b79717c 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -808,8 +808,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -826,8 +826,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2029,15 +2031,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2048,7 +2052,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v11-0012-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240327193750.3mlcmzqondpj27xe@liskov/13-v11-0012-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From f457af280f8d84c43e4f6cf5a0cbd76bb06f5f19 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v11 12/17] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fc..1dc4c99bf9 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639b..432fae5296 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v11-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch (20.7K, ../../20240327193750.3mlcmzqondpj27xe@liskov/14-v11-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch)
download | inline diff:
From 7fa873c6bc40682f58193d8bc00848d5e2f5cb16 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v11 13/17] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 30 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++-------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/executor/nodeBitmapHeapscan.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
8 files changed, 113 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb..3aa457a29e 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544..033d525339 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 81a7488007..f7e4d1094d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2180,7 +2180,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2192,9 +2192,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(scan->rs_bhs_iterator);
+ bhs_iterate(scan->rs_bhs_iterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2209,11 +2209,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2221,19 +2221,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2262,7 +2262,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2271,9 +2271,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2323,7 +2323,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 076e1ff674..78f79aafff 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -77,15 +77,16 @@ bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_ar
return result;
}
-TBMIterateResult *
-bhs_iterate(BitmapHeapIterator *iterator)
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result)
{
Assert(iterator);
+ Assert(result);
if (iterator->serial)
- return tbm_iterate(iterator->serial);
+ tbm_iterate(iterator->serial, result);
else
- return tbm_shared_iterate(iterator->parallel);
+ tbm_shared_iterate(iterator->parallel, result);
}
void
@@ -348,7 +349,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -360,8 +361,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -394,8 +395,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -462,10 +463,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -473,7 +476,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -482,13 +485,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -501,7 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -520,8 +523,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -529,17 +532,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf9..309a44bdb8 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae..3b432263bb 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index cb56d20dc6..3c330f86e6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -34,7 +34,7 @@ typedef struct BitmapHeapIterator
struct TBMSharedIterator *parallel;
} BitmapHeapIterator;
-extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result);
extern void bhs_end_iterate(BitmapHeapIterator *iterator);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae5296..f000c1af28 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v11-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (31.5K, ../../20240327193750.3mlcmzqondpj27xe@liskov/15-v11-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From 5eb00d2b400eb58e532a128946a57f79aef6434d Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v11 14/17] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 262 +++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 341 ++--------------------
src/include/access/heapam.h | 17 ++
src/include/access/relscan.h | 8 +
src/include/access/tableam.h | 26 +-
src/include/nodes/execnodes.h | 14 -
7 files changed, 355 insertions(+), 339 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index ed3a3607b7..614d715fc7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,8 +948,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1032,6 +1040,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1044,6 +1058,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1069,6 +1089,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index f7e4d1094d..68bbb6f88c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -61,6 +61,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2170,6 +2173,73 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2188,6 +2258,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2328,6 +2400,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->rs_pf_bhs_iterator &&
+ hscan->pfblockno > hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2338,6 +2422,154 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+
+ if (pstate == NULL)
+ {
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2363,6 +2595,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 78f79aafff..187b288e68 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
dsa_pointer shared_area,
@@ -122,7 +118,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -142,7 +137,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
@@ -154,7 +149,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -169,23 +173,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
@@ -216,19 +213,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
- node->pf_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
- dsa);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
+ scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
+ scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
+ dsa);
}
#endif /* USE_PREFETCH */
@@ -243,36 +240,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -296,23 +263,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->pf_iterator &&
- node->pfblockno > node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,219 +289,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -594,22 +334,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator)
- {
- bhs_end_iterate(node->pf_iterator);
- node->pf_iterator = NULL;
- }
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -648,14 +378,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->pf_iterator)
- bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -688,17 +414,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->pf_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -738,13 +458,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -828,7 +541,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index c7a538221a..4726d31403 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -82,6 +82,23 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used in serial BHS
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fb22f305bf..7938b741d6 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@
struct ParallelTableScanDescData;
struct BitmapHeapIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,6 +46,13 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
+ struct BitmapHeapIterator *rs_pf_bhs_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+ /* used to validate BHS prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index bb2b79717c..799ac013d4 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -811,17 +811,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -971,6 +960,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
+ result->rs_pf_bhs_iterator = NULL;
+ result->prefetch_maximum = 0;
+ result->bm_parallel = NULL;
return result;
}
@@ -1035,6 +1027,12 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1051,6 +1049,12 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 52cedd1b35..60916bf0d0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1785,18 +1785,11 @@ struct BitmapHeapIterator;
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1804,18 +1797,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v11-0015-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../20240327193750.3mlcmzqondpj27xe@liskov/16-v11-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 7ac2a79503d7110c19a7c335fd5ab5422166b6a6 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v11 15/17] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 614d715fc7..5fc052f019 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -319,6 +319,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 68bbb6f88c..80f210b405 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2167,12 +2167,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2206,8 +2200,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2570,30 +2564,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -2975,7 +2982,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index ce637a5a5d..1d6b03d1ca 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -92,8 +92,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 187b288e68..2f9387e51a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -230,44 +230,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
#endif /* USE_PREFETCH */
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 799ac013d4..5979ddee8b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -799,36 +799,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -2033,44 +2017,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2081,7 +2034,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v11-0016-v10-Read-Stream-API.patch (71.2K, ../../20240327193750.3mlcmzqondpj27xe@liskov/17-v11-0016-v10-Read-Stream-API.patch)
download | inline diff:
From 88dde25e1b9a9b340ae57161f098b6351d538f3c Mon Sep 17 00:00:00 2001
From: Thomas Munro <[email protected]>
Date: Mon, 26 Feb 2024 23:48:31 +1300
Subject: [PATCH v11 16/17] v10 Read Stream API
Part 1:
Provide vectored variant of ReadBuffer().
Break ReadBuffer() up into two steps: StartReadBuffers() and
WaitReadBuffers(). This has two advantages:
1. Multiple consecutive blocks can be read with one system call.
2. Advice (hints of future reads) can optionally be issued to the kernel.
The traditional ReadBuffer() function is now implemented in terms of
those functions, to avoid duplication. For now we still only read a
block at a time so there is no change to generated system calls yet, but
later commits will provide infrastructure to help build up larger calls.
Callers should respect the new GUC io_combine_limit, and the limit on
per-backend pins which is now exposed as a public interface.
With some more infrastructure in later work, StartReadBuffers() could
be extended to start real asynchronous I/O instead of advice.
Part 2:
Provide API for streaming relation data.
Introduce an abstraction where relation data can be accessed as a
stream of buffers, with an implementation that is more efficient than
the equivalent sequence of ReadBuffer() calls.
Client code supplies a callback that can say which block number is
wanted next, and then consumes individual buffers one at a time from the
stream. This division allows read_stream.c to build up large calls to
StartReadBuffers() up to io_combine_limit, and issue fadvise() advice
ahead of time in a systematic way when random access is detected.
This API is based on an idea from Andres Freund to pave the way for
asynchronous I/O in future work as required to support direct I/O. The
goal is to have an abstraction that insulates client code from future
changes to the I/O subsystem.
An extended API may be necessary in future for more complicated cases
(for example recovery, whose LsnReadQueue device in xlogprefetcher.c is
a distant cousin of this code that should eventually be replaced by it),
but this basic API is sufficient for many common usage patterns
involving predictable access to a single relation fork.
Author: Thomas Munro <[email protected]>
Author: Andres Freund <[email protected]> (optimization tweaks)
Reviewed-by: Melanie Plageman <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Reviewed-by: Nazir Bilal Yavuz <[email protected]>
Reviewed-by: Dilip Kumar <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Discussion: https://postgr.es/m/CA+hUKGJkOiOCa+mag4BF+zHo7qo=o9CFheB8=g6uT5TUm2gkvA@mail.gmail.com
---
doc/src/sgml/config.sgml | 14 +
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/read_stream.c | 733 ++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 709 +++++++++++------
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 14 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/storage/bufmgr.h | 41 +-
src/include/storage/read_stream.h | 62 ++
src/tools/pgindent/typedefs.list | 3 +
13 files changed, 1386 insertions(+), 227 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/read_stream.c
create mode 100644 src/include/storage/read_stream.h
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 5468637e2e..f3736000ad 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2719,6 +2719,20 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-io-combine-limit" xreflabel="io_combine_limit">
+ <term><varname>io_combine_limit</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>io_combine_limit</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Controls the largest I/O size in operations that combine I/O.
+ The default is 128kB.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-max-worker-processes" xreflabel="max_worker_processes">
<term><varname>max_worker_processes</varname> (<type>integer</type>)
<indexterm>
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca2..eec03f6f2b 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 0000000000..2f29a9ec4d
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ read_stream.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 0000000000..10e1aa3b20
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'read_stream.c',
+)
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
new file mode 100644
index 0000000000..4e293e0df6
--- /dev/null
+++ b/src/backend/storage/aio/read_stream.c
@@ -0,0 +1,733 @@
+/*-------------------------------------------------------------------------
+ *
+ * read_stream.c
+ * Mechanism for accessing buffered relation data with look-ahead
+ *
+ * Code that needs to access relation data typically pins blocks one at a
+ * time, often in a predictable order that might be sequential or data-driven.
+ * Calling the simple ReadBuffer() function for each block is inefficient,
+ * because blocks that are not yet in the buffer pool require I/O operations
+ * that are small and might stall waiting for storage. This mechanism looks
+ * into the future and calls StartReadBuffers() and WaitReadBuffers() to read
+ * neighboring blocks together and ahead of time, with an adaptive look-ahead
+ * distance.
+ *
+ * A user-provided callback generates a stream of block numbers that is used
+ * to form reads of up to io_combine_limit, by attempting to merge them with a
+ * pending read. When that isn't possible, the existing pending read is sent
+ * to StartReadBuffers() so that a new one can begin to form.
+ *
+ * The algorithm for controlling the look-ahead distance tries to classify the
+ * stream into three ideal behaviors:
+ *
+ * A) No I/O is necessary, because the requested blocks are fully cached
+ * already. There is no benefit to looking ahead more than one block, so
+ * distance is 1. This is the default initial assumption.
+ *
+ * B) I/O is necessary, but fadvise is undesirable because the access is
+ * sequential, or impossible because direct I/O is enabled or the system
+ * doesn't support advice. There is no benefit in looking ahead more than
+ * io_combine_limit, because in this case only goal is larger read system
+ * calls. Looking further ahead would pin many buffers and perform
+ * speculative work looking ahead for no benefit.
+ *
+ * C) I/O is necesssary, it appears random, and this system supports fadvise.
+ * We'll look further ahead in order to reach the configured level of I/O
+ * concurrency.
+ *
+ * The distance increases rapidly and decays slowly, so that it moves towards
+ * those levels as different I/O patterns are discovered. For example, a
+ * sequential scan of fully cached data doesn't bother looking ahead, but a
+ * sequential scan that hits a region of uncached blocks will start issuing
+ * increasingly wide read calls until it plateaus at io_combine_limit.
+ *
+ * The main data structure is a circular queue of buffers of size
+ * max_pinned_buffers plus some extra space for technical reasons, ready to be
+ * returned by read_stream_next_buffer(). Each buffer also has an optional
+ * variable sized object that is passed from the callback to the consumer of
+ * buffers.
+ *
+ * Parallel to the queue of buffers, there is a circular queue of in-progress
+ * I/Os that have been started with StartReadBuffers(), and for which
+ * WaitReadBuffers() must be called before returning the buffer.
+ *
+ * For example, if the callback return block numbers 10, 42, 43, 60 in
+ * successive calls, then these data structures might appear as follows:
+ *
+ * buffers buf/data ios
+ *
+ * +----+ +-----+ +--------+
+ * | | | | +----+ 42..44 | <- oldest_io_index
+ * +----+ +-----+ | +--------+
+ * oldest_buffer_index -> | 10 | | ? | | +--+ 60..60 |
+ * +----+ +-----+ | | +--------+
+ * | 42 | | ? |<-+ | | | <- next_io_index
+ * +----+ +-----+ | +--------+
+ * | 43 | | ? | | | |
+ * +----+ +-----+ | +--------+
+ * | 44 | | ? | | | |
+ * +----+ +-----+ | +--------+
+ * | 60 | | ? |<---+
+ * +----+ +-----+
+ * next_buffer_index -> | | | |
+ * +----+ +-----+
+ *
+ * In the example, 5 buffers are pinned, and the next buffer to be streamed to
+ * the client is block 10. Block 10 was a hit and has no associated I/O, but
+ * the range 42..44 requires an I/O wait before its buffers are returned, as
+ * does block 60.
+ *
+ *
+ * Portions Copyright (c) 2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/storage/aio/read_stream.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+#include "storage/read_stream.h"
+#include "utils/memdebug.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+typedef struct InProgressIO
+{
+ int16 buffer_index;
+ ReadBuffersOperation op;
+} InProgressIO;
+
+/*
+ * State for managing a stream of reads.
+ */
+struct ReadStream
+{
+ int16 max_ios;
+ int16 ios_in_progress;
+ int16 queue_size;
+ int16 max_pinned_buffers;
+ int16 pinned_buffers;
+ int16 distance;
+ bool advice_enabled;
+
+ /*
+ * Sometimes we need to be able to 'unget' a block number to resolve a
+ * flow control problem when I/Os are split.
+ */
+ BlockNumber unget_blocknum;
+ bool have_unget_blocknum;
+
+ /*
+ * The callback that will tell us which block numbers to read, and an
+ * opaque pointer that will be pass to it for its own purposes.
+ */
+ ReadStreamBlockNumberCB callback;
+ void *callback_private_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* The read operation we are currently preparing. */
+ BlockNumber pending_read_blocknum;
+ int16 pending_read_nblocks;
+
+ /* Space for buffers and optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Read operations that have been started but not waited for yet. */
+ InProgressIO *ios;
+ int16 oldest_io_index;
+ int16 next_io_index;
+
+ /* Circular queue of buffers. */
+ int16 oldest_buffer_index; /* Next pinned buffer to return */
+ int16 next_buffer_index; /* Index of next buffer to pin */
+ Buffer buffers[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static inline void *
+get_per_buffer_data(ReadStream *stream, int16 buffer_index)
+{
+ return (char *) stream->per_buffer_data +
+ stream->per_buffer_data_size * buffer_index;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow streaming_unget_block() to work.
+ */
+static inline BlockNumber
+read_stream_get_block(ReadStream *stream, void *per_buffer_data)
+{
+ if (!stream->have_unget_blocknum)
+ return stream->callback(stream,
+ stream->callback_private_data,
+ per_buffer_data);
+
+ /*
+ * You can only unget one block, and next_buffer_index can't change across
+ * a get, unget, get sequence, so the callback's per_buffer_data, if any,
+ * is still present in the correct slot. We just have to return the
+ * previous block number.
+ */
+ stream->have_unget_blocknum = false;
+ return stream->unget_blocknum;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later.
+ */
+static inline void
+read_stream_unget_block(ReadStream *stream, BlockNumber blocknum)
+{
+ Assert(!stream->have_unget_blocknum);
+ stream->have_unget_blocknum = true;
+ stream->unget_blocknum = blocknum;
+}
+
+static void
+read_stream_start_pending_read(ReadStream *stream, bool suppress_advice)
+{
+ bool need_wait;
+ int nblocks;
+ int flags;
+ int16 io_index;
+ int16 overflow;
+ int16 buffer_index;
+
+ /* This should only be called with a pending read. */
+ Assert(stream->pending_read_nblocks > 0);
+ Assert(stream->pending_read_nblocks <= io_combine_limit);
+
+ /* We had better not exceed the pin limit by starting this read. */
+ Assert(stream->pinned_buffers + stream->pending_read_nblocks <=
+ stream->max_pinned_buffers);
+
+ /* We had better not be overwriting an existing pinned buffer. */
+ if (stream->pinned_buffers > 0)
+ Assert(stream->next_buffer_index != stream->oldest_buffer_index);
+ else
+ Assert(stream->next_buffer_index == stream->oldest_buffer_index);
+
+ /*
+ * If advice hasn't been suppressed, this system supports it, and this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (!suppress_advice &&
+ stream->advice_enabled &&
+ stream->pending_read_blocknum != stream->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* We say how many blocks we want to read, but may be smaller on return. */
+ buffer_index = stream->next_buffer_index;
+ io_index = stream->next_io_index;
+ nblocks = stream->pending_read_nblocks;
+ need_wait = StartReadBuffers(&stream->ios[io_index].op,
+ &stream->buffers[buffer_index],
+ stream->pending_read_blocknum,
+ &nblocks,
+ flags);
+ stream->pinned_buffers += nblocks;
+
+ /* Remember whether we need to wait before returning this buffer. */
+ if (!need_wait)
+ {
+ /* Look-ahead distance decays, no I/O necessary (behavior A). */
+ if (stream->distance > 1)
+ stream->distance--;
+ }
+ else
+ {
+ /*
+ * Remember to call WaitReadBuffers() before returning head buffer.
+ * Look-ahead distance will be adjusted after waiting.
+ */
+ stream->ios[io_index].buffer_index = buffer_index;
+ if (++stream->next_io_index == stream->max_ios)
+ stream->next_io_index = 0;
+ Assert(stream->ios_in_progress < stream->max_ios);
+ stream->ios_in_progress++;
+ stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+ }
+
+ /*
+ * We gave a contiguous range of buffer space to StartReadBuffers(), but
+ * we want it to wrap around at queue_size. Slide overflowing buffers to
+ * the front of the array.
+ */
+ overflow = (buffer_index + nblocks) - stream->queue_size;
+ if (overflow > 0)
+ memmove(&stream->buffers[0],
+ &stream->buffers[stream->queue_size],
+ sizeof(stream->buffers[0]) * overflow);
+
+ /* Compute location of start of next read, without using % operator. */
+ buffer_index += nblocks;
+ if (buffer_index >= stream->queue_size)
+ buffer_index -= stream->queue_size;
+ Assert(buffer_index >= 0 && buffer_index < stream->queue_size);
+ stream->next_buffer_index = buffer_index;
+
+ /* Adjust the pending read to cover the remaining portion, if any. */
+ stream->pending_read_blocknum += nblocks;
+ stream->pending_read_nblocks -= nblocks;
+}
+
+static void
+read_stream_look_ahead(ReadStream *stream, bool suppress_advice)
+{
+ while (stream->ios_in_progress < stream->max_ios &&
+ stream->pinned_buffers + stream->pending_read_nblocks < stream->distance)
+ {
+ BlockNumber blocknum;
+ int16 buffer_index;
+ void *per_buffer_data;
+
+ if (stream->pending_read_nblocks == io_combine_limit)
+ {
+ read_stream_start_pending_read(stream, suppress_advice);
+ suppress_advice = false;
+ continue;
+ }
+
+ /*
+ * See which block the callback wants next in the stream. We need to
+ * compute the index of the Nth block of the pending read including
+ * wrap-around, but we don't want to use the expensive % operator.
+ */
+ buffer_index = stream->next_buffer_index + stream->pending_read_nblocks;
+ if (buffer_index >= stream->queue_size)
+ buffer_index -= stream->queue_size;
+ Assert(buffer_index >= 0 && buffer_index < stream->queue_size);
+ per_buffer_data = get_per_buffer_data(stream, buffer_index);
+ blocknum = read_stream_get_block(stream, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ stream->distance = 0;
+ break;
+ }
+
+ /* Can we merge it with the pending read? */
+ if (stream->pending_read_nblocks > 0 &&
+ stream->pending_read_blocknum + stream->pending_read_nblocks == blocknum)
+ {
+ stream->pending_read_nblocks++;
+ continue;
+ }
+
+ /* We have to start the pending read before we can build another. */
+ if (stream->pending_read_nblocks > 0)
+ {
+ read_stream_start_pending_read(stream, suppress_advice);
+ suppress_advice = false;
+ if (stream->ios_in_progress == stream->max_ios)
+ {
+ /* And we've hit the limit. Rewind, and stop here. */
+ read_stream_unget_block(stream, blocknum);
+ return;
+ }
+ }
+
+ /* This is the start of a new pending read. */
+ stream->pending_read_blocknum = blocknum;
+ stream->pending_read_nblocks = 1;
+ }
+
+ /*
+ * Normally we don't start the pending read just because we've hit a
+ * limit, preferring to give it another chance to grow to a larger size
+ * once more buffers have been consumed. However, in cases where that
+ * can't possibly happen, we might as well start the read immediately.
+ */
+ if (stream->pending_read_nblocks > 0 &&
+ (stream->distance == stream->pending_read_nblocks ||
+ stream->distance == 0) &&
+ stream->ios_in_progress < stream->max_ios)
+ read_stream_start_pending_read(stream, suppress_advice);
+}
+
+/*
+ * Create a new streaming read object that can be used to perform the
+ * equivalent of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead. The callback should return block numbers or InvalidBlockNumber to
+ * signal end-of-stream, and if per_buffer_data_size is non-zero, it may also
+ * write extra data for each block into the space provided to it. It will
+ * also receive callback_private_data for its own purposes.
+ */
+ReadStream *
+read_stream_begin_relation(int flags,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ ReadStreamBlockNumberCB callback,
+ void *callback_private_data,
+ size_t per_buffer_data_size)
+{
+ ReadStream *stream;
+ size_t size;
+ int16 queue_size;
+ int16 max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /* Make sure our bmr's smgr and persistent are populated. */
+ if (bmr.smgr == NULL)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many I/Os we will allow to run at the same time. That
+ * currently means advice to the kernel to tell it that we will soon read.
+ * This number also affects how far we look ahead for opportunities to
+ * start more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & READ_STREAM_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+ max_ios = Min(max_ios, PG_INT16_MAX);
+
+ /*
+ * Choose the maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least io_combine_limit
+ * so that we can have a chance to build up a full io_combine_limit sized
+ * read, even when max_ios is zero. Be careful not to allow int16 to
+ * overflow (even though that's not possible with the current GUC range
+ * limits), allowing also for the spare entry and the overflow space.
+ */
+ max_pinned_buffers = Max(max_ios * 4, io_combine_limit);
+ max_pinned_buffers = Min(max_pinned_buffers,
+ PG_INT16_MAX - io_combine_limit - 1);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * We need one extra entry for buffers and per-buffer data, because users
+ * of per-buffer data have access to the object until the next call to
+ * read_stream_next_buffer(), so we need a gap between the head and tail
+ * of the queue so that we don't clobber it.
+ */
+ queue_size = max_pinned_buffers + 1;
+
+ /*
+ * Allocate the object, the buffers, the ios and per_data_data space in
+ * one big chunk. Though we have queue_size buffers, we want to be able
+ * to assume that all the buffers for a single read are contiguous (i.e.
+ * don't wrap around halfway through), so we allow temporary overflows of
+ * up to the maximum possible read size by allocating an extra
+ * io_combine_limit - 1 elements.
+ */
+ size = offsetof(ReadStream, buffers);
+ size += sizeof(Buffer) * (queue_size + io_combine_limit - 1);
+ size += sizeof(InProgressIO) * Max(1, max_ios);
+ size += per_buffer_data_size * queue_size;
+ size += MAXIMUM_ALIGNOF * 2;
+ stream = (ReadStream *) palloc(size);
+ memset(stream, 0, offsetof(ReadStream, buffers));
+ stream->ios = (InProgressIO *)
+ MAXALIGN(&stream->buffers[queue_size + io_combine_limit - 1]);
+ if (per_buffer_data_size > 0)
+ stream->per_buffer_data = (void *)
+ MAXALIGN(&stream->ios[Max(1, max_ios)]);
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. We can use it as long as
+ * direct I/O isn't enabled, the caller hasn't promised sequential access
+ * (overriding our detection heuristics), and max_ios hasn't been set to
+ * zero.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & READ_STREAM_SEQUENTIAL) == 0 &&
+ max_ios > 0)
+ stream->advice_enabled = true;
+#endif
+
+ /*
+ * For now, max_ios = 0 is interpreted as max_ios = 1 with advice disabled
+ * above. If we had real asynchronous I/O we might need a slightly
+ * different definition.
+ */
+ if (max_ios == 0)
+ max_ios = 1;
+
+ stream->max_ios = max_ios;
+ stream->per_buffer_data_size = per_buffer_data_size;
+ stream->max_pinned_buffers = max_pinned_buffers;
+ stream->queue_size = queue_size;
+
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+ stream->callback = callback;
+ stream->callback_private_data = callback_private_data;
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out assuming we'll be
+ * doing full io_combine_limit sized reads (behavior B).
+ */
+ if (flags & READ_STREAM_FULL)
+ stream->distance = Min(max_pinned_buffers, io_combine_limit);
+ else
+ stream->distance = 1;
+
+ /*
+ * Since we always currently always access the same relation, we can
+ * initialize parts of the ReadBuffersOperation objects and leave them
+ * that way, to avoid wasting CPU cycles writing to them for each read.
+ */
+ for (int i = 0; i < max_ios; ++i)
+ {
+ stream->ios[i].op.bmr = bmr;
+ stream->ios[i].op.forknum = forknum;
+ stream->ios[i].op.strategy = strategy;
+ }
+
+ return stream;
+}
+
+/*
+ * Pull one pinned buffer out of a stream created with
+ * read_stream_begin_buffered(). Each call returns successive blocks in the
+ * order specified by the callback. If per_buffer_data_size was set to a
+ * non-zero size, *per_buffer_data receives a pointer to the extra per-buffer
+ * data that the callback had a chance to populate, which remains valid until
+ * the next call to read_stream_next_buffer(). When the stream runs out of
+ * data, InvalidBuffer is returned. The caller may decide to end the stream
+ * early at any time by calling read_stream_end_buffered().
+ */
+Buffer
+read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
+{
+ Buffer buffer;
+ int16 oldest_buffer_index;
+
+ /*
+ * A fast path for all-cached scans (behavior A). This is the same as the
+ * usual algorithm, but it is specialized for no I/O and no per-buffer
+ * data, so we can skip the queue management code, stay in the same buffer
+ * slot and use singular StartReadBuffer().
+ */
+ if (likely(per_buffer_data == NULL &&
+ stream->ios_in_progress == 0 &&
+ stream->pinned_buffers == 1 &&
+ stream->distance == 1))
+ {
+ BlockNumber next_blocknum;
+
+ /*
+ * We have a pinned buffer that we need to serve up, but we also want
+ * to probe the next one before we return, just in case we need to
+ * start an I/O. We can re-use the same buffer slot, and an arbitrary
+ * I/O slot since they're all free.
+ */
+ oldest_buffer_index = stream->oldest_buffer_index;
+ Assert((oldest_buffer_index + 1) % stream->queue_size ==
+ stream->next_buffer_index);
+ buffer = stream->buffers[oldest_buffer_index];
+ Assert(buffer != InvalidBuffer);
+ Assert(stream->pending_read_nblocks <= 1);
+ if (unlikely(stream->pending_read_nblocks == 1))
+ {
+ next_blocknum = stream->pending_read_blocknum;
+ stream->pending_read_nblocks = 0;
+ }
+ else
+ next_blocknum = read_stream_get_block(stream, NULL);
+ if (unlikely(next_blocknum == InvalidBlockNumber))
+ {
+ /* End of stream. */
+ stream->distance = 0;
+ stream->next_buffer_index = oldest_buffer_index;
+ /* Pin transferred to caller. */
+ stream->pinned_buffers = 0;
+ return buffer;
+ }
+ /* Call the special single block version, which is marginally faster. */
+ if (unlikely(StartReadBuffer(&stream->ios[0].op,
+ &stream->buffers[oldest_buffer_index],
+ next_blocknum,
+ stream->advice_enabled ?
+ READ_BUFFERS_ISSUE_ADVICE : 0)))
+ {
+ /* I/O needed. We'll take the general path next time. */
+ stream->oldest_io_index = 0;
+ stream->next_io_index = stream->max_ios > 1 ? 1 : 0;
+ stream->ios_in_progress = 1;
+ stream->ios[0].buffer_index = oldest_buffer_index;
+ stream->seq_blocknum = next_blocknum + 1;
+ /* Increase look ahead distance (move towards behavior B/C). */
+ stream->distance = Min(2, stream->max_pinned_buffers);
+ }
+ /* Pin transferred to caller, got another one, no net change. */
+ Assert(stream->pinned_buffers == 1);
+ return buffer;
+ }
+
+ if (stream->pinned_buffers == 0)
+ {
+ Assert(stream->oldest_buffer_index == stream->next_buffer_index);
+
+ /* End of stream reached? */
+ if (stream->distance == 0)
+ return InvalidBuffer;
+
+ /*
+ * The usual order of operations is that we look ahead at the bottom
+ * of this function after potentially finishing an I/O and making
+ * space for more, but if we're just starting up we'll need to crank
+ * the handle to get started.
+ */
+ read_stream_look_ahead(stream, true);
+
+ /* End of stream reached? */
+ if (stream->pinned_buffers == 0)
+ {
+ Assert(stream->distance == 0);
+ return InvalidBuffer;
+ }
+ }
+
+ /* Grab the oldest pinned buffer and associated per-buffer data. */
+ Assert(stream->pinned_buffers > 0);
+ oldest_buffer_index = stream->oldest_buffer_index;
+ Assert(oldest_buffer_index >= 0 &&
+ oldest_buffer_index < stream->queue_size);
+ buffer = stream->buffers[oldest_buffer_index];
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(stream, oldest_buffer_index);
+
+ Assert(BufferIsValid(buffer));
+
+ /* Do we have to wait for an associated I/O first? */
+ if (stream->ios_in_progress > 0 &&
+ stream->ios[stream->oldest_io_index].buffer_index == oldest_buffer_index)
+ {
+ int16 io_index = stream->oldest_io_index;
+ int16 distance;
+
+ /* Sanity check that we still agree on the buffers. */
+ Assert(stream->ios[io_index].op.buffers ==
+ &stream->buffers[oldest_buffer_index]);
+
+ WaitReadBuffers(&stream->ios[io_index].op);
+
+ Assert(stream->ios_in_progress > 0);
+ stream->ios_in_progress--;
+ if (++stream->oldest_io_index == stream->max_ios)
+ stream->oldest_io_index = 0;
+
+ if (stream->ios[io_index].op.flags & READ_BUFFERS_ISSUE_ADVICE)
+ {
+ /* Distance ramps up fast (behavior C). */
+ distance = stream->distance * 2;
+ distance = Min(distance, stream->max_pinned_buffers);
+ stream->distance = distance;
+ }
+ else
+ {
+ /* No advice; move towards io_combine_limit (behavior B). */
+ if (stream->distance > io_combine_limit)
+ {
+ stream->distance--;
+ }
+ else
+ {
+ distance = stream->distance * 2;
+ distance = Min(distance, io_combine_limit);
+ distance = Min(distance, stream->max_pinned_buffers);
+ stream->distance = distance;
+ }
+ }
+ }
+
+#ifdef CLOBBER_FREED_MEMORY
+ /* Clobber old buffer and per-buffer data for debugging purposes. */
+ stream->buffers[oldest_buffer_index] = InvalidBuffer;
+
+ /*
+ * The caller will get access to the per-buffer data, until the next call.
+ * We wipe the one before, which is never occupied because queue_size
+ * allowed one extra element. This will hopefully trip up client code
+ * that is holding a dangling pointer to it.
+ */
+ if (stream->per_buffer_data)
+ wipe_mem(get_per_buffer_data(stream,
+ oldest_buffer_index == 0 ?
+ stream->queue_size - 1 :
+ oldest_buffer_index - 1),
+ stream->per_buffer_data_size);
+#endif
+
+ /* Pin transferred to caller. */
+ Assert(stream->pinned_buffers > 0);
+ stream->pinned_buffers--;
+
+ /* Advance oldest buffer, with wrap-around. */
+ stream->oldest_buffer_index++;
+ if (stream->oldest_buffer_index == stream->queue_size)
+ stream->oldest_buffer_index = 0;
+
+ /* Prepare for the next call. */
+ read_stream_look_ahead(stream, false);
+
+ return buffer;
+}
+
+/*
+ * Release stream resources.
+ */
+void
+read_stream_end(ReadStream *stream)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ stream->distance = 0;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(stream->pinned_buffers == 0);
+ Assert(stream->ios_in_progress == 0);
+
+ /* Release memory. */
+ pfree(stream);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c..577bcf6e5d 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -160,6 +165,9 @@ int checkpoint_flush_after = DEFAULT_CHECKPOINT_FLUSH_AFTER;
int bgwriter_flush_after = DEFAULT_BGWRITER_FLUSH_AFTER;
int backend_flush_after = DEFAULT_BACKEND_FLUSH_AFTER;
+/* Limit on how many blocks should be handled in single I/O operations. */
+int io_combine_limit = DEFAULT_IO_COMBINE_LIMIT;
+
/* local state for LockBufferForCleanup */
static BufferDesc *PinCountWaitBuf = NULL;
@@ -471,10 +479,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +507,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +788,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +800,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +822,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +993,146 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
/*
- * ReadBuffer_common -- common logic for all ReadBuffer variants
- *
- * *hit is set to true if the request was satisfied from shared buffer cache.
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
*/
-static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
- BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
{
BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
+/*
+ * Pin a buffer for a given block. *foundPtr is set to true if the block was
+ * already present, or false if more work is required to either read it in or
+ * zero it.
+ */
+static inline Buffer
+PinBufferForBlock(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
IOContext io_context;
IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
- *hit = false;
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = bmr.relpersistence == RELPERSISTENCE_TEMP;
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(strategy);
+ io_object = IOOBJECT_RELATION;
+ }
+
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
+
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
+ {
+ VacuumPageHit++;
+ pgstat_count_io_op(io_object, io_context, IOOP_HIT);
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageHit;
+
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
+
+ return BufferDescriptorGetBuffer(bufHdr);
+}
+
+/*
+ * ReadBuffer_common -- common logic for all ReadBuffer variants
+ */
+static inline Buffer
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
+ BlockNumber blockNum, ReadBufferMode mode,
+ BufferAccessStrategy strategy)
+{
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1151,359 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
-
- if (isLocalBuf)
+ if (unlikely(mode == RBM_ZERO_AND_CLEANUP_LOCK ||
+ mode == RBM_ZERO_AND_LOCK))
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
- io_context = IOCONTEXT_NORMAL;
- io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
+ bool found;
+
+ if (bmr.smgr == NULL)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ buffer = PinBufferForBlock(bmr, forkNum, blockNum, strategy, &found);
+ ZeroBuffer(buffer, mode);
+ return buffer;
}
+
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
else
+ flags = 0;
+ operation.bmr = bmr;
+ operation.forknum = forkNum;
+ operation.strategy = strategy;
+ if (StartReadBuffer(&operation,
+ &buffer,
+ blockNum,
+ flags))
+ WaitReadBuffers(&operation);
+
+ return buffer;
+}
+
+/*
+ * Single block version of the StartReadBuffers(). This might save a few
+ * instructions when called from another translation unit, if the compiler
+ * inlines the code and specializes for nblocks == 1.
+ */
+bool
+StartReadBuffer(ReadBuffersOperation *operation,
+ Buffer *buffer,
+ BlockNumber blocknum,
+ int flags)
+{
+ int nblocks = 1;
+ bool result;
+
+ result = StartReadBuffers(operation, buffer, blocknum, &nblocks, flags);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ return result;
+}
+
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested. Caller sets some of the
+ * members of operation; see struct definition.
+ *
+ * If false is returned, no I/O is necessary. If true is returned, one I/O
+ * has been started, and WaitReadBuffers() must be called with the same
+ * operation object before the buffers are accessed. Along with the operation
+ * object, the caller-supplied array of buffers must remain valid until
+ * WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+inline bool
+StartReadBuffers(ReadBuffersOperation *operation,
+ Buffer *buffers,
+ BlockNumber blockNum,
+ int *nblocks,
+ int flags)
+{
+ int actual_nblocks = *nblocks;
+ int io_buffers_len = 0;
+
+ Assert(*nblocks > 0);
+ Assert(*nblocks <= MAX_IO_COMBINE_LIMIT);
+
+ if (!operation->bmr.smgr)
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
- io_context = IOContextForStrategy(strategy);
- io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
+ operation->bmr.smgr = RelationGetSmgr(operation->bmr.rel);
+ operation->bmr.relpersistence = operation->bmr.rel->rd_rel->relpersistence;
}
- /* At this point we do NOT hold any locks. */
-
- /* if it was already in the buffer pool, we're done */
- if (found)
+ for (int i = 0; i < actual_nblocks; ++i)
{
- /* Just need to update stats before we exit */
- *hit = true;
- VacuumPageHit++;
- pgstat_count_io_op(io_object, io_context, IOOP_HIT);
+ bool found;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageHit;
+ buffers[i] = PinBufferForBlock(operation->bmr,
+ operation->forknum,
+ blockNum + i,
+ operation->strategy,
+ &found);
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ io_buffers_len++;
+ }
+ }
+ *nblocks = actual_nblocks;
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
+ if (io_buffers_len > 0)
+ {
+ /* Populate information needed for I/O. */
+ operation->buffers = buffers;
+ operation->blocknum = blockNum;
+ operation->flags = flags;
+ operation->nblocks = actual_nblocks;
+ operation->io_buffers_len = io_buffers_len;
+
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
+ /*
+ * In theory we should only do this if PinBufferForBlock() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(operation->bmr.smgr,
+ operation->forknum,
+ blockNum,
+ operation->io_buffers_len);
}
- return BufferDescriptorGetBuffer(bufHdr);
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
+ }
+ else
+ {
+ return false;
}
+}
+
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
+ {
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
+ }
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
/*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
*/
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+
+ isLocalBuf = operation->bmr.relpersistence == RELPERSISTENCE_TEMP;
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
+ }
/*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
*/
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
else
+ pgBufferUsage.shared_blks_read += nblocks;
+
+ for (int i = 0; i < nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ int io_buffers_len;
+ Buffer io_buffers[MAX_IO_COMBINE_LIMIT];
+ void *io_pages[MAX_IO_COMBINE_LIMIT];
+ instr_time io_start;
+ BlockNumber io_first_block;
+
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PinBufferForBlock().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ operation->bmr.smgr->smgr_rlocator.locator.spcOid,
+ operation->bmr.smgr->smgr_rlocator.locator.dbOid,
+ operation->bmr.smgr->smgr_rlocator.locator.relNumber,
+ operation->bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(operation->bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
{
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
}
else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- }
- }
-
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
- {
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
- }
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
- if (isLocalBuf)
- {
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(operation->bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(operation->bmr.smgr->smgr_rlocator, forknum))));
+ }
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
- }
- else
- {
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
- }
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ operation->bmr.smgr->smgr_rlocator.locator.spcOid,
+ operation->bmr.smgr->smgr_rlocator.locator.dbOid,
+ operation->bmr.smgr->smgr_rlocator.locator.relNumber,
+ operation->bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
- return BufferDescriptorGetBuffer(bufHdr);
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for PinBufferForBlock. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1511,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1570,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1638,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1673,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +2029,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2294,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2317,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2193,7 +2453,7 @@ MarkBufferDirty(Buffer buffer)
uint32 old_buf_state;
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
{
@@ -2372,7 +2632,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2646,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3714,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -4560,7 +4825,7 @@ void
ReleaseBuffer(Buffer buffer)
{
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
UnpinLocalBuffer(buffer);
@@ -4627,7 +4892,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
Page page = BufferGetPage(buffer);
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
{
@@ -5184,9 +5449,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5470,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
@@ -5691,7 +5964,7 @@ ResOwnerReleaseBufferPin(Datum res)
/* Like ReleaseBuffer, but don't call ResourceOwnerForgetBuffer */
if (!BufferIsValid(buffer))
- elog(ERROR, "bad buffer ID: %d", buffer);
+ elog(PANIC, "bad buffer ID: %d", buffer);
if (BufferIsLocal(buffer))
UnpinLocalBufferNoOwner(buffer);
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a5..985a2c7049 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca2..739d13293f 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index abd9029451..313e393262 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3112,6 +3112,20 @@ struct config_int ConfigureNamesInt[] =
NULL
},
+ {
+ {"io_combine_limit",
+ PGC_USERSET,
+ RESOURCES_ASYNCHRONOUS,
+ gettext_noop("Limit on the size of data reads and writes."),
+ NULL,
+ GUC_UNIT_BLOCKS
+ },
+ &io_combine_limit,
+ DEFAULT_IO_COMBINE_LIMIT,
+ 1, MAX_IO_COMBINE_LIMIT,
+ NULL, NULL, NULL
+ },
+
{
{"backend_flush_after", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 2244ee52f7..7fa6d5a64c 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -203,6 +203,7 @@
#backend_flush_after = 0 # measured in pages, 0 disables
#effective_io_concurrency = 1 # 1-1000; 0 disables prefetching
#maintenance_io_concurrency = 10 # 1-1000; 0 disables prefetching
+#io_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
#max_worker_processes = 8 # (change requires restart)
#max_parallel_workers_per_gather = 2 # limited by max_parallel_workers
#max_parallel_maintenance_workers = 2 # limited by max_parallel_workers
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d335..241f68c45e 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -133,6 +134,10 @@ extern PGDLLIMPORT bool track_io_timing;
extern PGDLLIMPORT int effective_io_concurrency;
extern PGDLLIMPORT int maintenance_io_concurrency;
+#define MAX_IO_COMBINE_LIMIT PG_IOV_MAX
+#define DEFAULT_IO_COMBINE_LIMIT Min(MAX_IO_COMBINE_LIMIT, (128 * 1024) / BLCKSZ)
+extern PGDLLIMPORT int io_combine_limit;
+
extern PGDLLIMPORT int checkpoint_flush_after;
extern PGDLLIMPORT int backend_flush_after;
extern PGDLLIMPORT int bgwriter_flush_after;
@@ -158,7 +163,6 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
-
/*
* prototypes for functions in bufmgr.c
*/
@@ -177,6 +181,38 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+struct ReadBuffersOperation
+{
+ /* The following members should be set by the caller. */
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+ BufferAccessStrategy strategy;
+
+ /* The following private members should not be accessed directly. */
+ Buffer *buffers;
+ BlockNumber blocknum;
+ int flags;
+ int16 nblocks;
+ int16 io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffer(ReadBuffersOperation *operation,
+ Buffer *buffer,
+ BlockNumber blocknum,
+ int flags);
+extern bool StartReadBuffers(ReadBuffersOperation *operation,
+ Buffer *buffers,
+ BlockNumber blocknum,
+ int *nblocks,
+ int flags);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +286,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
new file mode 100644
index 0000000000..9e5fa2acf1
--- /dev/null
+++ b/src/include/storage/read_stream.h
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * read_stream.h
+ * Mechanism for accessing buffered relation data with look-ahead
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/read_stream.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef READ_STREAM_H
+#define READ_STREAM_H
+
+#include "storage/bufmgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define READ_STREAM_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users, and thus should be governed by maintenance_io_concurrency
+ * instead of effective_io_concurrency. For example, VACUUM or CREATE INDEX.
+ */
+#define READ_STREAM_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define READ_STREAM_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define READ_STREAM_FULL 0x04
+
+struct ReadStream;
+typedef struct ReadStream ReadStream;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*ReadStreamBlockNumberCB) (ReadStream *stream,
+ void *callback_private_data,
+ void *per_buffer_data);
+
+extern ReadStream *read_stream_begin_relation(int flags,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ ReadStreamBlockNumberCB callback,
+ void *callback_private_data,
+ size_t per_buffer_data_size);
+extern Buffer read_stream_next_buffer(ReadStream *stream, void **per_buffer_private);
+extern void read_stream_end(ReadStream *stream);
+
+#endif /* READ_STREAM_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a6562d19a6..16bfdf3444 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1215,6 +1215,7 @@ InjectionPointCacheEntry
InjectionPointEntry
InjectionPointSharedState
InlineCodeBlock
+InProgressIO
InsertStmt
Instrumentation
Int128AggState
@@ -2287,11 +2288,13 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
ReadLocalXLogPageNoWaitPrivate
ReadReplicationSlotCmd
+ReadStream
ReassignOwnedStmt
RecheckForeignScan_function
RecordCacheArrayEntry
--
2.40.1
[text/x-diff] v11-0017-BitmapHeapScan-uses-read-stream-API.patch (26.3K, ../../20240327193750.3mlcmzqondpj27xe@liskov/18-v11-0017-BitmapHeapScan-uses-read-stream-API.patch)
download | inline diff:
From bc1f127ff17a27f5bd9ac6dff96576707bbe1855 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v11 17/17] BitmapHeapScan uses read stream API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the read stream API prefetching. Heap table AM implements a read
stream callback which uses the iterator to get the next valid block that
needs to be fetched for the read stream API.
ci-os-only:
---
src/backend/access/heap/heapam.c | 96 ++++--
src/backend/access/heap/heapam_handler.c | 347 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 43 +--
src/include/access/heapam.h | 21 +-
src/include/access/relscan.h | 6 -
src/include/access/tableam.h | 14 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 114 insertions(+), 422 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 5fc052f019..f7b1012b26 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -108,6 +108,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_stream_read_next(ReadStream *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -330,6 +332,22 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
if (key != NULL && scan->rs_base.rs_nkeys > 0)
memcpy(scan->rs_base.rs_key, key, scan->rs_base.rs_nkeys * sizeof(ScanKeyData));
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
+
+ scan->rs_read_stream = read_stream_begin_relation(READ_STREAM_DEFAULT,
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_stream_read_next,
+ scan,
+ sizeof(TBMIterateResult));
+
+
+ }
+
/*
* Currently, we only have a stats counter for sequential heap scans (but
* e.g for bitmap scans the underlying bitmap index scans will be counted,
@@ -950,16 +968,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
+ scan->rs_read_stream = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1042,12 +1053,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1060,12 +1065,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1091,12 +1090,6 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1114,6 +1107,9 @@ heap_endscan(TableScanDesc sscan)
if (scan->rs_base.rs_flags & SO_TEMP_SNAPSHOT)
UnregisterSnapshot(scan->rs_base.rs_snapshot);
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
+
pfree(scan);
}
@@ -10130,3 +10126,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_stream_read_next(ReadStream *pgsr, void *private_data,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) private_data;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_bhs_iterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 80f210b405..2e17a85d00 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -61,9 +61,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2167,147 +2164,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *per_buffer_data;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_read_stream);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_bhs_iterator, &tbmres);
+ hscan->rs_cbuf = read_stream_next_buffer(hscan->rs_read_stream, &per_buffer_data);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(per_buffer_data);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = per_buffer_data;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2328,7 +2246,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2337,9 +2255,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2389,23 +2307,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_bhs_iterator &&
- hscan->pfblockno > hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2416,153 +2322,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2597,41 +2356,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2f9387e51a..f2662ea542 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -131,14 +131,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -149,15 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,13 +157,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -213,22 +190,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- {
- scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
- scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
- dsa);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
}
@@ -525,14 +492,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -563,11 +526,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4726d31403..bb701cca08 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/read_stream.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ ReadStream *rs_read_stream;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -82,23 +86,6 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /*
- * These fields only used for prefetching in bitmap table scans
- */
-
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used in serial BHS
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
-
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 7938b741d6..02893bf99b 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -46,13 +46,7 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
- struct BitmapHeapIterator *rs_pf_bhs_iterator;
-
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate BHS prefetch and current block stay in sync */
- BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 5979ddee8b..2b24d7441a 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -944,8 +944,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
- result->rs_pf_bhs_iterator = NULL;
- result->prefetch_maximum = 0;
result->bm_parallel = NULL;
return result;
}
@@ -1011,12 +1009,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1033,12 +1025,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 60916bf0d0..430668f597 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1758,11 +1758,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1770,10 +1766,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
@ 2024-03-28 05:20 ` Thomas Munro <[email protected]>
2024-03-28 18:01 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
1 sibling, 1 reply; 21+ messages in thread
From: Thomas Munro @ 2024-03-28 05:20 UTC (permalink / raw)
To: Melanie Plageman <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
With the unexplained but apparently somewhat systematic regression
patterns on certain tests and settings, I wonder if they might be due
to read_stream.c trying to form larger reads, making it a bit lazier.
It tries to see what the next block will be before issuing the
fadvise. I think that means that with small I/O concurrency settings,
there might be contrived access patterns where it loses, and needs
effective_io_concurrency to be set one notch higher to keep up, or
something like that. One way to test that idea would be to run the
tests with io_combine_limit = 1 (meaning 1 block). It issues advise
eagerly when io_combine_limit is reached, so I suppose it should be
exactly as eager as master. The only difference then should be that
it automatically suppresses sequential fadvise calls.
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 05:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
@ 2024-03-28 18:01 ` Tomas Vondra <[email protected]>
2024-03-28 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Tomas Vondra @ 2024-03-28 18:01 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; Melanie Plageman <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On 3/28/24 06:20, Thomas Munro wrote:
> With the unexplained but apparently somewhat systematic regression
> patterns on certain tests and settings, I wonder if they might be due
> to read_stream.c trying to form larger reads, making it a bit lazier.
> It tries to see what the next block will be before issuing the
> fadvise. I think that means that with small I/O concurrency settings,
> there might be contrived access patterns where it loses, and needs
> effective_io_concurrency to be set one notch higher to keep up, or
> something like that.
Yes, I think we've speculated this might be the root cause before, but
IIRC we didn't manage to verify it actually is the problem.
FWIW I don't think the tests use synthetic data, but I don't think it's
particularly contrived.
> One way to test that idea would be to run the
> tests with io_combine_limit = 1 (meaning 1 block). It issues advise
> eagerly when io_combine_limit is reached, so I suppose it should be
> exactly as eager as master. The only difference then should be that
> it automatically suppresses sequential fadvise calls.
Sure, I'll give that a try. What are some good values to test? Perhaps
32 and 1, i.e. the default and "no coalescing"?
If this turns out to be the problem, does that mean we would consider
using a more conservative default value? Is there some "auto tuning" we
could do? For example, could we reduce the value combine limit if we
start not finding buffers in memory, or something like that?
I recognize this may not be possible with buffered I/O, due to not
having any insight into page cache. And maybe it's misguided anyway,
because how would we know if the right response is to increase or reduce
the combine limit?
Anyway, doesn't the combine limit work against the idea that
effective_io_concurrency is "prefetch distance"? With eic=32 I'd expect
we issue prefetch 32 pages ahead, i.e. if we prefetch page X, we should
then process 32 pages before we actually need X (and we expect the page
to already be in memory, thanks to the gap). But with the combine limit
set to 32, is this still true?
I've tried going through read_stream_* to determine how this will
behave, but read_stream_look_ahead/read_stream_start_pending_read does
not make this very clear. I'll have to experiment with some tracing.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 05:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-28 18:01 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
@ 2024-03-28 21:19 ` Thomas Munro <[email protected]>
0 siblings, 0 replies; 21+ messages in thread
From: Thomas Munro @ 2024-03-28 21:19 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Fri, Mar 29, 2024 at 7:01 AM Tomas Vondra
<[email protected]> wrote:
> On 3/28/24 06:20, Thomas Munro wrote:
> > With the unexplained but apparently somewhat systematic regression
> > patterns on certain tests and settings, I wonder if they might be due
> > to read_stream.c trying to form larger reads, making it a bit lazier.
> > It tries to see what the next block will be before issuing the
> > fadvise. I think that means that with small I/O concurrency settings,
> > there might be contrived access patterns where it loses, and needs
> > effective_io_concurrency to be set one notch higher to keep up, or
> > something like that.
>
> Yes, I think we've speculated this might be the root cause before, but
> IIRC we didn't manage to verify it actually is the problem.
Another factor could be the bug in master that allows it to get out of
sync -- can it allow *more* concurrency than it intended to? Or fewer
hints, but somehow that goes faster because of the
stepping-on-kernel-toes problem?
> > One way to test that idea would be to run the
> > tests with io_combine_limit = 1 (meaning 1 block). It issues advise
> > eagerly when io_combine_limit is reached, so I suppose it should be
> > exactly as eager as master. The only difference then should be that
> > it automatically suppresses sequential fadvise calls.
>
> Sure, I'll give that a try. What are some good values to test? Perhaps
> 32 and 1, i.e. the default and "no coalescing"?
Thanks! Yeah. The default is actually 16, computed backwards from
128kB. (Explanation: POSIX requires 16 as minimum IOV_MAX, ie number
of vectors acceptable to writev/readv and related functions, though
actual acceptable number is usually much higher, and it also seems to
be a conservative acceptable number for hardware scatter/gather lists
in various protocols, ie if doing direct I/O, the transfer won't be
chopped up into more than one physical I/O command because the disk
and DMA engine can handle it as a single I/O in theory at least.
Actual limit on random SSDs might be more like 33, a weird number but
that's what I'm seeing; Mr Axboe wrote a nice short article[1] to get
some starting points for terminology on that topic on Linux. Also,
just anecdotally, returns seem to diminish after that with huge
transfers of buffered I/O so it seems like an OK number if you have to
pick one; but IDK, YMMV, subject for future research as direct I/O
grows in relevance, hence GUC.)
> If this turns out to be the problem, does that mean we would consider
> using a more conservative default value? Is there some "auto tuning" we
> could do? For example, could we reduce the value combine limit if we
> start not finding buffers in memory, or something like that?
Hmm, not sure... I like that number for seq scans. I also like
auto-tuning. But it seems to me that *if* the problem is that we're
not allowing ourselves as many concurrent I/Os as master BHS because
we're waiting to see if the next block is consecutive, that might
indicate that the distance needs to be higher so that we can have a
better chance to see the 'edge' (the non-contiguous next block) and
start the I/O, not that the io_combine_limit needs to be lower. But I
could be way off, given the fuzziness on this problem so far...
> Anyway, doesn't the combine limit work against the idea that
> effective_io_concurrency is "prefetch distance"? With eic=32 I'd expect
> we issue prefetch 32 pages ahead, i.e. if we prefetch page X, we should
> then process 32 pages before we actually need X (and we expect the page
> to already be in memory, thanks to the gap). But with the combine limit
> set to 32, is this still true?
Hmm. It's different. (1) Master BHS has prefetch_maximum, which is
indeed directly taken from the eic setting, while read_stream.c is
prepared to look much ahead further than that (potentially as far as
max_pinned_buffers) if it's been useful recently, to find
opportunities to coalesce and start I/O. (2) Master BHS has
prefetch_target to control the look-ahead window, which starts small
and ramps up until it hits prefetch_maximum, while read_stream.c has
distance which goes up and down according to a more complex algorithm
described at the top.
> I've tried going through read_stream_* to determine how this will
> behave, but read_stream_look_ahead/read_stream_start_pending_read does
> not make this very clear. I'll have to experiment with some tracing.
I'm going to try to set up something like your experiment here too,
and figure out some way to visualise or trace what's going on...
The differences come from (1) multi-block I/Os, requiring two separate
numbers: how many blocks ahead we're looking, and how many I/Os are
running, and (2) being more aggressive about trying to reach the
desired I/O level. Let me try to describe the approach again.
"distance" is the size of a window that we're searching for
opportunities to start I/Os. read_stream_look_ahead() will keep
looking ahead until we already have max_ios I/Os running, or we hit
the end of that window. That's the two conditions in the while loop
at the top:
while (stream->ios_in_progress < stream->max_ios &&
stream->pinned_buffers + stream->pending_read_nblocks <
stream->distance)
If that window is not large enough, we won't be able to find enough
I/Os to reach max_ios. So, every time we finish up starting a random
(non-sequential) I/O, we increase the distance, widening the window
until it can hopefully reach the I/O goal. (I call that behaviour C
in the comments and code.) In other words, master BHS can only find
opportunities to start I/Os in a smaller window, and can only reach
the full I/O concurrency target if they are right next to each other
in that window, but read_stream.c will look much further ahead, but
only if that has recently proven to be useful.
If we find I/Os that need doing, but they're all sequential, the
window size moves towards io_combine_limit, because we know that
issuing advice won't help, so there is no point in making the window
wider than one maximum-sized I/O. For example, sequential scans or
bitmap heapscans with lots of consecutive page bits fall into this
pattern. (Behaviour B in the code comments.) This is a pattern that
master BHS doesn't have anything like.
If we find that we don't need to do any I/O, we slowly move the window
size towards 1 (also the initial value) as there is no point in doing
anything special as it can't help. In contrast, master BHS never
shrinks its prefetch_target, it only goes up until it hits eic.
[1] https://kernel.dk/when-2mb-turns-into-512k.pdf
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
@ 2024-03-28 21:43 ` Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
1 sibling, 1 reply; 21+ messages in thread
From: Tomas Vondra @ 2024-03-28 21:43 UTC (permalink / raw)
To: Melanie Plageman <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Thomas Munro <[email protected]>; Nazir Bilal Yavuz <[email protected]>
On 3/27/24 20:37, Melanie Plageman wrote:
> On Mon, Mar 25, 2024 at 12:07:09PM -0400, Melanie Plageman wrote:
>> On Sun, Mar 24, 2024 at 06:37:20PM -0400, Melanie Plageman wrote:
>>> On Sun, Mar 24, 2024 at 5:59 PM Tomas Vondra
>>> <[email protected]> wrote:
>>>>
>>>> BTW when you say "up to 'Make table_scan_bitmap_next_block() async
>>>> friendly'" do you mean including that patch, or that this is the first
>>>> patch that is not one of the independently useful patches.
>>>
>>> I think the code is easier to understand with "Make
>>> table_scan_bitmap_next_block() async friendly". Prior to that commit,
>>> table_scan_bitmap_next_block() could return false even when the bitmap
>>> has more blocks and expects the caller to handle this and invoke it
>>> again. I think that interface is very confusing. The downside of the
>>> code in that state is that the code for prefetching is still in the
>>> BitmapHeapNext() code and the code for getting the current block is in
>>> the heap AM-specific code. I took a stab at fixing this in v9's 0013,
>>> but the outcome wasn't very attractive.
>>>
>>> What I will do tomorrow is reorder and group the commits such that all
>>> of the commits that are useful independent of streaming read are first
>>> (I think 0014 and 0015 are independently valuable but they are on top
>>> of some things that are only useful to streaming read because they are
>>> more recently requested changes). I think I can actually do a bit of
>>> simplification in terms of how many commits there are and what is in
>>> each. Just to be clear, v9 is still reviewable. I am just going to go
>>> back and change what is included in each commit.
>>
>> So, attached v10 does not include the new version of streaming read API.
>> I focused instead on the refactoring patches commit regrouping I
>> mentioned here.
>
> Attached v11 has the updated Read Stream API Thomas sent this morning
> [1]. No other changes.
>
I think there's some sort of bug, triggering this assert in heapam
Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
I haven't looked for the root cause, and it's not exactly deterministic,
but try this:
create table t (a int, b text);
insert into t select 10000 * random(), md5(i::text)
from generate_series(1,10000000) s(i);^C
create index on t (a);
explain analyze select * from t where a = 200;
explain analyze select * from t where a < 200;
and then vary the condition a bit (different values, inequalities,
etc.). For me it hits the assert in a couple tries.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
0x000079ff48a73d37 in epoll_wait () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.37-18.fc38.x86_64 libgcc-13.2.1-4.fc38.x86_64 libicu-72.1-2.fc38.x86_64 libstdc++-13.2.1-4.fc38.x86_64 zlib-1.2.13-3.fc38.x86_64
(gdb) c
Continuing.
Program received signal SIGABRT, Aborted.
0x000079ff489ef884 in __pthread_kill_implementation () from /lib64/libc.so.6
(gdb) bt
#0 0x000079ff489ef884 in __pthread_kill_implementation () from /lib64/libc.so.6
#1 0x000079ff4899eafe in raise () from /lib64/libc.so.6
#2 0x000079ff4898787f in abort () from /lib64/libc.so.6
#3 0x00000000009ba563 in ExceptionalCondition (conditionName=conditionName@entry=0xa209e0 "BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno", fileName=fileName@entry=0xa205b4 "heapam_handler.c", lineNumber=lineNumber@entry=2221) at assert.c:66
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
#5 heapam_scan_bitmap_next_tuple (scan=0x1e73548, slot=<optimized out>, recheck=<optimized out>, lossy_pages=<optimized out>, exact_pages=<optimized out>) at heapam_handler.c:2359
#6 0x00000000006f58bb in table_scan_bitmap_next_tuple (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, slot=<optimized out>, scan=<optimized out>) at ../../../src/include/access/tableam.h:2022
#7 BitmapHeapNext (node=0x1e71fc8) at nodeBitmapHeapscan.c:202
#8 0x00000000006e46b8 in ExecProcNodeInstr (node=0x1e71fc8) at execProcnode.c:480
#9 0x00000000006dd6fa in ExecProcNode (node=0x1e71fc8) at ../../../src/include/executor/executor.h:274
#10 ExecutePlan (execute_once=<optimized out>, dest=0xb755e0 <donothingDR>, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0x1e71fc8, estate=0x1e71d80) at execMain.c:1644
#11 standard_ExecutorRun (queryDesc=0x1e6c500, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
#12 0x000000000067af84 in ExplainOnePlan (plannedstmt=plannedstmt@entry=0x1e6c3f0, into=into@entry=0x0, es=es@entry=0x1db3138, queryString=queryString@entry=0x1d87f60 "explain analyze select * from t where a = 200;", params=params@entry=0x0, queryEnv=queryEnv@entry=0x0,
planduration=0x7ffcbe111ac8, bufusage=0x0, mem_counters=0x0) at explain.c:645
#13 0x000000000067b417 in standard_ExplainOneQuery (query=<optimized out>, cursorOptions=2048, into=0x0, es=0x1db3138, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", params=0x0, queryEnv=0x0) at explain.c:459
#14 0x000000000067badb in ExplainQuery (pstate=0x1db3028, stmt=0x1d88e10, params=0x0, dest=0x1db2f98) at ../../../src/include/nodes/nodes.h:172
#15 0x0000000000894026 in standard_ProcessUtility (pstmt=0x1d88f60, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", readOnlyTree=<optimized out>, context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0, dest=0x1db2f98, qc=0x7ffcbe111d80)
at utility.c:863
#16 0x00000000008923cf in PortalRunUtility (portal=portal@entry=0x1e04680, pstmt=0x1d88f60, isTopLevel=true, setHoldSnapshot=setHoldSnapshot@entry=true, dest=dest@entry=0x1db2f98, qc=qc@entry=0x7ffcbe111d80) at pquery.c:1158
#17 0x0000000000892850 in FillPortalStore (portal=portal@entry=0x1e04680, isTopLevel=isTopLevel@entry=true) at ../../../src/include/nodes/nodes.h:172
#18 0x0000000000892b6d in PortalRun (portal=portal@entry=0x1e04680, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0x1dbb518, altdest=altdest@entry=0x1dbb518, qc=0x7ffcbe111f60) at pquery.c:763
#19 0x000000000088ec7f in exec_simple_query (query_string=0x1d87f60 "explain analyze select * from t where a = 200;") at postgres.c:1274
#20 0x000000000089050e in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
#21 0x000000000088b65d in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
#22 0x00000000007f52e5 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffcbe1123ac "", startup_data_len=startup_data_len@entry=4, client_sock=client_sock@entry=0x7ffcbe1123b0) at launch_backend.c:265
#23 0x00000000007f8dbe in BackendStartup (client_sock=0x7ffcbe1123b0) at postmaster.c:3593
#24 ServerLoop () at postmaster.c:1674
#25 0x00000000007fa8f8 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x1d816a0) at postmaster.c:1372
#26 0x000000000051b499 in main (argc=3, argv=0x1d816a0) at main.c:197
(gdb) up
#1 0x000079ff4899eafe in raise () from /lib64/libc.so.6
(gdb) up
#2 0x000079ff4898787f in abort () from /lib64/libc.so.6
(gdb) up
#3 0x00000000009ba563 in ExceptionalCondition (conditionName=conditionName@entry=0xa209e0 "BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno", fileName=fileName@entry=0xa205b4 "heapam_handler.c", lineNumber=lineNumber@entry=2221) at assert.c:66
66 abort();
(gdb) up
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
2221 Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
(gdb) up
#5 heapam_scan_bitmap_next_tuple (scan=0x1e73548, slot=<optimized out>, recheck=<optimized out>, lossy_pages=<optimized out>, exact_pages=<optimized out>) at heapam_handler.c:2359
2359 if (!heapam_scan_bitmap_next_block(scan, recheck,
(gdb) down
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
2221 Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
(gdb) p hscan->rs_cbuf
$1 = 6079
(gdb) p tmbres->blockno
No symbol "tmbres" in current context.
(gdb) p tmbres->blckno
No symbol "tmbres" in current context.
(gdb) p tbmres->blckno
There is no member named blckno.
(gdb) p tbmres->blockno
$2 = 66433
(gdb) bt full
#0 0x000079ff489ef884 in __pthread_kill_implementation () from /lib64/libc.so.6
No symbol table info available.
#1 0x000079ff4899eafe in raise () from /lib64/libc.so.6
No symbol table info available.
#2 0x000079ff4898787f in abort () from /lib64/libc.so.6
No symbol table info available.
#3 0x00000000009ba563 in ExceptionalCondition (conditionName=conditionName@entry=0xa209e0 "BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno", fileName=fileName@entry=0xa205b4 "heapam_handler.c", lineNumber=lineNumber@entry=2221) at assert.c:66
No locals.
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
hscan = 0x1e73548
per_buffer_data = 0x1e793d8
block = <optimized out>
buffer = <optimized out>
snapshot = <optimized out>
ntup = <optimized out>
tbmres = 0x1e793d8
hscan = <optimized out>
per_buffer_data = <optimized out>
block = <optimized out>
buffer = <optimized out>
snapshot = <optimized out>
ntup = <optimized out>
tbmres = <optimized out>
curslot = <optimized out>
offnum = <optimized out>
tid = <optimized out>
heapTuple = <optimized out>
page = <optimized out>
maxoff = <optimized out>
offnum = <optimized out>
lp = <optimized out>
loctup = <optimized out>
valid = <optimized out>
#5 heapam_scan_bitmap_next_tuple (scan=0x1e73548, slot=<optimized out>, recheck=<optimized out>, lossy_pages=<optimized out>, exact_pages=<optimized out>) at heapam_handler.c:2359
hscan = 0x1e73548
targoffset = <optimized out>
page = <optimized out>
lp = <optimized out>
#6 0x00000000006f58bb in table_scan_bitmap_next_tuple (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, slot=<optimized out>, scan=<optimized out>) at ../../../src/include/access/tableam.h:2022
__func__ = "table_scan_bitmap_next_tuple"
__errno_location = <optimized out>
#7 BitmapHeapNext (node=0x1e71fc8) at nodeBitmapHeapscan.c:202
econtext = 0x1e721d8
scan = <optimized out>
tbm = <optimized out>
slot = 0x1e729e8
dsa = <optimized out>
__func__ = "BitmapHeapNext"
#8 0x00000000006e46b8 in ExecProcNodeInstr (node=0x1e71fc8) at execProcnode.c:480
result = <optimized out>
#9 0x00000000006dd6fa in ExecProcNode (node=0x1e71fc8) at ../../../src/include/executor/executor.h:274
No locals.
#10 ExecutePlan (execute_once=<optimized out>, dest=0xb755e0 <donothingDR>, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0x1e71fc8, estate=0x1e71d80) at execMain.c:1644
slot = <optimized out>
current_tuple_count = 812
slot = <optimized out>
current_tuple_count = <optimized out>
#11 standard_ExecutorRun (queryDesc=0x1e6c500, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
estate = 0x1e71d80
operation = CMD_SELECT
dest = 0xb755e0 <donothingDR>
sendTuples = <optimized out>
oldcontext = 0x1db2df0
__func__ = "standard_ExecutorRun"
#12 0x000000000067af84 in ExplainOnePlan (plannedstmt=plannedstmt@entry=0x1e6c3f0, into=into@entry=0x0, es=es@entry=0x1db3138, queryString=queryString@entry=0x1d87f60 "explain analyze select * from t where a = 200;", params=params@entry=0x0, queryEnv=queryEnv@entry=0x0,
planduration=0x7ffcbe111ac8, bufusage=0x0, mem_counters=0x0) at explain.c:645
dir = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
dest = <optimized out>
queryDesc = 0x1e6c500
starttime = <optimized out>
totaltime = 0
eflags = <optimized out>
instrument_option = <optimized out>
#13 0x000000000067b417 in standard_ExplainOneQuery (query=<optimized out>, cursorOptions=2048, into=0x0, es=0x1db3138, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", params=0x0, queryEnv=0x0) at explain.c:459
plan = 0x1e6c3f0
planstart = <optimized out>
planduration = {ticks = 314825}
bufusage_start = {shared_blks_hit = 1, shared_blks_read = 134137333210088, shared_blks_dirtied = 16407, shared_blks_written = 8868755, local_blks_hit = 70467528441856, local_blks_read = 72057594037927936, local_blks_dirtied = 134137333210088,
local_blks_written = 1, temp_blks_read = 0, temp_blks_written = 10151148, shared_blk_read_time = {ticks = 0}, shared_blk_write_time = {ticks = 5433722}, local_blk_read_time = {ticks = 134137333210088}, local_blk_write_time = {ticks = 70368744194071},
temp_blk_read_time = {ticks = 30970240}, temp_blk_write_time = {ticks = 8648921}}
bufusage = {shared_blks_hit = 387345, shared_blks_read = 306578389, shared_blks_dirtied = 0, shared_blks_written = 0, local_blks_hit = 0, local_blks_read = 201863462913, local_blks_dirtied = 65, local_blks_written = 7, temp_blks_read = 5,
temp_blks_written = 10379061, shared_blk_read_time = {ticks = 0}, shared_blk_write_time = {ticks = 7608353}, local_blk_read_time = {ticks = 0}, local_blk_write_time = {ticks = 30969968}, temp_blk_read_time = {ticks = 0}, temp_blk_write_time = {ticks = 7609255}}
mem_counters = {nblocks = 0, freechunks = 8876259, totalspace = 70467528441856, freespace = 72057594037927936}
planner_ctx = 0x0
saved_ctx = 0x0
#14 0x000000000067badb in ExplainQuery (pstate=0x1db3028, stmt=0x1d88e10, params=0x0, dest=0x1db2f98) at ../../../src/include/nodes/nodes.h:172
l__state = {l = <optimized out>, i = 0}
l = 0x1e74140
es = 0x1db3138
tstate = <optimized out>
jstate = <optimized out>
query = <optimized out>
rewritten = 0x1e74128
lc = <optimized out>
timing_set = <optimized out>
summary_set = <optimized out>
__func__ = "ExplainQuery"
#15 0x0000000000894026 in standard_ProcessUtility (pstmt=0x1d88f60, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", readOnlyTree=<optimized out>, context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0, dest=0x1db2f98, qc=0x7ffcbe111d80)
at utility.c:863
parsetree = 0x1d88e10
isTopLevel = <optimized out>
isAtomicContext = false
pstate = 0x1db3028
readonly_flags = <optimized out>
__func__ = "standard_ProcessUtility"
#16 0x00000000008923cf in PortalRunUtility (portal=portal@entry=0x1e04680, pstmt=0x1d88f60, isTopLevel=true, setHoldSnapshot=setHoldSnapshot@entry=true, dest=dest@entry=0x1db2f98, qc=qc@entry=0x7ffcbe111d80) at pquery.c:1158
No locals.
#17 0x0000000000892850 in FillPortalStore (portal=portal@entry=0x1e04680, isTopLevel=isTopLevel@entry=true) at ../../../src/include/nodes/nodes.h:172
treceiver = 0x1db2f98
qc = {commandTag = CMDTAG_UNKNOWN, nprocessed = 0}
__func__ = "FillPortalStore"
#18 0x0000000000892b6d in PortalRun (portal=portal@entry=0x1e04680, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0x1dbb518, altdest=altdest@entry=0x1dbb518, qc=0x7ffcbe111f60) at pquery.c:763
_save_exception_stack = 0x7ffcbe112230
_save_context_stack = 0x0
_local_sigjmp_buf = {{__jmpbuf = {31149072, -109934348456944260, 30969640, 140723497279328, 31475328, 31175960, 108104918559890812, -109935865491088004}, __mask_was_saved = 0, __saved_mask = {__val = {31486888, 2817148525, 0, 0, 10501697, 0, 31483536, 12414709,
31475328, 12414709, 1, 153, 30969616, 31176984, 10382623, 31475328}}}}
_do_rethrow = <optimized out>
result = <optimized out>
nprocessed = <optimized out>
saveTopTransactionResourceOwner = 0x1dc4018
saveTopTransactionContext = 0x1db4c10
saveActivePortal = 0x0
saveResourceOwner = 0x1dc4018
savePortalContext = 0x0
saveMemoryContext = 0x1db4c10
__func__ = "PortalRun"
#19 0x000000000088ec7f in exec_simple_query (query_string=0x1d87f60 "explain analyze select * from t where a = 200;") at postgres.c:1274
cmdtaglen = 7
snapshot_set = <optimized out>
per_parsetree_context = 0x0
plantree_list = <optimized out>
parsetree = <optimized out>
commandTag = <optimized out>
qc = {commandTag = CMDTAG_UNKNOWN, nprocessed = 0}
--Type <RET> for more, q to quit, c to continue without paging--
querytree_list = <optimized out>
portal = 0x1e04680
receiver = 0x1dbb518
format = 0
cmdtagname = <optimized out>
parsetree_item__state = {l = 0x1d88f10, i = <optimized out>}
dest = DestRemote
oldcontext = 0x1db4c10
parsetree_list = 0x1d88f10
parsetree_item = <optimized out>
save_log_statement_stats = false
was_logged = false
use_implicit_block = false
msec_str = '\000' <repeats 16 times>, "Q\000\000\000\000\000\000\000Q\000\000\000\000\000\000"
__func__ = "exec_simple_query"
#20 0x000000000089050e in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
query_string = 0x1d87f60 "explain analyze select * from t where a = 200;"
firstchar = <optimized out>
input_message = {data = 0x1d87f60 "explain analyze select * from t where a = 200;", len = 47, maxlen = 1024, cursor = 47}
local_sigjmp_buf = {{__jmpbuf = {31205072, -109934379334361732, 4, 0, 1711661613, 30973184, 108104918503267708, -109935863615054468}, __mask_was_saved = 1, __saved_mask = {__val = {4194304, 979, 18446744073709551552, 15624, 30973184, 140723497280224,
134137342057842, 0, 7944937237029841664, 140723497280304, 15616, 30938944, 0, 15680, 10344002, 30938944}}}}
send_ready_for_query = false
idle_in_transaction_timeout_enabled = false
idle_session_timeout_enabled = false
__func__ = "PostgresMain"
#21 0x000000000088b65d in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
bsdata = <optimized out>
#22 0x00000000007f52e5 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffcbe1123ac "", startup_data_len=startup_data_len@entry=4, client_sock=client_sock@entry=0x7ffcbe1123b0) at launch_backend.c:265
pid = <optimized out>
#23 0x00000000007f8dbe in BackendStartup (client_sock=0x7ffcbe1123b0) at postmaster.c:3593
bn = 0x1d89d00
pid = <optimized out>
startup_data = {canAcceptConnections = CAC_OK}
bn = <optimized out>
pid = <optimized out>
startup_data = <optimized out>
__func__ = "BackendStartup"
__errno_location = <optimized out>
__errno_location = <optimized out>
save_errno = <optimized out>
__errno_location = <optimized out>
__errno_location = <optimized out>
#24 ServerLoop () at postmaster.c:1674
s = {sock = 10, raddr = {addr = {ss_family = 1,
__ss_padding = "\000\000\000\000\000\000\360\207\330\001\000\000\000\000\b\344\225H\377y\000\000\000\000\000\000\000\000\000\000H\374\225H\377y\000\000\350$\021\276\374\177\000\000\000\000\000\000\374\177\000\000\024$\021\276\374\177\000\000\250\333\371\003\000\000\000\000\320cA\000\000\000\000\000\032\\E\000\000\000\000\000`d\327", '\000' <repeats 13 times>, "\b\344\225H\377y\000\000\320\374\225H\377y\000", __ss_align = 134137344681544}, salen = 2}}
i = 0
now = <optimized out>
last_lockfile_recheck_time = 1711661738
last_touch_time = 1711661613
events = {{pos = 3, events = 2, fd = 8, user_data = 0x0}, {pos = 0, events = 0, fd = -1106171568, user_data = 0x79ff4895fcd0}, {pos = 1221060168, events = 31231, fd = 4544639, user_data = 0x7c96e577}, {pos = -1106171472, events = 32764, fd = -1106171504,
user_data = 0x79ff48c535dc <_dl_lookup_symbol_x+300>}, {pos = 5, events = 0, fd = 1217789136, user_data = 0x1}, {pos = 0, events = 0, fd = 1, user_data = 0x79ff48c7e2c0}, {pos = -1106171632, events = 32764, fd = 1218436466, user_data = 0x1d857b0}, {
pos = 30965736, events = 0, fd = 0, user_data = 0x79ff48c7e648}, {pos = -1106171600, events = 32764, fd = 1, user_data = 0x79ff48c7e2c0}, {pos = -1106171608, events = 32764, fd = 9, user_data = 0xffffffff}, {pos = 1217855464, events = 31231, fd = 1220838640,
user_data = 0xca10}, {pos = 2100392, events = 0, fd = 2, user_data = 0x1000041c0}, {pos = 1000, events = 0, fd = 1221059264, user_data = 0xf2}, {pos = 4776592, events = 0, fd = 13948816, user_data = 0x1}, {pos = -1106171408, events = 32764, fd = 1220910653,
user_data = 0x1}, {pos = 0, events = 0, fd = 1217855464, user_data = 0x79ff48a3bee0 <fork>}, {pos = 0, events = 0, fd = -1106170496, user_data = 0x0}, {pos = 0, events = 0, fd = 30965736, user_data = 0x1}, {pos = 0, events = 0, fd = 1220920014,
user_data = 0x0}, {pos = 1218398648, events = 31231, fd = 0, user_data = 0xd6a120 <BlockSig>}, {pos = 2, events = 0, fd = 30955440, user_data = 0x10}, {pos = 1219714176, events = 31231, fd = 255, user_data = 0xffffffffffffffc0}, {pos = 0, events = 0,
fd = 8096, user_data = 0x7ffcbe112aa0}, {pos = 1218436466, events = 31231, fd = 1217889328, user_data = 0x79ff489e0b50 <fputc>}, {pos = 30955440, events = 0, fd = 4096, user_data = 0x3ff}, {pos = 0, events = 0, fd = -1106171024,
user_data = 0x79ff489d7869 <_IO_file_doallocate+185>}, {pos = 51728, events = 0, fd = 2100403, user_data = 0x1}, {pos = 33152, events = 1000, fd = 1000, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 65535,
user_data = 0xff0000000000ffff}, {pos = 65535, events = 4278190080, fd = 65535, user_data = 0x79ff48b35ce0 <main_arena+96>}, {pos = 0, events = 0, fd = 0, user_data = 0x100000000000000}, {pos = 0, events = 0, fd = 255, user_data = 0x5c5c5c5c5c5c5c5c}, {
pos = 1549556828, events = 1549556828, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x3f3f3f3f3f3f3f3f}, {pos = 1061109567, events = 1061109567, fd = -1717986919, user_data = 0x9999999999999999}, {pos = 538976288, events = 538976288,
fd = 538976288, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x1db5e70}, {pos = -120, events = 4294967295, fd = 0, user_data = 0x1db3bf0}, {pos = 31134352, events = 0, fd = 30965344,
user_data = 0x7ffcbe112c50}, {pos = 1218437646, events = 31231, fd = 0, user_data = 0x1db11a0}, {pos = 31134112, events = 0, fd = 31134312, user_data = 0x2}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 200243,
user_data = 0xffffffffffffff88}, {pos = 0, events = 0, fd = -1106170480, user_data = 0x79ff48a3c0eb <fork+523>}, {pos = 31155248, events = 0, fd = 10350584, user_data = 0x1d87e60}, {pos = 31134112, events = 0, fd = 31134112, user_data = 0x0}, {
pos = -1106170576, events = 32764, fd = 1218426939, user_data = 0x1d857b0}, {pos = 30965344, events = 1, fd = 31134112, user_data = 0x727a9a <tokenize_auth_file+2170>}, {pos = -1106170512, events = 32764, fd = -1343231232, user_data = 0x7ffcbe112dc0}, {
pos = -120, events = 4294967295, fd = 0, user_data = 0x79ff489e79e8 <_IO_flush_all_lockp+632>}, {pos = 0, events = 0, fd = 0, user_data = 0x79ff489e5fd0 <flush_cleanup>}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 30955440, events = 0,
fd = -1343231232, user_data = 0x0}, {pos = 11, events = 0, fd = 0, user_data = 0x6e421652afefeb00}, {pos = -1106170480, events = 32764, fd = 1218047373, user_data = 0x0}, {pos = 8343831, events = 0, fd = 4194304, user_data = 0x1dad740}, {pos = 31134112,
events = 0, fd = 30965736, user_data = 0x0}, {pos = 10350288, events = 0, fd = 30965344, user_data = 0x1dad740}, {pos = 31119168, events = 0, fd = 31134112, user_data = 0x1d87fe8}, {pos = 10376739, events = 0, fd = 30938784, user_data = 0x0}, {pos = 30955440,
--Type <RET> for more, q to quit, c to continue without paging--
events = 0, fd = 7520150, user_data = 0xb}, {pos = 0, events = 0, fd = 0, user_data = 0x0}}
nevents = <optimized out>
__func__ = "ServerLoop"
#25 0x00000000007fa8f8 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x1d816a0) at postmaster.c:1372
opt = <optimized out>
status = <optimized out>
userDoption = <optimized out>
listen_addr_saved = true
output_config_variable = <optimized out>
__func__ = "PostmasterMain"
#26 0x000000000051b499 in main (argc=3, argv=0x1d816a0) at main.c:197
do_check_root = <optimized out>
(gdb)
(gdb) q
A debugging session is active.
Inferior 1 [process 200436] will be detached.
Attachments:
[text/plain] bt.txt (22.7K, ../../[email protected]/2-bt.txt)
download | inline:
0x000079ff48a73d37 in epoll_wait () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.37-18.fc38.x86_64 libgcc-13.2.1-4.fc38.x86_64 libicu-72.1-2.fc38.x86_64 libstdc++-13.2.1-4.fc38.x86_64 zlib-1.2.13-3.fc38.x86_64
(gdb) c
Continuing.
Program received signal SIGABRT, Aborted.
0x000079ff489ef884 in __pthread_kill_implementation () from /lib64/libc.so.6
(gdb) bt
#0 0x000079ff489ef884 in __pthread_kill_implementation () from /lib64/libc.so.6
#1 0x000079ff4899eafe in raise () from /lib64/libc.so.6
#2 0x000079ff4898787f in abort () from /lib64/libc.so.6
#3 0x00000000009ba563 in ExceptionalCondition (conditionName=conditionName@entry=0xa209e0 "BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno", fileName=fileName@entry=0xa205b4 "heapam_handler.c", lineNumber=lineNumber@entry=2221) at assert.c:66
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
#5 heapam_scan_bitmap_next_tuple (scan=0x1e73548, slot=<optimized out>, recheck=<optimized out>, lossy_pages=<optimized out>, exact_pages=<optimized out>) at heapam_handler.c:2359
#6 0x00000000006f58bb in table_scan_bitmap_next_tuple (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, slot=<optimized out>, scan=<optimized out>) at ../../../src/include/access/tableam.h:2022
#7 BitmapHeapNext (node=0x1e71fc8) at nodeBitmapHeapscan.c:202
#8 0x00000000006e46b8 in ExecProcNodeInstr (node=0x1e71fc8) at execProcnode.c:480
#9 0x00000000006dd6fa in ExecProcNode (node=0x1e71fc8) at ../../../src/include/executor/executor.h:274
#10 ExecutePlan (execute_once=<optimized out>, dest=0xb755e0 <donothingDR>, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0x1e71fc8, estate=0x1e71d80) at execMain.c:1644
#11 standard_ExecutorRun (queryDesc=0x1e6c500, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
#12 0x000000000067af84 in ExplainOnePlan (plannedstmt=plannedstmt@entry=0x1e6c3f0, into=into@entry=0x0, es=es@entry=0x1db3138, queryString=queryString@entry=0x1d87f60 "explain analyze select * from t where a = 200;", params=params@entry=0x0, queryEnv=queryEnv@entry=0x0,
planduration=0x7ffcbe111ac8, bufusage=0x0, mem_counters=0x0) at explain.c:645
#13 0x000000000067b417 in standard_ExplainOneQuery (query=<optimized out>, cursorOptions=2048, into=0x0, es=0x1db3138, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", params=0x0, queryEnv=0x0) at explain.c:459
#14 0x000000000067badb in ExplainQuery (pstate=0x1db3028, stmt=0x1d88e10, params=0x0, dest=0x1db2f98) at ../../../src/include/nodes/nodes.h:172
#15 0x0000000000894026 in standard_ProcessUtility (pstmt=0x1d88f60, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", readOnlyTree=<optimized out>, context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0, dest=0x1db2f98, qc=0x7ffcbe111d80)
at utility.c:863
#16 0x00000000008923cf in PortalRunUtility (portal=portal@entry=0x1e04680, pstmt=0x1d88f60, isTopLevel=true, setHoldSnapshot=setHoldSnapshot@entry=true, dest=dest@entry=0x1db2f98, qc=qc@entry=0x7ffcbe111d80) at pquery.c:1158
#17 0x0000000000892850 in FillPortalStore (portal=portal@entry=0x1e04680, isTopLevel=isTopLevel@entry=true) at ../../../src/include/nodes/nodes.h:172
#18 0x0000000000892b6d in PortalRun (portal=portal@entry=0x1e04680, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0x1dbb518, altdest=altdest@entry=0x1dbb518, qc=0x7ffcbe111f60) at pquery.c:763
#19 0x000000000088ec7f in exec_simple_query (query_string=0x1d87f60 "explain analyze select * from t where a = 200;") at postgres.c:1274
#20 0x000000000089050e in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
#21 0x000000000088b65d in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
#22 0x00000000007f52e5 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffcbe1123ac "", startup_data_len=startup_data_len@entry=4, client_sock=client_sock@entry=0x7ffcbe1123b0) at launch_backend.c:265
#23 0x00000000007f8dbe in BackendStartup (client_sock=0x7ffcbe1123b0) at postmaster.c:3593
#24 ServerLoop () at postmaster.c:1674
#25 0x00000000007fa8f8 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x1d816a0) at postmaster.c:1372
#26 0x000000000051b499 in main (argc=3, argv=0x1d816a0) at main.c:197
(gdb) up
#1 0x000079ff4899eafe in raise () from /lib64/libc.so.6
(gdb) up
#2 0x000079ff4898787f in abort () from /lib64/libc.so.6
(gdb) up
#3 0x00000000009ba563 in ExceptionalCondition (conditionName=conditionName@entry=0xa209e0 "BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno", fileName=fileName@entry=0xa205b4 "heapam_handler.c", lineNumber=lineNumber@entry=2221) at assert.c:66
66 abort();
(gdb) up
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
2221 Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
(gdb) up
#5 heapam_scan_bitmap_next_tuple (scan=0x1e73548, slot=<optimized out>, recheck=<optimized out>, lossy_pages=<optimized out>, exact_pages=<optimized out>) at heapam_handler.c:2359
2359 if (!heapam_scan_bitmap_next_block(scan, recheck,
(gdb) down
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
2221 Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
(gdb) p hscan->rs_cbuf
$1 = 6079
(gdb) p tmbres->blockno
No symbol "tmbres" in current context.
(gdb) p tmbres->blckno
No symbol "tmbres" in current context.
(gdb) p tbmres->blckno
There is no member named blckno.
(gdb) p tbmres->blockno
$2 = 66433
(gdb) bt full
#0 0x000079ff489ef884 in __pthread_kill_implementation () from /lib64/libc.so.6
No symbol table info available.
#1 0x000079ff4899eafe in raise () from /lib64/libc.so.6
No symbol table info available.
#2 0x000079ff4898787f in abort () from /lib64/libc.so.6
No symbol table info available.
#3 0x00000000009ba563 in ExceptionalCondition (conditionName=conditionName@entry=0xa209e0 "BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno", fileName=fileName@entry=0xa205b4 "heapam_handler.c", lineNumber=lineNumber@entry=2221) at assert.c:66
No locals.
#4 0x000000000057759f in heapam_scan_bitmap_next_block (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, scan=0x1e73548) at heapam_handler.c:2221
hscan = 0x1e73548
per_buffer_data = 0x1e793d8
block = <optimized out>
buffer = <optimized out>
snapshot = <optimized out>
ntup = <optimized out>
tbmres = 0x1e793d8
hscan = <optimized out>
per_buffer_data = <optimized out>
block = <optimized out>
buffer = <optimized out>
snapshot = <optimized out>
ntup = <optimized out>
tbmres = <optimized out>
curslot = <optimized out>
offnum = <optimized out>
tid = <optimized out>
heapTuple = <optimized out>
page = <optimized out>
maxoff = <optimized out>
offnum = <optimized out>
lp = <optimized out>
loctup = <optimized out>
valid = <optimized out>
#5 heapam_scan_bitmap_next_tuple (scan=0x1e73548, slot=<optimized out>, recheck=<optimized out>, lossy_pages=<optimized out>, exact_pages=<optimized out>) at heapam_handler.c:2359
hscan = 0x1e73548
targoffset = <optimized out>
page = <optimized out>
lp = <optimized out>
#6 0x00000000006f58bb in table_scan_bitmap_next_tuple (exact_pages=<optimized out>, lossy_pages=<optimized out>, recheck=<optimized out>, slot=<optimized out>, scan=<optimized out>) at ../../../src/include/access/tableam.h:2022
__func__ = "table_scan_bitmap_next_tuple"
__errno_location = <optimized out>
#7 BitmapHeapNext (node=0x1e71fc8) at nodeBitmapHeapscan.c:202
econtext = 0x1e721d8
scan = <optimized out>
tbm = <optimized out>
slot = 0x1e729e8
dsa = <optimized out>
__func__ = "BitmapHeapNext"
#8 0x00000000006e46b8 in ExecProcNodeInstr (node=0x1e71fc8) at execProcnode.c:480
result = <optimized out>
#9 0x00000000006dd6fa in ExecProcNode (node=0x1e71fc8) at ../../../src/include/executor/executor.h:274
No locals.
#10 ExecutePlan (execute_once=<optimized out>, dest=0xb755e0 <donothingDR>, direction=<optimized out>, numberTuples=0, sendTuples=true, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0x1e71fc8, estate=0x1e71d80) at execMain.c:1644
slot = <optimized out>
current_tuple_count = 812
slot = <optimized out>
current_tuple_count = <optimized out>
#11 standard_ExecutorRun (queryDesc=0x1e6c500, direction=<optimized out>, count=0, execute_once=<optimized out>) at execMain.c:363
estate = 0x1e71d80
operation = CMD_SELECT
dest = 0xb755e0 <donothingDR>
sendTuples = <optimized out>
oldcontext = 0x1db2df0
__func__ = "standard_ExecutorRun"
#12 0x000000000067af84 in ExplainOnePlan (plannedstmt=plannedstmt@entry=0x1e6c3f0, into=into@entry=0x0, es=es@entry=0x1db3138, queryString=queryString@entry=0x1d87f60 "explain analyze select * from t where a = 200;", params=params@entry=0x0, queryEnv=queryEnv@entry=0x0,
planduration=0x7ffcbe111ac8, bufusage=0x0, mem_counters=0x0) at explain.c:645
dir = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
dest = <optimized out>
queryDesc = 0x1e6c500
starttime = <optimized out>
totaltime = 0
eflags = <optimized out>
instrument_option = <optimized out>
#13 0x000000000067b417 in standard_ExplainOneQuery (query=<optimized out>, cursorOptions=2048, into=0x0, es=0x1db3138, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", params=0x0, queryEnv=0x0) at explain.c:459
plan = 0x1e6c3f0
planstart = <optimized out>
planduration = {ticks = 314825}
bufusage_start = {shared_blks_hit = 1, shared_blks_read = 134137333210088, shared_blks_dirtied = 16407, shared_blks_written = 8868755, local_blks_hit = 70467528441856, local_blks_read = 72057594037927936, local_blks_dirtied = 134137333210088,
local_blks_written = 1, temp_blks_read = 0, temp_blks_written = 10151148, shared_blk_read_time = {ticks = 0}, shared_blk_write_time = {ticks = 5433722}, local_blk_read_time = {ticks = 134137333210088}, local_blk_write_time = {ticks = 70368744194071},
temp_blk_read_time = {ticks = 30970240}, temp_blk_write_time = {ticks = 8648921}}
bufusage = {shared_blks_hit = 387345, shared_blks_read = 306578389, shared_blks_dirtied = 0, shared_blks_written = 0, local_blks_hit = 0, local_blks_read = 201863462913, local_blks_dirtied = 65, local_blks_written = 7, temp_blks_read = 5,
temp_blks_written = 10379061, shared_blk_read_time = {ticks = 0}, shared_blk_write_time = {ticks = 7608353}, local_blk_read_time = {ticks = 0}, local_blk_write_time = {ticks = 30969968}, temp_blk_read_time = {ticks = 0}, temp_blk_write_time = {ticks = 7609255}}
mem_counters = {nblocks = 0, freechunks = 8876259, totalspace = 70467528441856, freespace = 72057594037927936}
planner_ctx = 0x0
saved_ctx = 0x0
#14 0x000000000067badb in ExplainQuery (pstate=0x1db3028, stmt=0x1d88e10, params=0x0, dest=0x1db2f98) at ../../../src/include/nodes/nodes.h:172
l__state = {l = <optimized out>, i = 0}
l = 0x1e74140
es = 0x1db3138
tstate = <optimized out>
jstate = <optimized out>
query = <optimized out>
rewritten = 0x1e74128
lc = <optimized out>
timing_set = <optimized out>
summary_set = <optimized out>
__func__ = "ExplainQuery"
#15 0x0000000000894026 in standard_ProcessUtility (pstmt=0x1d88f60, queryString=0x1d87f60 "explain analyze select * from t where a = 200;", readOnlyTree=<optimized out>, context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0, dest=0x1db2f98, qc=0x7ffcbe111d80)
at utility.c:863
parsetree = 0x1d88e10
isTopLevel = <optimized out>
isAtomicContext = false
pstate = 0x1db3028
readonly_flags = <optimized out>
__func__ = "standard_ProcessUtility"
#16 0x00000000008923cf in PortalRunUtility (portal=portal@entry=0x1e04680, pstmt=0x1d88f60, isTopLevel=true, setHoldSnapshot=setHoldSnapshot@entry=true, dest=dest@entry=0x1db2f98, qc=qc@entry=0x7ffcbe111d80) at pquery.c:1158
No locals.
#17 0x0000000000892850 in FillPortalStore (portal=portal@entry=0x1e04680, isTopLevel=isTopLevel@entry=true) at ../../../src/include/nodes/nodes.h:172
treceiver = 0x1db2f98
qc = {commandTag = CMDTAG_UNKNOWN, nprocessed = 0}
__func__ = "FillPortalStore"
#18 0x0000000000892b6d in PortalRun (portal=portal@entry=0x1e04680, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0x1dbb518, altdest=altdest@entry=0x1dbb518, qc=0x7ffcbe111f60) at pquery.c:763
_save_exception_stack = 0x7ffcbe112230
_save_context_stack = 0x0
_local_sigjmp_buf = {{__jmpbuf = {31149072, -109934348456944260, 30969640, 140723497279328, 31475328, 31175960, 108104918559890812, -109935865491088004}, __mask_was_saved = 0, __saved_mask = {__val = {31486888, 2817148525, 0, 0, 10501697, 0, 31483536, 12414709,
31475328, 12414709, 1, 153, 30969616, 31176984, 10382623, 31475328}}}}
_do_rethrow = <optimized out>
result = <optimized out>
nprocessed = <optimized out>
saveTopTransactionResourceOwner = 0x1dc4018
saveTopTransactionContext = 0x1db4c10
saveActivePortal = 0x0
saveResourceOwner = 0x1dc4018
savePortalContext = 0x0
saveMemoryContext = 0x1db4c10
__func__ = "PortalRun"
#19 0x000000000088ec7f in exec_simple_query (query_string=0x1d87f60 "explain analyze select * from t where a = 200;") at postgres.c:1274
cmdtaglen = 7
snapshot_set = <optimized out>
per_parsetree_context = 0x0
plantree_list = <optimized out>
parsetree = <optimized out>
commandTag = <optimized out>
qc = {commandTag = CMDTAG_UNKNOWN, nprocessed = 0}
--Type <RET> for more, q to quit, c to continue without paging--
querytree_list = <optimized out>
portal = 0x1e04680
receiver = 0x1dbb518
format = 0
cmdtagname = <optimized out>
parsetree_item__state = {l = 0x1d88f10, i = <optimized out>}
dest = DestRemote
oldcontext = 0x1db4c10
parsetree_list = 0x1d88f10
parsetree_item = <optimized out>
save_log_statement_stats = false
was_logged = false
use_implicit_block = false
msec_str = '\000' <repeats 16 times>, "Q\000\000\000\000\000\000\000Q\000\000\000\000\000\000"
__func__ = "exec_simple_query"
#20 0x000000000089050e in PostgresMain (dbname=<optimized out>, username=<optimized out>) at postgres.c:4680
query_string = 0x1d87f60 "explain analyze select * from t where a = 200;"
firstchar = <optimized out>
input_message = {data = 0x1d87f60 "explain analyze select * from t where a = 200;", len = 47, maxlen = 1024, cursor = 47}
local_sigjmp_buf = {{__jmpbuf = {31205072, -109934379334361732, 4, 0, 1711661613, 30973184, 108104918503267708, -109935863615054468}, __mask_was_saved = 1, __saved_mask = {__val = {4194304, 979, 18446744073709551552, 15624, 30973184, 140723497280224,
134137342057842, 0, 7944937237029841664, 140723497280304, 15616, 30938944, 0, 15680, 10344002, 30938944}}}}
send_ready_for_query = false
idle_in_transaction_timeout_enabled = false
idle_session_timeout_enabled = false
__func__ = "PostgresMain"
#21 0x000000000088b65d in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at backend_startup.c:101
bsdata = <optimized out>
#22 0x00000000007f52e5 in postmaster_child_launch (child_type=child_type@entry=B_BACKEND, startup_data=startup_data@entry=0x7ffcbe1123ac "", startup_data_len=startup_data_len@entry=4, client_sock=client_sock@entry=0x7ffcbe1123b0) at launch_backend.c:265
pid = <optimized out>
#23 0x00000000007f8dbe in BackendStartup (client_sock=0x7ffcbe1123b0) at postmaster.c:3593
bn = 0x1d89d00
pid = <optimized out>
startup_data = {canAcceptConnections = CAC_OK}
bn = <optimized out>
pid = <optimized out>
startup_data = <optimized out>
__func__ = "BackendStartup"
__errno_location = <optimized out>
__errno_location = <optimized out>
save_errno = <optimized out>
__errno_location = <optimized out>
__errno_location = <optimized out>
#24 ServerLoop () at postmaster.c:1674
s = {sock = 10, raddr = {addr = {ss_family = 1,
__ss_padding = "\000\000\000\000\000\000\360\207\330\001\000\000\000\000\b\344\225H\377y\000\000\000\000\000\000\000\000\000\000H\374\225H\377y\000\000\350$\021\276\374\177\000\000\000\000\000\000\374\177\000\000\024$\021\276\374\177\000\000\250\333\371\003\000\000\000\000\320cA\000\000\000\000\000\032\\E\000\000\000\000\000`d\327", '\000' <repeats 13 times>, "\b\344\225H\377y\000\000\320\374\225H\377y\000", __ss_align = 134137344681544}, salen = 2}}
i = 0
now = <optimized out>
last_lockfile_recheck_time = 1711661738
last_touch_time = 1711661613
events = {{pos = 3, events = 2, fd = 8, user_data = 0x0}, {pos = 0, events = 0, fd = -1106171568, user_data = 0x79ff4895fcd0}, {pos = 1221060168, events = 31231, fd = 4544639, user_data = 0x7c96e577}, {pos = -1106171472, events = 32764, fd = -1106171504,
user_data = 0x79ff48c535dc <_dl_lookup_symbol_x+300>}, {pos = 5, events = 0, fd = 1217789136, user_data = 0x1}, {pos = 0, events = 0, fd = 1, user_data = 0x79ff48c7e2c0}, {pos = -1106171632, events = 32764, fd = 1218436466, user_data = 0x1d857b0}, {
pos = 30965736, events = 0, fd = 0, user_data = 0x79ff48c7e648}, {pos = -1106171600, events = 32764, fd = 1, user_data = 0x79ff48c7e2c0}, {pos = -1106171608, events = 32764, fd = 9, user_data = 0xffffffff}, {pos = 1217855464, events = 31231, fd = 1220838640,
user_data = 0xca10}, {pos = 2100392, events = 0, fd = 2, user_data = 0x1000041c0}, {pos = 1000, events = 0, fd = 1221059264, user_data = 0xf2}, {pos = 4776592, events = 0, fd = 13948816, user_data = 0x1}, {pos = -1106171408, events = 32764, fd = 1220910653,
user_data = 0x1}, {pos = 0, events = 0, fd = 1217855464, user_data = 0x79ff48a3bee0 <fork>}, {pos = 0, events = 0, fd = -1106170496, user_data = 0x0}, {pos = 0, events = 0, fd = 30965736, user_data = 0x1}, {pos = 0, events = 0, fd = 1220920014,
user_data = 0x0}, {pos = 1218398648, events = 31231, fd = 0, user_data = 0xd6a120 <BlockSig>}, {pos = 2, events = 0, fd = 30955440, user_data = 0x10}, {pos = 1219714176, events = 31231, fd = 255, user_data = 0xffffffffffffffc0}, {pos = 0, events = 0,
fd = 8096, user_data = 0x7ffcbe112aa0}, {pos = 1218436466, events = 31231, fd = 1217889328, user_data = 0x79ff489e0b50 <fputc>}, {pos = 30955440, events = 0, fd = 4096, user_data = 0x3ff}, {pos = 0, events = 0, fd = -1106171024,
user_data = 0x79ff489d7869 <_IO_file_doallocate+185>}, {pos = 51728, events = 0, fd = 2100403, user_data = 0x1}, {pos = 33152, events = 1000, fd = 1000, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 65535,
user_data = 0xff0000000000ffff}, {pos = 65535, events = 4278190080, fd = 65535, user_data = 0x79ff48b35ce0 <main_arena+96>}, {pos = 0, events = 0, fd = 0, user_data = 0x100000000000000}, {pos = 0, events = 0, fd = 255, user_data = 0x5c5c5c5c5c5c5c5c}, {
pos = 1549556828, events = 1549556828, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x3f3f3f3f3f3f3f3f}, {pos = 1061109567, events = 1061109567, fd = -1717986919, user_data = 0x9999999999999999}, {pos = 538976288, events = 538976288,
fd = 538976288, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 0, user_data = 0x1db5e70}, {pos = -120, events = 4294967295, fd = 0, user_data = 0x1db3bf0}, {pos = 31134352, events = 0, fd = 30965344,
user_data = 0x7ffcbe112c50}, {pos = 1218437646, events = 31231, fd = 0, user_data = 0x1db11a0}, {pos = 31134112, events = 0, fd = 31134312, user_data = 0x2}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 0, events = 0, fd = 200243,
user_data = 0xffffffffffffff88}, {pos = 0, events = 0, fd = -1106170480, user_data = 0x79ff48a3c0eb <fork+523>}, {pos = 31155248, events = 0, fd = 10350584, user_data = 0x1d87e60}, {pos = 31134112, events = 0, fd = 31134112, user_data = 0x0}, {
pos = -1106170576, events = 32764, fd = 1218426939, user_data = 0x1d857b0}, {pos = 30965344, events = 1, fd = 31134112, user_data = 0x727a9a <tokenize_auth_file+2170>}, {pos = -1106170512, events = 32764, fd = -1343231232, user_data = 0x7ffcbe112dc0}, {
pos = -120, events = 4294967295, fd = 0, user_data = 0x79ff489e79e8 <_IO_flush_all_lockp+632>}, {pos = 0, events = 0, fd = 0, user_data = 0x79ff489e5fd0 <flush_cleanup>}, {pos = 0, events = 0, fd = 0, user_data = 0x0}, {pos = 30955440, events = 0,
fd = -1343231232, user_data = 0x0}, {pos = 11, events = 0, fd = 0, user_data = 0x6e421652afefeb00}, {pos = -1106170480, events = 32764, fd = 1218047373, user_data = 0x0}, {pos = 8343831, events = 0, fd = 4194304, user_data = 0x1dad740}, {pos = 31134112,
events = 0, fd = 30965736, user_data = 0x0}, {pos = 10350288, events = 0, fd = 30965344, user_data = 0x1dad740}, {pos = 31119168, events = 0, fd = 31134112, user_data = 0x1d87fe8}, {pos = 10376739, events = 0, fd = 30938784, user_data = 0x0}, {pos = 30955440,
--Type <RET> for more, q to quit, c to continue without paging--
events = 0, fd = 7520150, user_data = 0xb}, {pos = 0, events = 0, fd = 0, user_data = 0x0}}
nevents = <optimized out>
__func__ = "ServerLoop"
#25 0x00000000007fa8f8 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0x1d816a0) at postmaster.c:1372
opt = <optimized out>
status = <optimized out>
userDoption = <optimized out>
listen_addr_saved = true
output_config_variable = <optimized out>
__func__ = "PostmasterMain"
#26 0x000000000051b499 in main (argc=3, argv=0x1d816a0) at main.c:197
do_check_root = <optimized out>
(gdb)
(gdb) q
A debugging session is active.
Inferior 1 [process 200436] will be detached.
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
@ 2024-03-29 01:12 ` Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Munro @ 2024-03-29 01:12 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
<[email protected]> wrote:
> I think there's some sort of bug, triggering this assert in heapam
>
> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
Thanks for the repro. I can't seem to reproduce it (still trying) but
I assume this is with Melanie's v11 patch set which had
v11-0016-v10-Read-Stream-API.patch.
Would you mind removing that commit and instead applying the v13
stream_read.c patches[1]? v10 stream_read.c was a little confused
about random I/O combining, which I fixed with a small adjustment to
the conditions for the "if" statement right at the end of
read_stream_look_ahead(). Sorry about that. The fixed version, with
eic=4, with your test query using WHERE a < a, ends its scan with:
...
posix_fadvise(32,0x28aee000,0x4000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0@4\M-5:\0\0\^D\0\M-x\^A"...,40960,0x28acc000) = 40960 (0xa000)
posix_fadvise(32,0x28af4000,0x4000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0\^XC\M-6:\0\0\^D\0\M-x"...,32768,0x28ad8000) = 32768 (0x8000)
posix_fadvise(32,0x28afc000,0x4000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0\M-XQ\M-7:\0\0\^D\0\M-x"...,24576,0x28ae4000) = 24576 (0x6000)
posix_fadvise(32,0x28b02000,0x8000,POSIX_FADV_WILLNEED) = 0 (0x0)
pread(32,"\0\0\0\0\M^@3\M-8:\0\0\^D\0\M-x"...,16384,0x28aee000) = 16384 (0x4000)
pread(32,"\0\0\0\0\M-`\M-:\M-8:\0\0\^D\0"...,16384,0x28af4000) = 16384 (0x4000)
pread(32,"\0\0\0\0po\M-9:\0\0\^D\0\M-x\^A"...,16384,0x28afc000) = 16384 (0x4000)
pread(32,"\0\0\0\0\M-P\M-v\M-9:\0\0\^D\0"...,32768,0x28b02000) = 32768 (0x8000)
In other words it's able to coalesce, but v10 was a bit b0rked in that
respect and wouldn't do as well at that. Then if you set
io_combine_limit = 1, it looks more like master, eg lots of little
reads, but not as many fadvises as master because of sequential
access:
...
posix_fadvise(32,0x28af4000,0x2000,POSIX_FADV_WILLNEED) = 0 (0x0) -+
pread(32,...,8192,0x28ae8000) = 8192 (0x2000) |
pread(32,...,8192,0x28aee000) = 8192 (0x2000) |
posix_fadvise(32,0x28afc000,0x2000,POSIX_FADV_WILLNEED) = 0 (0x0) ---+
pread(32,...,8192,0x28af0000) = 8192 (0x2000) | |
pread(32,...,8192,0x28af4000) = 8192 (0x2000) <--------------------+ |
posix_fadvise(32,0x28b02000,0x2000,POSIX_FADV_WILLNEED) = 0 (0x0) -----+
pread(32,...,8192,0x28af6000) = 8192 (0x2000) | |
pread(32,...,8192,0x28afc000) = 8192 (0x2000) <----------------------+ |
pread(32,...,8192,0x28afe000) = 8192 (0x2000) }-- no advice |
pread(32,...,8192,0x28b02000) = 8192 (0x2000) <------------------------+
pread(32,...,8192,0x28b04000) = 8192 (0x2000) }
pread(32,...,8192,0x28b06000) = 8192 (0x2000) }-- no advice
pread(32,...,8192,0x28b08000) = 8192 (0x2000) }
It becomes slightly less eager to start I/Os as soon as
io_combine_limit > 1, because when it has hit max_ios, if ... <thinks>
yeah if the average block that it can combine is bigger than 4, an
arbitrary number from:
max_pinned_buffers = Max(max_ios * 4, io_combine_limit);
.... then it can run out of look ahead window before it can reach
max_ios (aka eic), so that's a kind of arbitrary/bogus I/O depth
constraint, which is another way of saying what I was saying earlier:
maybe it just needs more distance. So let's see the average combined
I/O length in your test query... for me it works out to 27,169 bytes.
But I think there must be times when it runs out of window due to
clustering. So you could also try increasing that 4->8 to see what
happens to performance.
[1] https://www.postgresql.org/message-id/CA%2BhUKG%2B5UofvseJWv6YqKmuc_%3Drguc7VqKcNEG1eawKh3MzHXQ%40ma...
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
@ 2024-03-29 11:05 ` Tomas Vondra <[email protected]>
2024-03-29 11:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-31 15:45 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
0 siblings, 2 replies; 21+ messages in thread
From: Tomas Vondra @ 2024-03-29 11:05 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On 3/29/24 02:12, Thomas Munro wrote:
> On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
> <[email protected]> wrote:
>> I think there's some sort of bug, triggering this assert in heapam
>>
>> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
>
> Thanks for the repro. I can't seem to reproduce it (still trying) but
> I assume this is with Melanie's v11 patch set which had
> v11-0016-v10-Read-Stream-API.patch.
>
> Would you mind removing that commit and instead applying the v13
> stream_read.c patches[1]? v10 stream_read.c was a little confused
> about random I/O combining, which I fixed with a small adjustment to
> the conditions for the "if" statement right at the end of
> read_stream_look_ahead(). Sorry about that. The fixed version, with
> eic=4, with your test query using WHERE a < a, ends its scan with:
>
I'll give that a try. Unfortunately unfortunately the v11 still has the
problem I reported about a week ago:
ERROR: prefetch and main iterators are out of sync
So I can't run the full benchmarks :-( but master vs. streaming read API
should work, I think.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
@ 2024-03-29 11:17 ` Thomas Munro <[email protected]>
2024-03-29 13:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
1 sibling, 1 reply; 21+ messages in thread
From: Thomas Munro @ 2024-03-29 11:17 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
I spent a bit of time today testing Melanie's v11, except with
read_stream.c v13, on Linux, ext4, and 3000 IOPS cloud storage. I
think I now know roughly what's going on. Here are some numbers,
using your random table from above and a simple SELECT * FROM t WHERE
a < 100 OR a = 123456. I'll keep parallelism out of this for now.
These are milliseconds:
eic unpatched patched
0 4172 9572
1 30846 10376
2 18435 5562
4 18980 3503
8 18980 2680
16 18976 3233
So with eic=0, unpatched wins. The reason is that Linux readahead
wakes up and scans the table at 150MB/s, because there are enough
clusters to trigger it. But patched doesn't look quite so sequential
because we removed the sequential accesses by I/O combining...
At eic=1, unpatched completely collapses. I'm not sure why exactly.
Once you go above eic=1, Linux seems to get out of the way and just do
what we asked it to do: iostat shows exactly 3000 IOPS, exactly 8KB
avg read size, and (therefore) throughput of 24MB/sec, though you can
see the queue depth being exactly what we asked it to do,eg 7.9 or
whatever for eic=8, while patched eats it for breakfast because it
issues wide requests, averaging around 27KB.
It seems more informative to look at the absolute numbers rather than
the A/B ratios, because then you can see how the numbers themselves
are already completely nuts, sort of interference patterns from
interaction with kernel heuristics.
On the other hand this might be a pretty unusual data distribution.
People who store random numbers or hashes or whatever probably don't
really search for ranges of them (unless they're trying to mine
bitcoins in SQL). I dunno. Maybe we need more realistic tests, or
maybe we're just discovering all the things that are bad about the
pre-existing code.
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 11:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
@ 2024-03-29 13:36 ` Thomas Munro <[email protected]>
2024-03-29 15:52 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Munro @ 2024-03-29 13:36 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Sat, Mar 30, 2024 at 12:17 AM Thomas Munro <[email protected]> wrote:
> eic unpatched patched
> 0 4172 9572
> 1 30846 10376
> 2 18435 5562
> 4 18980 3503
> 8 18980 2680
> 16 18976 3233
... but the patched version gets down to a low number for eic=0 too if
you turn up the blockdev --setra so that it also gets Linux RA
treatment, making it the clear winner on all eic settings. Patched
doesn't improve. So, for low IOPS storage at least, when you're on
the borderline between random and sequential, ie bitmap with a lot of
1s in it, it seems there are cases where patched doesn't trigger Linux
RA but unpatched does, and you can tune your way out of that, and then
there are cases where the IOPS limit is reached due to small reads,
but patched does better because of larger I/Os that are likely under
the same circumstances. Does that make sense?
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 11:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 13:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
@ 2024-03-29 15:52 ` Tomas Vondra <[email protected]>
2024-03-29 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Tomas Vondra @ 2024-03-29 15:52 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On 3/29/24 14:36, Thomas Munro wrote:
> On Sat, Mar 30, 2024 at 12:17 AM Thomas Munro <[email protected]> wrote:
>> eic unpatched patched
>> 0 4172 9572
>> 1 30846 10376
>> 2 18435 5562
>> 4 18980 3503
>> 8 18980 2680
>> 16 18976 3233
>
> ... but the patched version gets down to a low number for eic=0 too if
> you turn up the blockdev --setra so that it also gets Linux RA
> treatment, making it the clear winner on all eic settings. Patched
> doesn't improve. So, for low IOPS storage at least, when you're on
> the borderline between random and sequential, ie bitmap with a lot of
> 1s in it, it seems there are cases where patched doesn't trigger Linux
> RA but unpatched does, and you can tune your way out of that, and then
> there are cases where the IOPS limit is reached due to small reads,
> but patched does better because of larger I/Os that are likely under
> the same circumstances. Does that make sense?
I think you meant "unpatched version gets down" in the first sentence,
right? Still, it seems clear this changes the interaction with readahead
done by the kernel.
However, you seem to focus only on eic=0/eic=1 cases, but IIRC that was
just an example. There are regression with higher eic values too.
I do have some early results from the benchmarks - it's just from the
NVMe machine, with 1M tables (~300MB), and it's just one incomplete run
(so there might be some noise etc.).
Attached is a PDF with charts for different subsets of the runs:
- optimal (would optimizer pick bitmapscan or not)
- readahead (yes/no)
- workers (serial vs. 4 workers)
- combine limit (8kB / 128kB)
The most interesting cases are first two rows, i.e. optimal plans.
Either with readahead enabled (first row) or disabled (second row).
Two observations:
* The combine limit seems to have negligible impact. There's no visible
difference between combine_limit=8kB and 128kB.
* Parallel queries seem to work about the same as master (especially for
optimal cases, but even for not optimal ones).
The optimal plans with kernel readahead (two charts in the first row)
look fairly good. There are a couple regressed cases, but a bunch of
faster ones too.
The optimal plans without kernel read ahead (two charts in the second
row) perform pretty poorly - there are massive regressions. But I think
the obvious reason is that the streaming read API skips prefetches for
sequential access patterns, relying on kernel to do the readahead. But
if the kernel readahead is disabled for the device, that obviously can't
happen ...
I think the question is how much we can (want to) rely on the readahead
to be done by the kernel. Maybe there should be some flag to force
issuing fadvise even for sequential patterns, perhaps at the tablespace
level? I don't recall seeing a system with disabled readahead, but I'm
sure there are cases where it may not really work - it clearly can't
work with direct I/O, but I've also not been very successful with
prefetching on ZFS.
The non-optimal plans (second half of the charts) shows about the same
behavior, but the regressions are more frequent / significant.
I'm also attaching results for the 5k "optimal" runs, showing the timing
for master and patched build, sorted by (patched/master). The most
significant regressions are with readahead=0, but if you filter that out
you'll see the regressions affect a mix of data sets, not just the
uniformly random data used as example before.
On 3/29/24 12:17, Thomas Munro wrote:
> ...
> On the other hand this might be a pretty unusual data distribution.
> People who store random numbers or hashes or whatever probably don't
> really search for ranges of them (unless they're trying to mine
> bitcoins in SQL). I dunno. Maybe we need more realistic tests, or
> maybe we're just discovering all the things that are bad about the
> pre-existing code.
I certainly admit the data sets are synthetic and perhaps adversarial.
My intent was to cover a wide range of data sets, to trigger even less
common cases. It's certainly up to debate how serious the regressions on
those data sets are in practice, I'm not suggesting "this strange data
set makes it slower than master, so we can't commit this".
But I'd also point that what matters is the access pattern, not the
exact query generating it. I agree people probably don't do random
numbers or hashes with range conditions, but that's irrelevant - what
it's all about the page access pattern. If you have IPv4 addresses and
query that, that's likely going to be pretty random, for example.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
rows | dataset | workers | wm | eic | matches | caching | readahead | combine_limit | cnt | master | patched | ?column?
---------+---------------+---------+-------+-----+---------+-----------+-----------+---------------+-----+------------------------+------------------------+------------------------
1000000 | uniform | 0 | 4096 | 32 | 256 | uncached | 0 | 8 | 2 | 197.3750000000000000 | 1710.7450000000000000 | 8.6674857504749842
1000000 | uniform | 0 | 4096 | 32 | 1024 | uncached | 0 | 8 | 2 | 388.3380000000000000 | 3323.4390000000000000 | 8.5581091729369776
1000000 | uniform | 0 | 65536 | 32 | 1024 | uncached | 0 | 8 | 2 | 407.7540000000000000 | 3380.9000000000000000 | 8.2915189060070533
1000000 | uniform | 0 | 65536 | 32 | 2048 | uncached | 0 | 8 | 2 | 439.8440000000000000 | 3547.9540000000000000 | 8.0663917207009758
1000000 | uniform | 0 | 65536 | 16 | 1024 | uncached | 0 | 8 | 2 | 424.1210000000000000 | 3404.8380000000000000 | 8.0279872960782418
1000000 | uniform | 0 | 4096 | 16 | 2048 | uncached | 0 | 8 | 2 | 469.9440000000000000 | 3614.8390000000000000 | 7.6920633096709395
1000000 | uniform | 0 | 4096 | 32 | 2048 | uncached | 0 | 8 | 2 | 470.5050000000000000 | 3561.9470000000000000 | 7.5704764030137831
1000000 | uniform | 0 | 65536 | 16 | 4048 | uncached | 0 | 128 | 2 | 474.8690000000000000 | 3517.6140000000000000 | 7.4075460811297432
1000000 | uniform | 0 | 4096 | 32 | 4048 | uncached | 0 | 128 | 2 | 473.5260000000000000 | 3474.0360000000000000 | 7.3365263998175391
1000000 | uniform_pages | 0 | 65536 | 32 | 211 | uncached | 0 | 8 | 2 | 284.9390000000000000 | 2050.9960000000000000 | 7.1980178213582556
1000000 | uniform | 0 | 4096 | 32 | 4048 | uncached | 0 | 8 | 2 | 520.9200000000000000 | 3626.4810000000000000 | 6.9616850956000921
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 52 | uncached | 0 | 8 | 2 | 295.4450000000000000 | 2054.9440000000000000 | 6.9554197904855388
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 16 | uncached | 0 | 8 | 2 | 99.9790000000000000 | 691.6020000000000000 | 6.9174726692605447
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 34.9790000000000000 | 241.0800000000000000 | 6.8921352811687012
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 16 | uncached | 0 | 8 | 2 | 101.0870000000000000 | 695.6470000000000000 | 6.8816662874553602
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 191.7020000000000000 | 1303.3400000000000000 | 6.7987814420298171
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 24.0290000000000000 | 159.4270000000000000 | 6.6347746473011777
1000000 | cyclic | 0 | 65536 | 16 | 4 | uncached | 0 | 8 | 2 | 33.1760000000000000 | 219.1070000000000000 | 6.6043826862792380
1000000 | cyclic | 0 | 65536 | 16 | 8 | uncached | 0 | 8 | 2 | 55.6630000000000000 | 367.3740000000000000 | 6.5999676625406464
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 2 | uncached | 0 | 8 | 2 | 24.1790000000000000 | 159.3560000000000000 | 6.5906778609537202
1000000 | cyclic | 0 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 20.7600000000000000 | 136.4900000000000000 | 6.5746628131021195
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 16 | uncached | 0 | 8 | 2 | 105.8640000000000000 | 693.0790000000000000 | 6.5468809038011033
1000000 | cyclic | 0 | 65536 | 16 | 2 | uncached | 0 | 8 | 2 | 21.0990000000000000 | 137.9350000000000000 | 6.5375136262382103
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 0 | 8 | 2 | 59.9430000000000000 | 386.9210000000000000 | 6.4548154079709057
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 52 | uncached | 0 | 8 | 2 | 311.6300000000000000 | 1999.8530000000000000 | 6.4173956294323396
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 32 | uncached | 0 | 8 | 2 | 204.7160000000000000 | 1311.0530000000000000 | 6.4042527208425331
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 32 | uncached | 0 | 8 | 2 | 196.1600000000000000 | 1255.4800000000000000 | 6.4002854812398042
1000000 | uniform | 0 | 65536 | 8 | 2048 | uncached | 0 | 8 | 2 | 566.2560000000000000 | 3598.4170000000000000 | 6.3547529739199231
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 0 | 8 | 2 | 36.9560000000000000 | 234.7780000000000000 | 6.3529061586751813
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 56.7750000000000000 | 360.2920000000000000 | 6.3459621312197270
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 8 | uncached | 0 | 8 | 2 | 59.7330000000000000 | 378.6380000000000000 | 6.3388411765690657
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 52 | uncached | 0 | 8 | 2 | 330.1280000000000000 | 2089.7620000000000000 | 6.3301567876702370
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 0 | 8 | 2 | 36.9260000000000000 | 230.7940000000000000 | 6.2501760277311380
1000000 | cyclic | 0 | 4096 | 32 | 52 | uncached | 0 | 128 | 2 | 285.1420000000000000 | 1779.4320000000000000 | 6.2405117450252856
1000000 | cyclic | 0 | 4096 | 32 | 32 | uncached | 0 | 8 | 2 | 200.0930000000000000 | 1244.2080000000000000 | 6.2181485609191726
1000000 | uniform | 0 | 4096 | 8 | 2048 | uncached | 0 | 8 | 2 | 579.0620000000000000 | 3600.1250000000000000 | 6.2171667282605317
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 52 | uncached | 0 | 128 | 2 | 292.2530000000000000 | 1804.1090000000000000 | 6.1731068628893459
1000000 | uniform | 0 | 4096 | 16 | 256 | uncached | 0 | 8 | 2 | 275.0420000000000000 | 1696.9760000000000000 | 6.1698795093113052
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 27.1780000000000000 | 167.3230000000000000 | 6.1565604533078225
1000000 | uniform | 0 | 4096 | 8 | 4048 | uncached | 0 | 8 | 2 | 581.9640000000000000 | 3582.8660000000000000 | 6.1565079626918504
1000000 | cyclic | 0 | 4096 | 16 | 52 | uncached | 0 | 8 | 2 | 321.3510000000000000 | 1974.7440000000000000 | 6.1451310249540222
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 52 | uncached | 0 | 128 | 2 | 294.6290000000000000 | 1801.8960000000000000 | 6.1158134467414952
1000000 | uniform | 0 | 65536 | 8 | 1024 | uncached | 0 | 8 | 2 | 562.2080000000000000 | 3422.6260000000000000 | 6.0878287039672150
1000000 | cyclic | 0 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 22.7190000000000000 | 137.5250000000000000 | 6.0533034024384876
1000000 | cyclic | 0 | 65536 | 16 | 16 | uncached | 0 | 8 | 2 | 106.0580000000000000 | 641.5630000000000000 | 6.0491712082068302
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 32 | uncached | 0 | 128 | 2 | 191.1380000000000000 | 1154.9810000000000000 | 6.0426550450459877
1000000 | cyclic | 0 | 65536 | 16 | 52 | uncached | 0 | 128 | 2 | 306.2530000000000000 | 1847.7090000000000000 | 6.0332764087208942
1000000 | uniform | 0 | 4096 | 8 | 4048 | uncached | 0 | 128 | 2 | 591.1120000000000000 | 3544.5280000000000000 | 5.9963729377850560
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 0 | 8 | 2 | 59.7360000000000000 | 354.0220000000000000 | 5.9264430159367885
1000000 | uniform | 0 | 4096 | 8 | 1024 | uncached | 0 | 8 | 2 | 563.3670000000000000 | 3332.3490000000000000 | 5.9150589225140983
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 68.2930000000000000 | 403.5720000000000000 | 5.9094197062656495
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 52 | uncached | 0 | 128 | 2 | 312.1930000000000000 | 1826.3510000000000000 | 5.8500703090716320
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 32 | uncached | 0 | 128 | 2 | 194.5990000000000000 | 1136.9650000000000000 | 5.8426045354806551
1000000 | uniform_pages | 0 | 65536 | 16 | 211 | uncached | 0 | 8 | 2 | 349.2250000000000000 | 2036.6470000000000000 | 5.8319049323502040
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 32 | uncached | 0 | 8 | 2 | 224.9200000000000000 | 1299.4710000000000000 | 5.7774808820914103
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 36.7270000000000000 | 210.6900000000000000 | 5.7366515097884390
1000000 | uniform | 4 | 65536 | 32 | 128 | uncached | 0 | 8 | 2 | 117.8920000000000000 | 672.3060000000000000 | 5.7027279204695823
1000000 | cyclic | 0 | 4096 | 32 | 32 | uncached | 0 | 128 | 2 | 194.4810000000000000 | 1095.4010000000000000 | 5.6324319599343895
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 16 | uncached | 0 | 8 | 2 | 122.4230000000000000 | 689.0290000000000000 | 5.6282642967416253
1000000 | cyclic | 0 | 4096 | 8 | 8 | uncached | 0 | 8 | 2 | 66.8700000000000000 | 376.3480000000000000 | 5.6280544339763721
1000000 | cyclic | 0 | 65536 | 32 | 32 | uncached | 0 | 128 | 2 | 192.0460000000000000 | 1075.7940000000000000 | 5.6017516636639139
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 52 | uncached | 0 | 128 | 2 | 340.7140000000000000 | 1899.0370000000000000 | 5.5736981750089518
1000000 | uniform | 0 | 65536 | 32 | 128 | uncached | 0 | 8 | 2 | 118.6750000000000000 | 654.2880000000000000 | 5.5132757531072256
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 8 | uncached | 0 | 8 | 2 | 70.9930000000000000 | 390.8210000000000000 | 5.5050638795374192
1000000 | uniform | 0 | 4096 | 32 | 128 | uncached | 0 | 8 | 2 | 123.2220000000000000 | 678.0760000000000000 | 5.5028809790459496
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 16 | uncached | 0 | 8 | 2 | 125.0850000000000000 | 686.1930000000000000 | 5.4858136467202302
1000000 | cyclic | 0 | 65536 | 8 | 16 | uncached | 0 | 8 | 2 | 116.4450000000000000 | 635.9870000000000000 | 5.4616943621452188
1000000 | cyclic | 0 | 65536 | 8 | 8 | uncached | 0 | 8 | 2 | 67.3550000000000000 | 365.1860000000000000 | 5.4218098136738178
1000000 | cyclic | 0 | 65536 | 8 | 4 | uncached | 0 | 8 | 2 | 40.0240000000000000 | 216.1760000000000000 | 5.4011593044173496
1000000 | cyclic | 0 | 4096 | 32 | 16 | uncached | 0 | 128 | 2 | 98.5040000000000000 | 522.8400000000000000 | 5.3078047591975960
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 2 | uncached | 0 | 8 | 2 | 31.1260000000000000 | 165.0880000000000000 | 5.3038617233181263
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 16 | uncached | 0 | 128 | 2 | 100.8680000000000000 | 533.8580000000000000 | 5.2926398857913312
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 52 | uncached | 0 | 128 | 2 | 359.3540000000000000 | 1900.1160000000000000 | 5.2875882834196920
1000000 | cyclic | 0 | 128 | 8 | 4 | uncached | 0 | 8 | 2 | 40.6010000000000000 | 212.8740000000000000 | 5.2430728307184552
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 4 | uncached | 0 | 8 | 2 | 44.8610000000000000 | 234.9410000000000000 | 5.2370878937161454
1000000 | uniform | 0 | 65536 | 8 | 512 | uncached | 0 | 8 | 2 | 561.6130000000000000 | 2927.0930000000000000 | 5.2119395384366103
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 2 | uncached | 0 | 8 | 2 | 32.5860000000000000 | 167.2210000000000000 | 5.1316823175596882
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 52 | uncached | 0 | 128 | 2 | 354.6300000000000000 | 1806.6930000000000000 | 5.0945859064376956
1000000 | cyclic | 0 | 4096 | 8 | 4 | uncached | 0 | 8 | 2 | 41.0710000000000000 | 209.0330000000000000 | 5.0895522388059701
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 16 | uncached | 0 | 128 | 2 | 108.6830000000000000 | 547.8050000000000000 | 5.0403927017104791
1000000 | cyclic | 0 | 65536 | 8 | 32 | uncached | 0 | 128 | 2 | 216.3170000000000000 | 1090.0050000000000000 | 5.0389243563843803
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 16 | uncached | 0 | 128 | 2 | 106.5780000000000000 | 535.4800000000000000 | 5.0243014505807953
1000000 | cyclic | 0 | 4096 | 16 | 16 | uncached | 0 | 128 | 2 | 102.7870000000000000 | 515.0000000000000000 | 5.0103612324515746
1000000 | uniform_pages | 0 | 65536 | 32 | 128 | uncached | 0 | 8 | 2 | 179.2900000000000000 | 893.3930000000000000 | 4.9829494115678510
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 16 | uncached | 0 | 128 | 2 | 109.3960000000000000 | 544.4020000000000000 | 4.9764342389118432
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 32 | uncached | 0 | 128 | 2 | 227.0300000000000000 | 1112.7580000000000000 | 4.9013698630136986
1000000 | cyclic | 0 | 65536 | 8 | 2 | uncached | 0 | 8 | 2 | 28.3590000000000000 | 137.1950000000000000 | 4.8377939983779400
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 4 | uncached | 0 | 8 | 2 | 46.3160000000000000 | 223.6080000000000000 | 4.8278780550997495
1000000 | cyclic | 0 | 4096 | 8 | 2 | uncached | 0 | 8 | 2 | 28.2550000000000000 | 134.2970000000000000 | 4.7530348610865334
1000000 | cyclic | 0 | 65536 | 8 | 16 | uncached | 0 | 128 | 2 | 116.0290000000000000 | 523.3420000000000000 | 4.5104413551784468
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 16 | uncached | 0 | 128 | 2 | 122.5160000000000000 | 544.2480000000000000 | 4.4422606026967906
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 8 | uncached | 0 | 128 | 2 | 62.5820000000000000 | 277.7300000000000000 | 4.4378575309194337
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 0 | 128 | 2 | 59.9480000000000000 | 260.9390000000000000 | 4.3527557216254087
1000000 | cyclic | 0 | 65536 | 16 | 8 | uncached | 0 | 128 | 2 | 57.1420000000000000 | 246.7910000000000000 | 4.3189072836092541
1000000 | uniform | 0 | 65536 | 16 | 128 | uncached | 0 | 8 | 2 | 158.5070000000000000 | 683.4640000000000000 | 4.3118852795144694
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 0 | 128 | 2 | 57.2670000000000000 | 245.9700000000000000 | 4.2951438000942951
1000000 | uniform_pages | 0 | 4096 | 16 | 128 | uncached | 0 | 8 | 2 | 216.4120000000000000 | 921.5400000000000000 | 4.2582666395578803
1000000 | cyclic | 0 | 4096 | 8 | 16 | uncached | 0 | 128 | 2 | 122.8530000000000000 | 516.0870000000000000 | 4.2008497960977754
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 8 | uncached | 0 | 128 | 2 | 63.5440000000000000 | 265.6500000000000000 | 4.1805677955432456
1000000 | uniform | 4 | 4096 | 16 | 128 | uncached | 0 | 8 | 2 | 156.7080000000000000 | 654.5110000000000000 | 4.1766278683921689
1000000 | uniform_pages | 0 | 65536 | 16 | 128 | uncached | 0 | 8 | 2 | 221.5370000000000000 | 916.9630000000000000 | 4.1390964037609970
1000000 | uniform | 0 | 4096 | 8 | 256 | uncached | 0 | 8 | 2 | 419.8300000000000000 | 1736.6300000000000000 | 4.1365076340423505
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 0 | 128 | 2 | 62.6310000000000000 | 255.2420000000000000 | 4.0753301080934362
1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 0 | 128 | 2 | 61.5600000000000000 | 249.2100000000000000 | 4.0482456140350877
1000000 | uniform | 0 | 65536 | 8 | 256 | uncached | 0 | 8 | 2 | 422.4420000000000000 | 1682.5090000000000000 | 3.9828165760033330
1000000 | uniform_pages | 0 | 4096 | 8 | 211 | uncached | 0 | 8 | 2 | 525.4410000000000000 | 2058.2480000000000000 | 3.9171819481159635
1000000 | uniform | 0 | 65536 | 32 | 256 | uncached | 256 | 8 | 2 | 194.2230000000000000 | 738.6780000000000000 | 3.8032467833366800
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 8 | uncached | 0 | 128 | 2 | 70.1740000000000000 | 258.6350000000000000 | 3.6856243052982586
1000000 | cyclic | 0 | 65536 | 8 | 8 | uncached | 0 | 128 | 2 | 65.4570000000000000 | 236.2300000000000000 | 3.6089341094153414
1000000 | uniform | 0 | 4096 | 32 | 256 | uncached | 256 | 8 | 2 | 206.7000000000000000 | 719.9230000000000000 | 3.4829366231253024
1000000 | uniform | 4 | 4096 | 32 | 128 | uncached | 256 | 8 | 2 | 114.6170000000000000 | 396.9590000000000000 | 3.4633518587993055
1000000 | uniform | 0 | 4096 | 32 | 128 | uncached | 256 | 8 | 2 | 112.5310000000000000 | 385.1310000000000000 | 3.4224435933209516
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 8 | uncached | 0 | 128 | 2 | 76.9600000000000000 | 253.7770000000000000 | 3.2975181912681913
1000000 | uniform | 4 | 65536 | 32 | 128 | uncached | 256 | 8 | 2 | 117.9170000000000000 | 379.7920000000000000 | 3.2208417785391419
1000000 | uniform | 0 | 4096 | 32 | 64 | uncached | 0 | 8 | 2 | 66.6170000000000000 | 214.0450000000000000 | 3.2130687362084753
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 0 | 128 | 2 | 36.8970000000000000 | 117.3080000000000000 | 3.1793370734748082
1000000 | uniform_pages | 0 | 4096 | 32 | 211 | uncached | 256 | 8 | 2 | 258.8790000000000000 | 804.4270000000000000 | 3.1073474480355687
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 0 | 128 | 2 | 41.3400000000000000 | 127.8690000000000000 | 3.0931059506531205
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 4 | uncached | 0 | 128 | 2 | 38.7230000000000000 | 118.5540000000000000 | 3.0615913023267825
1000000 | uniform_pages | 0 | 4096 | 32 | 128 | uncached | 256 | 8 | 2 | 157.9840000000000000 | 479.5550000000000000 | 3.0354656167713186
1000000 | uniform | 0 | 65536 | 32 | 1024 | uncached | 0 | 128 | 2 | 427.4170000000000000 | 1260.5530000000000000 | 2.9492345882358446
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 0 | 128 | 2 | 38.1820000000000000 | 112.4390000000000000 | 2.9448169294431931
1000000 | cyclic | 0 | 4096 | 32 | 4 | uncached | 0 | 128 | 2 | 32.1370000000000000 | 93.3530000000000000 | 2.9048448828453185
1000000 | cyclic | 0 | 128 | 32 | 4 | uncached | 0 | 128 | 2 | 32.3880000000000000 | 93.5350000000000000 | 2.8879523280227245
1000000 | uniform | 4 | 65536 | 16 | 64 | uncached | 0 | 8 | 2 | 79.0960000000000000 | 226.3410000000000000 | 2.8615985637706079
1000000 | uniform_pages | 0 | 65536 | 32 | 64 | uncached | 0 | 8 | 2 | 92.7190000000000000 | 263.3310000000000000 | 2.8400974988945092
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 0 | 128 | 2 | 32.8580000000000000 | 92.7740000000000000 | 2.8234828656643740
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 2 | uncached | 256 | 8 | 2 | 23.3090000000000000 | 64.9670000000000000 | 2.7872066583723025
1000000 | uniform | 0 | 65536 | 8 | 128 | uncached | 0 | 8 | 2 | 248.8750000000000000 | 691.0720000000000000 | 2.7767835258663988
1000000 | uniform | 4 | 65536 | 8 | 128 | uncached | 0 | 8 | 2 | 251.4070000000000000 | 686.0470000000000000 | 2.7288301439498502
1000000 | cyclic | 0 | 4096 | 16 | 2 | uncached | 256 | 8 | 2 | 19.6590000000000000 | 52.9390000000000000 | 2.6928633195991658
1000000 | cyclic | 0 | 65536 | 16 | 2 | uncached | 256 | 8 | 2 | 18.9040000000000000 | 50.5270000000000000 | 2.6728205670757512
1000000 | uniform_pages | 0 | 4096 | 16 | 64 | uncached | 0 | 8 | 2 | 105.2800000000000000 | 278.6990000000000000 | 2.6472169452887538
1000000 | uniform | 0 | 4096 | 16 | 256 | uncached | 256 | 8 | 2 | 276.5640000000000000 | 728.0710000000000000 | 2.6325588290594582
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 2 | uncached | 256 | 8 | 2 | 24.9930000000000000 | 65.7020000000000000 | 2.6288160684991798
1000000 | uniform | 4 | 4096 | 8 | 128 | uncached | 0 | 8 | 2 | 249.0900000000000000 | 649.7000000000000000 | 2.6082941908547111
1000000 | cyclic_fuzz | 4 | 65536 | 32 | 2 | uncached | 0 | 8 | 2 | 18.7190000000000000 | 48.4340000000000000 | 2.5874245419092900
1000000 | uniform_pages | 0 | 4096 | 16 | 128 | uncached | 256 | 8 | 2 | 197.1760000000000000 | 505.2560000000000000 | 2.5624619629163793
1000000 | uniform_pages | 0 | 4096 | 16 | 211 | uncached | 256 | 8 | 2 | 318.4840000000000000 | 812.6740000000000000 | 2.5516949046105927
1000000 | uniform_pages | 0 | 65536 | 16 | 211 | uncached | 256 | 8 | 2 | 318.0390000000000000 | 811.2280000000000000 | 2.5507186225588686
1000000 | cyclic | 0 | 4096 | 16 | 4 | uncached | 0 | 128 | 2 | 34.5700000000000000 | 86.4800000000000000 | 2.5015909748336708
1000000 | cyclic | 4 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 17.6580000000000000 | 43.6340000000000000 | 2.4710612753426209
1000000 | uniform | 0 | 4096 | 32 | 512 | uncached | 256 | 8 | 2 | 293.6830000000000000 | 716.3800000000000000 | 2.4392967928004004
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 4 | uncached | 0 | 128 | 2 | 46.9810000000000000 | 112.9930000000000000 | 2.4050786488154786
1000000 | uniform | 0 | 4096 | 16 | 128 | uncached | 256 | 8 | 2 | 157.6140000000000000 | 377.7630000000000000 | 2.3967604400624310
1000000 | cyclic_fuzz | 4 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 19.3970000000000000 | 46.3440000000000000 | 2.3892354487807393
1000000 | uniform | 0 | 65536 | 8 | 1024 | uncached | 0 | 128 | 2 | 548.1220000000000000 | 1307.4100000000000000 | 2.3852536479105017
1000000 | cyclic | 4 | 128 | 32 | 2 | uncached | 0 | 8 | 2 | 17.1170000000000000 | 40.8000000000000000 | 2.3835952561780686
1000000 | uniform | 4 | 65536 | 16 | 128 | uncached | 256 | 8 | 2 | 154.9780000000000000 | 366.8740000000000000 | 2.3672650311657138
1000000 | cyclic | 0 | 128 | 16 | 4 | uncached | 256 | 8 | 2 | 30.4870000000000000 | 70.9150000000000000 | 2.3260734083379801
1000000 | uniform | 4 | 65536 | 32 | 2048 | uncached | 0 | 128 | 2 | 264.8540000000000000 | 615.1720000000000000 | 2.3226834406880772
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 4 | uncached | 0 | 128 | 2 | 50.5090000000000000 | 116.0020000000000000 | 2.2966600011879071
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 256 | 8 | 2 | 29.7680000000000000 | 67.9860000000000000 | 2.2838618650900296
1000000 | cyclic | 4 | 65536 | 32 | 2 | uncached | 0 | 8 | 2 | 18.4270000000000000 | 42.0330000000000000 | 2.2810549736799262
1000000 | uniform | 4 | 4096 | 32 | 2048 | uncached | 0 | 128 | 2 | 260.3540000000000000 | 592.9440000000000000 | 2.2774530062914340
1000000 | uniform | 4 | 4096 | 32 | 64 | uncached | 256 | 8 | 2 | 69.7780000000000000 | 158.7150000000000000 | 2.2745707816217146
1000000 | uniform | 0 | 4096 | 8 | 1024 | uncached | 0 | 128 | 2 | 579.3350000000000000 | 1299.1340000000000000 | 2.2424573001803792
1000000 | uniform_pages | 0 | 4096 | 32 | 64 | uncached | 256 | 8 | 2 | 85.3150000000000000 | 188.8470000000000000 | 2.2135263435503722
1000000 | uniform | 0 | 4096 | 32 | 32 | uncached | 0 | 8 | 2 | 38.1710000000000000 | 84.1300000000000000 | 2.2040292368552042
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 2 | uncached | 256 | 8 | 2 | 29.6710000000000000 | 64.5820000000000000 | 2.1766034174783459
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 256 | 8 | 2 | 33.0140000000000000 | 71.8290000000000000 | 2.1757133337372024
1000000 | uniform | 0 | 4096 | 32 | 64 | uncached | 256 | 8 | 2 | 70.5350000000000000 | 152.9810000000000000 | 2.1688665201672928
1000000 | uniform | 4 | 65536 | 16 | 32 | uncached | 0 | 8 | 2 | 40.9560000000000000 | 86.4240000000000000 | 2.1101670084969235
1000000 | cyclic | 4 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 20.2450000000000000 | 42.5380000000000000 | 2.1011607804396147
1000000 | uniform | 0 | 4096 | 0 | 512 | uncached | 256 | 128 | 2 | 594.5600000000000000 | 1247.3030000000000000 | 2.0978589208826695
1000000 | cyclic_fuzz | 4 | 128 | 32 | 4 | uncached | 0 | 8 | 2 | 25.7620000000000000 | 54.0310000000000000 | 2.0973138731464948
1000000 | cyclic | 0 | 65536 | 8 | 4 | uncached | 0 | 128 | 2 | 40.6250000000000000 | 84.8690000000000000 | 2.0890830769230769
1000000 | cyclic_fuzz | 4 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 22.1670000000000000 | 46.0590000000000000 | 2.0778183786709974
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 4 | uncached | 256 | 8 | 2 | 35.3820000000000000 | 73.3000000000000000 | 2.0716748629246510
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 256 | 8 | 2 | 35.2720000000000000 | 73.0640000000000000 | 2.0714447720571558
1000000 | cyclic_fuzz | 4 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 26.9920000000000000 | 55.7600000000000000 | 2.0657972732661529
1000000 | uniform_pages | 0 | 65536 | 32 | 32 | uncached | 0 | 8 | 2 | 51.6610000000000000 | 105.8500000000000000 | 2.0489343992566927
1000000 | cyclic_fuzz | 0 | 4096 | 1 | 4 | uncached | 0 | 8 | 2 | 118.4810000000000000 | 241.3620000000000000 | 2.0371367561043543
1000000 | uniform | 0 | 65536 | 16 | 512 | uncached | 256 | 8 | 2 | 371.4000000000000000 | 754.8360000000000000 | 2.0324071082390953
1000000 | cyclic_fuzz | 4 | 65536 | 16 | 2 | uncached | 0 | 8 | 2 | 22.3170000000000000 | 45.1480000000000000 | 2.0230317695030694
1000000 | cyclic | 0 | 4096 | 8 | 2 | uncached | 256 | 8 | 2 | 26.1960000000000000 | 52.8250000000000000 | 2.0165292411055123
1000000 | cyclic | 4 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 23.1710000000000000 | 46.7180000000000000 | 2.0162271805273834
1000000 | cyclic | 4 | 128 | 32 | 4 | uncached | 0 | 8 | 2 | 23.1320000000000000 | 46.4200000000000000 | 2.0067439045478126
1000000 | uniform | 0 | 65536 | 32 | 32 | uncached | 0 | 8 | 2 | 37.3090000000000000 | 74.6180000000000000 | 2.0000000000000000
1000000 | cyclic_fuzz | 4 | 128 | 16 | 2 | uncached | 0 | 8 | 2 | 21.8060000000000000 | 43.5620000000000000 | 1.9977070531046501
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 256 | 8 | 2 | 36.1660000000000000 | 71.9780000000000000 | 1.9902118011391915
1000000 | uniform | 4 | 4096 | 8 | 64 | uncached | 0 | 8 | 2 | 128.0980000000000000 | 253.8370000000000000 | 1.9815844119346126
1000000 | uniform | 0 | 4096 | 8 | 256 | uncached | 256 | 8 | 2 | 419.2600000000000000 | 823.3580000000000000 | 1.9638362829747651
1000000 | uniform | 4 | 65536 | 8 | 64 | uncached | 0 | 8 | 2 | 126.1750000000000000 | 247.4340000000000000 | 1.9610382405389340
1000000 | uniform | 4 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 42.8650000000000000 | 83.2560000000000000 | 1.9422839146156538
1000000 | uniform | 0 | 65536 | 8 | 256 | uncached | 256 | 8 | 2 | 423.4570000000000000 | 817.4440000000000000 | 1.9304061569415549
1000000 | uniform | 0 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 41.7190000000000000 | 80.1330000000000000 | 1.9207795009468108
1000000 | uniform | 0 | 65536 | 0 | 512 | uncached | 256 | 128 | 2 | 649.1030000000000000 | 1244.8640000000000000 | 1.9178219789463306
1000000 | uniform_pages | 0 | 4096 | 16 | 64 | uncached | 256 | 8 | 2 | 95.7390000000000000 | 183.5780000000000000 | 1.9174839929391366
1000000 | cyclic | 4 | 128 | 16 | 2 | uncached | 0 | 8 | 2 | 20.7090000000000000 | 39.5300000000000000 | 1.9088319088319088
1000000 | cyclic | 4 | 65536 | 16 | 2 | uncached | 0 | 8 | 2 | 19.6800000000000000 | 37.3750000000000000 | 1.8991361788617886
1000000 | uniform_pages | 0 | 65536 | 16 | 32 | uncached | 0 | 8 | 2 | 54.5230000000000000 | 102.9430000000000000 | 1.8880655869999817
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 2 | uncached | 0 | 128 | 2 | 24.0170000000000000 | 44.3860000000000000 | 1.8481075904567598
1000000 | cyclic | 0 | 128 | 8 | 4 | uncached | 256 | 8 | 2 | 38.9400000000000000 | 71.9120000000000000 | 1.8467385721623010
1000000 | cyclic_fuzz | 4 | 65536 | 32 | 8 | uncached | 0 | 8 | 2 | 41.2820000000000000 | 76.0580000000000000 | 1.8424010464609273
1000000 | uniform | 0 | 65536 | 16 | 64 | uncached | 256 | 8 | 2 | 79.0730000000000000 | 145.6630000000000000 | 1.8421332186713543
1000000 | uniform | 4 | 65536 | 32 | 32 | uncached | 256 | 8 | 2 | 36.1090000000000000 | 66.3960000000000000 | 1.8387659586252735
1000000 | uniform | 4 | 4096 | 16 | 2048 | uncached | 0 | 128 | 2 | 332.7450000000000000 | 607.7430000000000000 | 1.8264526889960781
1000000 | cyclic | 4 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 38.8040000000000000 | 70.6930000000000000 | 1.8217967219874240
1000000 | uniform_pages | 0 | 65536 | 8 | 128 | uncached | 256 | 8 | 2 | 304.1250000000000000 | 534.8750000000000000 | 1.7587340731607069
1000000 | uniform_pages | 0 | 4096 | 8 | 128 | uncached | 256 | 8 | 2 | 302.8000000000000000 | 531.6400000000000000 | 1.7557463672391017
1000000 | uniform_pages | 0 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 53.4580000000000000 | 93.3620000000000000 | 1.7464551610610199
1000000 | uniform_pages | 0 | 65536 | 0 | 211 | uncached | 256 | 128 | 2 | 823.1190000000000000 | 1431.1640000000000000 | 1.7387084977992247
1000000 | uniform | 4 | 65536 | 8 | 128 | uncached | 256 | 8 | 2 | 241.5110000000000000 | 408.9400000000000000 | 1.6932562077917776
1000000 | cyclic_fuzz | 4 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 39.9330000000000000 | 67.5440000000000000 | 1.6914331505271329
1000000 | uniform_pages | 0 | 4096 | 0 | 211 | uncached | 256 | 128 | 2 | 835.8970000000000000 | 1407.5400000000000000 | 1.6838677492561883
1000000 | uniform | 4 | 4096 | 8 | 128 | uncached | 256 | 8 | 2 | 250.1520000000000000 | 420.7920000000000000 | 1.6821452556845438
1000000 | cyclic_fuzz | 0 | 4096 | 1 | 32 | uncached | 0 | 8 | 2 | 757.9950000000000000 | 1274.2050000000000000 | 1.6810203233530564
1000000 | cyclic | 0 | 128 | 16 | 4 | uncached | 256 | 128 | 2 | 28.6620000000000000 | 47.9570000000000000 | 1.6731909845788849
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 16 | uncached | 0 | 8 | 2 | 420.6100000000000000 | 695.4390000000000000 | 1.6534057678134139
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 256 | 8 | 2 | 51.8940000000000000 | 85.7260000000000000 | 1.6519443480941920
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 8 | uncached | 0 | 8 | 2 | 243.8420000000000000 | 402.3430000000000000 | 1.6500151737600577
1000000 | uniform | 4 | 4096 | 16 | 16 | uncached | 0 | 8 | 2 | 21.1470000000000000 | 34.8840000000000000 | 1.6495956873315364
1000000 | cyclic | 0 | 4096 | 32 | 4 | uncached | 256 | 128 | 2 | 28.4450000000000000 | 46.7740000000000000 | 1.6443663209702935
1000000 | uniform | 0 | 65536 | 8 | 32 | uncached | 0 | 8 | 2 | 58.8180000000000000 | 96.5620000000000000 | 1.6417083205821347
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 256 | 128 | 2 | 48.6230000000000000 | 79.5400000000000000 | 1.6358513460707895
1000000 | uniform | 4 | 4096 | 32 | 4 | cached | 256 | 128 | 2 | 0.76200000000000000000 | 1.24600000000000000000 | 1.63517060367454068241
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 2 | uncached | 0 | 8 | 2 | 104.5580000000000000 | 170.5210000000000000 | 1.6308747298150309
1000000 | uniform | 4 | 65536 | 32 | 16 | uncached | 0 | 8 | 2 | 20.9970000000000000 | 34.0890000000000000 | 1.6235176453779111
1000000 | cyclic_fuzz | 0 | 4096 | 1 | 16 | uncached | 0 | 8 | 2 | 428.8660000000000000 | 695.0810000000000000 | 1.6207416768874194
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 256 | 8 | 2 | 56.5290000000000000 | 90.8090000000000000 | 1.6064144067646695
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 4 | uncached | 0 | 8 | 2 | 151.9910000000000000 | 243.1920000000000000 | 1.6000421077563803
1000000 | uniform | 0 | 4096 | 8 | 32 | uncached | 0 | 8 | 2 | 60.7930000000000000 | 96.3030000000000000 | 1.5841133025183821
1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 256 | 8 | 2 | 55.6950000000000000 | 87.8090000000000000 | 1.5766047221474100
1000000 | uniform | 4 | 65536 | 1 | 1024 | uncached | 256 | 128 | 2 | 655.7590000000000000 | 1029.9260000000000000 | 1.5705861452149341
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 256 | 128 | 2 | 32.6060000000000000 | 51.1150000000000000 | 1.5676562595841256
1000000 | cyclic | 0 | 4096 | 16 | 4 | uncached | 256 | 128 | 2 | 30.7160000000000000 | 48.0980000000000000 | 1.5658939966141425
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 256 | 8 | 2 | 55.8840000000000000 | 87.4530000000000000 | 1.5649022976164913
1000000 | cyclic | 0 | 4096 | 1 | 32 | uncached | 0 | 8 | 2 | 807.6740000000000000 | 1259.1210000000000000 | 1.5589470504188571
1000000 | uniform | 4 | 4096 | 1 | 1024 | uncached | 256 | 128 | 2 | 657.6370000000000000 | 1024.9000000000000000 | 1.5584585417183036
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 256 | 128 | 2 | 29.7980000000000000 | 46.4080000000000000 | 1.5574199610712128
1000000 | uniform | 4 | 4096 | 32 | 16 | uncached | 0 | 8 | 2 | 19.9800000000000000 | 31.0990000000000000 | 1.5565065065065065
1000000 | uniform | 4 | 4096 | 1 | 2048 | uncached | 256 | 128 | 2 | 599.4480000000000000 | 930.6370000000000000 | 1.5524899574275000
1000000 | cyclic | 0 | 65536 | 1 | 32 | uncached | 0 | 8 | 2 | 813.1960000000000000 | 1248.6470000000000000 | 1.5354809910525876
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 256 | 128 | 2 | 52.5890000000000000 | 80.6930000000000000 | 1.5344083363441024
1000000 | cyclic | 0 | 65536 | 1 | 16 | uncached | 0 | 8 | 2 | 438.0680000000000000 | 671.1000000000000000 | 1.5319539432234265
1000000 | uniform | 4 | 65536 | 1 | 2048 | uncached | 256 | 128 | 2 | 610.4480000000000000 | 933.1770000000000000 | 1.5286756611537756
1000000 | uniform | 0 | 65536 | 1 | 2048 | uncached | 0 | 8 | 2 | 2372.3930000000000000 | 3624.7540000000000000 | 1.5278893505418369
1000000 | cyclic | 0 | 4096 | 1 | 52 | uncached | 0 | 8 | 2 | 1314.9080000000000000 | 2004.4420000000000000 | 1.5243971441347988
1000000 | uniform | 0 | 65536 | 16 | 32 | uncached | 256 | 8 | 2 | 40.2340000000000000 | 61.3310000000000000 | 1.5243575085748372
1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 256 | 128 | 2 | 48.6230000000000000 | 73.9910000000000000 | 1.5217284001398515
1000000 | uniform | 0 | 65536 | 16 | 8 | uncached | 0 | 8 | 2 | 12.8680000000000000 | 19.5690000000000000 | 1.5207491451663040
1000000 | cyclic | 0 | 4096 | 1 | 8 | uncached | 0 | 8 | 2 | 238.0610000000000000 | 361.4180000000000000 | 1.5181739134087482
1000000 | uniform | 4 | 65536 | 8 | 32 | uncached | 0 | 8 | 2 | 63.2790000000000000 | 96.0280000000000000 | 1.5175334629181877
1000000 | uniform_pages | 4 | 4096 | 0 | 4 | cached | 256 | 8 | 2 | 1.74500000000000000000 | 2.6380000000000000 | 1.51174785100286532951
1000000 | cyclic | 0 | 4096 | 1 | 16 | uncached | 0 | 8 | 2 | 450.1170000000000000 | 679.8840000000000000 | 1.5104606135738041
1000000 | uniform | 0 | 65536 | 1 | 4048 | uncached | 0 | 8 | 2 | 2384.6790000000000000 | 3601.4130000000000000 | 1.5102296787114744
1000000 | cyclic | 0 | 65536 | 1 | 8 | uncached | 0 | 8 | 2 | 232.1420000000000000 | 350.5240000000000000 | 1.5099551136804197
1000000 | cyclic | 0 | 4096 | 1 | 4 | uncached | 0 | 8 | 2 | 143.2900000000000000 | 215.2850000000000000 | 1.5024425989252565
1000000 | cyclic | 0 | 128 | 1 | 4 | uncached | 0 | 8 | 2 | 141.2470000000000000 | 211.7170000000000000 | 1.4989132512548939
Attachments:
[application/x-shellscript] run-randomized.sh (9.2K, ../../[email protected]/2-run-randomized.sh)
download
[application/pdf] results.pdf (252.0K, ../../[email protected]/3-results.pdf)
download
[text/plain] results.txt (44.3K, ../../[email protected]/4-results.txt)
download | inline:
rows | dataset | workers | wm | eic | matches | caching | readahead | combine_limit | cnt | master | patched | ?column?
---------+---------------+---------+-------+-----+---------+-----------+-----------+---------------+-----+------------------------+------------------------+------------------------
1000000 | uniform | 0 | 4096 | 32 | 256 | uncached | 0 | 8 | 2 | 197.3750000000000000 | 1710.7450000000000000 | 8.6674857504749842
1000000 | uniform | 0 | 4096 | 32 | 1024 | uncached | 0 | 8 | 2 | 388.3380000000000000 | 3323.4390000000000000 | 8.5581091729369776
1000000 | uniform | 0 | 65536 | 32 | 1024 | uncached | 0 | 8 | 2 | 407.7540000000000000 | 3380.9000000000000000 | 8.2915189060070533
1000000 | uniform | 0 | 65536 | 32 | 2048 | uncached | 0 | 8 | 2 | 439.8440000000000000 | 3547.9540000000000000 | 8.0663917207009758
1000000 | uniform | 0 | 65536 | 16 | 1024 | uncached | 0 | 8 | 2 | 424.1210000000000000 | 3404.8380000000000000 | 8.0279872960782418
1000000 | uniform | 0 | 4096 | 16 | 2048 | uncached | 0 | 8 | 2 | 469.9440000000000000 | 3614.8390000000000000 | 7.6920633096709395
1000000 | uniform | 0 | 4096 | 32 | 2048 | uncached | 0 | 8 | 2 | 470.5050000000000000 | 3561.9470000000000000 | 7.5704764030137831
1000000 | uniform | 0 | 65536 | 16 | 4048 | uncached | 0 | 128 | 2 | 474.8690000000000000 | 3517.6140000000000000 | 7.4075460811297432
1000000 | uniform | 0 | 4096 | 32 | 4048 | uncached | 0 | 128 | 2 | 473.5260000000000000 | 3474.0360000000000000 | 7.3365263998175391
1000000 | uniform_pages | 0 | 65536 | 32 | 211 | uncached | 0 | 8 | 2 | 284.9390000000000000 | 2050.9960000000000000 | 7.1980178213582556
1000000 | uniform | 0 | 4096 | 32 | 4048 | uncached | 0 | 8 | 2 | 520.9200000000000000 | 3626.4810000000000000 | 6.9616850956000921
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 52 | uncached | 0 | 8 | 2 | 295.4450000000000000 | 2054.9440000000000000 | 6.9554197904855388
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 16 | uncached | 0 | 8 | 2 | 99.9790000000000000 | 691.6020000000000000 | 6.9174726692605447
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 34.9790000000000000 | 241.0800000000000000 | 6.8921352811687012
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 16 | uncached | 0 | 8 | 2 | 101.0870000000000000 | 695.6470000000000000 | 6.8816662874553602
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 191.7020000000000000 | 1303.3400000000000000 | 6.7987814420298171
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 24.0290000000000000 | 159.4270000000000000 | 6.6347746473011777
1000000 | cyclic | 0 | 65536 | 16 | 4 | uncached | 0 | 8 | 2 | 33.1760000000000000 | 219.1070000000000000 | 6.6043826862792380
1000000 | cyclic | 0 | 65536 | 16 | 8 | uncached | 0 | 8 | 2 | 55.6630000000000000 | 367.3740000000000000 | 6.5999676625406464
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 2 | uncached | 0 | 8 | 2 | 24.1790000000000000 | 159.3560000000000000 | 6.5906778609537202
1000000 | cyclic | 0 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 20.7600000000000000 | 136.4900000000000000 | 6.5746628131021195
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 16 | uncached | 0 | 8 | 2 | 105.8640000000000000 | 693.0790000000000000 | 6.5468809038011033
1000000 | cyclic | 0 | 65536 | 16 | 2 | uncached | 0 | 8 | 2 | 21.0990000000000000 | 137.9350000000000000 | 6.5375136262382103
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 0 | 8 | 2 | 59.9430000000000000 | 386.9210000000000000 | 6.4548154079709057
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 52 | uncached | 0 | 8 | 2 | 311.6300000000000000 | 1999.8530000000000000 | 6.4173956294323396
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 32 | uncached | 0 | 8 | 2 | 204.7160000000000000 | 1311.0530000000000000 | 6.4042527208425331
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 32 | uncached | 0 | 8 | 2 | 196.1600000000000000 | 1255.4800000000000000 | 6.4002854812398042
1000000 | uniform | 0 | 65536 | 8 | 2048 | uncached | 0 | 8 | 2 | 566.2560000000000000 | 3598.4170000000000000 | 6.3547529739199231
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 0 | 8 | 2 | 36.9560000000000000 | 234.7780000000000000 | 6.3529061586751813
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 56.7750000000000000 | 360.2920000000000000 | 6.3459621312197270
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 8 | uncached | 0 | 8 | 2 | 59.7330000000000000 | 378.6380000000000000 | 6.3388411765690657
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 52 | uncached | 0 | 8 | 2 | 330.1280000000000000 | 2089.7620000000000000 | 6.3301567876702370
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 0 | 8 | 2 | 36.9260000000000000 | 230.7940000000000000 | 6.2501760277311380
1000000 | cyclic | 0 | 4096 | 32 | 52 | uncached | 0 | 128 | 2 | 285.1420000000000000 | 1779.4320000000000000 | 6.2405117450252856
1000000 | cyclic | 0 | 4096 | 32 | 32 | uncached | 0 | 8 | 2 | 200.0930000000000000 | 1244.2080000000000000 | 6.2181485609191726
1000000 | uniform | 0 | 4096 | 8 | 2048 | uncached | 0 | 8 | 2 | 579.0620000000000000 | 3600.1250000000000000 | 6.2171667282605317
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 52 | uncached | 0 | 128 | 2 | 292.2530000000000000 | 1804.1090000000000000 | 6.1731068628893459
1000000 | uniform | 0 | 4096 | 16 | 256 | uncached | 0 | 8 | 2 | 275.0420000000000000 | 1696.9760000000000000 | 6.1698795093113052
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 27.1780000000000000 | 167.3230000000000000 | 6.1565604533078225
1000000 | uniform | 0 | 4096 | 8 | 4048 | uncached | 0 | 8 | 2 | 581.9640000000000000 | 3582.8660000000000000 | 6.1565079626918504
1000000 | cyclic | 0 | 4096 | 16 | 52 | uncached | 0 | 8 | 2 | 321.3510000000000000 | 1974.7440000000000000 | 6.1451310249540222
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 52 | uncached | 0 | 128 | 2 | 294.6290000000000000 | 1801.8960000000000000 | 6.1158134467414952
1000000 | uniform | 0 | 65536 | 8 | 1024 | uncached | 0 | 8 | 2 | 562.2080000000000000 | 3422.6260000000000000 | 6.0878287039672150
1000000 | cyclic | 0 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 22.7190000000000000 | 137.5250000000000000 | 6.0533034024384876
1000000 | cyclic | 0 | 65536 | 16 | 16 | uncached | 0 | 8 | 2 | 106.0580000000000000 | 641.5630000000000000 | 6.0491712082068302
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 32 | uncached | 0 | 128 | 2 | 191.1380000000000000 | 1154.9810000000000000 | 6.0426550450459877
1000000 | cyclic | 0 | 65536 | 16 | 52 | uncached | 0 | 128 | 2 | 306.2530000000000000 | 1847.7090000000000000 | 6.0332764087208942
1000000 | uniform | 0 | 4096 | 8 | 4048 | uncached | 0 | 128 | 2 | 591.1120000000000000 | 3544.5280000000000000 | 5.9963729377850560
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 0 | 8 | 2 | 59.7360000000000000 | 354.0220000000000000 | 5.9264430159367885
1000000 | uniform | 0 | 4096 | 8 | 1024 | uncached | 0 | 8 | 2 | 563.3670000000000000 | 3332.3490000000000000 | 5.9150589225140983
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 68.2930000000000000 | 403.5720000000000000 | 5.9094197062656495
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 52 | uncached | 0 | 128 | 2 | 312.1930000000000000 | 1826.3510000000000000 | 5.8500703090716320
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 32 | uncached | 0 | 128 | 2 | 194.5990000000000000 | 1136.9650000000000000 | 5.8426045354806551
1000000 | uniform_pages | 0 | 65536 | 16 | 211 | uncached | 0 | 8 | 2 | 349.2250000000000000 | 2036.6470000000000000 | 5.8319049323502040
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 32 | uncached | 0 | 8 | 2 | 224.9200000000000000 | 1299.4710000000000000 | 5.7774808820914103
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 36.7270000000000000 | 210.6900000000000000 | 5.7366515097884390
1000000 | uniform | 4 | 65536 | 32 | 128 | uncached | 0 | 8 | 2 | 117.8920000000000000 | 672.3060000000000000 | 5.7027279204695823
1000000 | cyclic | 0 | 4096 | 32 | 32 | uncached | 0 | 128 | 2 | 194.4810000000000000 | 1095.4010000000000000 | 5.6324319599343895
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 16 | uncached | 0 | 8 | 2 | 122.4230000000000000 | 689.0290000000000000 | 5.6282642967416253
1000000 | cyclic | 0 | 4096 | 8 | 8 | uncached | 0 | 8 | 2 | 66.8700000000000000 | 376.3480000000000000 | 5.6280544339763721
1000000 | cyclic | 0 | 65536 | 32 | 32 | uncached | 0 | 128 | 2 | 192.0460000000000000 | 1075.7940000000000000 | 5.6017516636639139
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 52 | uncached | 0 | 128 | 2 | 340.7140000000000000 | 1899.0370000000000000 | 5.5736981750089518
1000000 | uniform | 0 | 65536 | 32 | 128 | uncached | 0 | 8 | 2 | 118.6750000000000000 | 654.2880000000000000 | 5.5132757531072256
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 8 | uncached | 0 | 8 | 2 | 70.9930000000000000 | 390.8210000000000000 | 5.5050638795374192
1000000 | uniform | 0 | 4096 | 32 | 128 | uncached | 0 | 8 | 2 | 123.2220000000000000 | 678.0760000000000000 | 5.5028809790459496
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 16 | uncached | 0 | 8 | 2 | 125.0850000000000000 | 686.1930000000000000 | 5.4858136467202302
1000000 | cyclic | 0 | 65536 | 8 | 16 | uncached | 0 | 8 | 2 | 116.4450000000000000 | 635.9870000000000000 | 5.4616943621452188
1000000 | cyclic | 0 | 65536 | 8 | 8 | uncached | 0 | 8 | 2 | 67.3550000000000000 | 365.1860000000000000 | 5.4218098136738178
1000000 | cyclic | 0 | 65536 | 8 | 4 | uncached | 0 | 8 | 2 | 40.0240000000000000 | 216.1760000000000000 | 5.4011593044173496
1000000 | cyclic | 0 | 4096 | 32 | 16 | uncached | 0 | 128 | 2 | 98.5040000000000000 | 522.8400000000000000 | 5.3078047591975960
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 2 | uncached | 0 | 8 | 2 | 31.1260000000000000 | 165.0880000000000000 | 5.3038617233181263
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 16 | uncached | 0 | 128 | 2 | 100.8680000000000000 | 533.8580000000000000 | 5.2926398857913312
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 52 | uncached | 0 | 128 | 2 | 359.3540000000000000 | 1900.1160000000000000 | 5.2875882834196920
1000000 | cyclic | 0 | 128 | 8 | 4 | uncached | 0 | 8 | 2 | 40.6010000000000000 | 212.8740000000000000 | 5.2430728307184552
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 4 | uncached | 0 | 8 | 2 | 44.8610000000000000 | 234.9410000000000000 | 5.2370878937161454
1000000 | uniform | 0 | 65536 | 8 | 512 | uncached | 0 | 8 | 2 | 561.6130000000000000 | 2927.0930000000000000 | 5.2119395384366103
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 2 | uncached | 0 | 8 | 2 | 32.5860000000000000 | 167.2210000000000000 | 5.1316823175596882
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 52 | uncached | 0 | 128 | 2 | 354.6300000000000000 | 1806.6930000000000000 | 5.0945859064376956
1000000 | cyclic | 0 | 4096 | 8 | 4 | uncached | 0 | 8 | 2 | 41.0710000000000000 | 209.0330000000000000 | 5.0895522388059701
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 16 | uncached | 0 | 128 | 2 | 108.6830000000000000 | 547.8050000000000000 | 5.0403927017104791
1000000 | cyclic | 0 | 65536 | 8 | 32 | uncached | 0 | 128 | 2 | 216.3170000000000000 | 1090.0050000000000000 | 5.0389243563843803
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 16 | uncached | 0 | 128 | 2 | 106.5780000000000000 | 535.4800000000000000 | 5.0243014505807953
1000000 | cyclic | 0 | 4096 | 16 | 16 | uncached | 0 | 128 | 2 | 102.7870000000000000 | 515.0000000000000000 | 5.0103612324515746
1000000 | uniform_pages | 0 | 65536 | 32 | 128 | uncached | 0 | 8 | 2 | 179.2900000000000000 | 893.3930000000000000 | 4.9829494115678510
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 16 | uncached | 0 | 128 | 2 | 109.3960000000000000 | 544.4020000000000000 | 4.9764342389118432
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 32 | uncached | 0 | 128 | 2 | 227.0300000000000000 | 1112.7580000000000000 | 4.9013698630136986
1000000 | cyclic | 0 | 65536 | 8 | 2 | uncached | 0 | 8 | 2 | 28.3590000000000000 | 137.1950000000000000 | 4.8377939983779400
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 4 | uncached | 0 | 8 | 2 | 46.3160000000000000 | 223.6080000000000000 | 4.8278780550997495
1000000 | cyclic | 0 | 4096 | 8 | 2 | uncached | 0 | 8 | 2 | 28.2550000000000000 | 134.2970000000000000 | 4.7530348610865334
1000000 | cyclic | 0 | 65536 | 8 | 16 | uncached | 0 | 128 | 2 | 116.0290000000000000 | 523.3420000000000000 | 4.5104413551784468
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 16 | uncached | 0 | 128 | 2 | 122.5160000000000000 | 544.2480000000000000 | 4.4422606026967906
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 8 | uncached | 0 | 128 | 2 | 62.5820000000000000 | 277.7300000000000000 | 4.4378575309194337
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 0 | 128 | 2 | 59.9480000000000000 | 260.9390000000000000 | 4.3527557216254087
1000000 | cyclic | 0 | 65536 | 16 | 8 | uncached | 0 | 128 | 2 | 57.1420000000000000 | 246.7910000000000000 | 4.3189072836092541
1000000 | uniform | 0 | 65536 | 16 | 128 | uncached | 0 | 8 | 2 | 158.5070000000000000 | 683.4640000000000000 | 4.3118852795144694
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 0 | 128 | 2 | 57.2670000000000000 | 245.9700000000000000 | 4.2951438000942951
1000000 | uniform_pages | 0 | 4096 | 16 | 128 | uncached | 0 | 8 | 2 | 216.4120000000000000 | 921.5400000000000000 | 4.2582666395578803
1000000 | cyclic | 0 | 4096 | 8 | 16 | uncached | 0 | 128 | 2 | 122.8530000000000000 | 516.0870000000000000 | 4.2008497960977754
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 8 | uncached | 0 | 128 | 2 | 63.5440000000000000 | 265.6500000000000000 | 4.1805677955432456
1000000 | uniform | 4 | 4096 | 16 | 128 | uncached | 0 | 8 | 2 | 156.7080000000000000 | 654.5110000000000000 | 4.1766278683921689
1000000 | uniform_pages | 0 | 65536 | 16 | 128 | uncached | 0 | 8 | 2 | 221.5370000000000000 | 916.9630000000000000 | 4.1390964037609970
1000000 | uniform | 0 | 4096 | 8 | 256 | uncached | 0 | 8 | 2 | 419.8300000000000000 | 1736.6300000000000000 | 4.1365076340423505
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 0 | 128 | 2 | 62.6310000000000000 | 255.2420000000000000 | 4.0753301080934362
1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 0 | 128 | 2 | 61.5600000000000000 | 249.2100000000000000 | 4.0482456140350877
1000000 | uniform | 0 | 65536 | 8 | 256 | uncached | 0 | 8 | 2 | 422.4420000000000000 | 1682.5090000000000000 | 3.9828165760033330
1000000 | uniform_pages | 0 | 4096 | 8 | 211 | uncached | 0 | 8 | 2 | 525.4410000000000000 | 2058.2480000000000000 | 3.9171819481159635
1000000 | uniform | 0 | 65536 | 32 | 256 | uncached | 256 | 8 | 2 | 194.2230000000000000 | 738.6780000000000000 | 3.8032467833366800
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 8 | uncached | 0 | 128 | 2 | 70.1740000000000000 | 258.6350000000000000 | 3.6856243052982586
1000000 | cyclic | 0 | 65536 | 8 | 8 | uncached | 0 | 128 | 2 | 65.4570000000000000 | 236.2300000000000000 | 3.6089341094153414
1000000 | uniform | 0 | 4096 | 32 | 256 | uncached | 256 | 8 | 2 | 206.7000000000000000 | 719.9230000000000000 | 3.4829366231253024
1000000 | uniform | 4 | 4096 | 32 | 128 | uncached | 256 | 8 | 2 | 114.6170000000000000 | 396.9590000000000000 | 3.4633518587993055
1000000 | uniform | 0 | 4096 | 32 | 128 | uncached | 256 | 8 | 2 | 112.5310000000000000 | 385.1310000000000000 | 3.4224435933209516
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 8 | uncached | 0 | 128 | 2 | 76.9600000000000000 | 253.7770000000000000 | 3.2975181912681913
1000000 | uniform | 4 | 65536 | 32 | 128 | uncached | 256 | 8 | 2 | 117.9170000000000000 | 379.7920000000000000 | 3.2208417785391419
1000000 | uniform | 0 | 4096 | 32 | 64 | uncached | 0 | 8 | 2 | 66.6170000000000000 | 214.0450000000000000 | 3.2130687362084753
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 0 | 128 | 2 | 36.8970000000000000 | 117.3080000000000000 | 3.1793370734748082
1000000 | uniform_pages | 0 | 4096 | 32 | 211 | uncached | 256 | 8 | 2 | 258.8790000000000000 | 804.4270000000000000 | 3.1073474480355687
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 0 | 128 | 2 | 41.3400000000000000 | 127.8690000000000000 | 3.0931059506531205
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 4 | uncached | 0 | 128 | 2 | 38.7230000000000000 | 118.5540000000000000 | 3.0615913023267825
1000000 | uniform_pages | 0 | 4096 | 32 | 128 | uncached | 256 | 8 | 2 | 157.9840000000000000 | 479.5550000000000000 | 3.0354656167713186
1000000 | uniform | 0 | 65536 | 32 | 1024 | uncached | 0 | 128 | 2 | 427.4170000000000000 | 1260.5530000000000000 | 2.9492345882358446
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 0 | 128 | 2 | 38.1820000000000000 | 112.4390000000000000 | 2.9448169294431931
1000000 | cyclic | 0 | 4096 | 32 | 4 | uncached | 0 | 128 | 2 | 32.1370000000000000 | 93.3530000000000000 | 2.9048448828453185
1000000 | cyclic | 0 | 128 | 32 | 4 | uncached | 0 | 128 | 2 | 32.3880000000000000 | 93.5350000000000000 | 2.8879523280227245
1000000 | uniform | 4 | 65536 | 16 | 64 | uncached | 0 | 8 | 2 | 79.0960000000000000 | 226.3410000000000000 | 2.8615985637706079
1000000 | uniform_pages | 0 | 65536 | 32 | 64 | uncached | 0 | 8 | 2 | 92.7190000000000000 | 263.3310000000000000 | 2.8400974988945092
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 0 | 128 | 2 | 32.8580000000000000 | 92.7740000000000000 | 2.8234828656643740
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 2 | uncached | 256 | 8 | 2 | 23.3090000000000000 | 64.9670000000000000 | 2.7872066583723025
1000000 | uniform | 0 | 65536 | 8 | 128 | uncached | 0 | 8 | 2 | 248.8750000000000000 | 691.0720000000000000 | 2.7767835258663988
1000000 | uniform | 4 | 65536 | 8 | 128 | uncached | 0 | 8 | 2 | 251.4070000000000000 | 686.0470000000000000 | 2.7288301439498502
1000000 | cyclic | 0 | 4096 | 16 | 2 | uncached | 256 | 8 | 2 | 19.6590000000000000 | 52.9390000000000000 | 2.6928633195991658
1000000 | cyclic | 0 | 65536 | 16 | 2 | uncached | 256 | 8 | 2 | 18.9040000000000000 | 50.5270000000000000 | 2.6728205670757512
1000000 | uniform_pages | 0 | 4096 | 16 | 64 | uncached | 0 | 8 | 2 | 105.2800000000000000 | 278.6990000000000000 | 2.6472169452887538
1000000 | uniform | 0 | 4096 | 16 | 256 | uncached | 256 | 8 | 2 | 276.5640000000000000 | 728.0710000000000000 | 2.6325588290594582
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 2 | uncached | 256 | 8 | 2 | 24.9930000000000000 | 65.7020000000000000 | 2.6288160684991798
1000000 | uniform | 4 | 4096 | 8 | 128 | uncached | 0 | 8 | 2 | 249.0900000000000000 | 649.7000000000000000 | 2.6082941908547111
1000000 | cyclic_fuzz | 4 | 65536 | 32 | 2 | uncached | 0 | 8 | 2 | 18.7190000000000000 | 48.4340000000000000 | 2.5874245419092900
1000000 | uniform_pages | 0 | 4096 | 16 | 128 | uncached | 256 | 8 | 2 | 197.1760000000000000 | 505.2560000000000000 | 2.5624619629163793
1000000 | uniform_pages | 0 | 4096 | 16 | 211 | uncached | 256 | 8 | 2 | 318.4840000000000000 | 812.6740000000000000 | 2.5516949046105927
1000000 | uniform_pages | 0 | 65536 | 16 | 211 | uncached | 256 | 8 | 2 | 318.0390000000000000 | 811.2280000000000000 | 2.5507186225588686
1000000 | cyclic | 0 | 4096 | 16 | 4 | uncached | 0 | 128 | 2 | 34.5700000000000000 | 86.4800000000000000 | 2.5015909748336708
1000000 | cyclic | 4 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 17.6580000000000000 | 43.6340000000000000 | 2.4710612753426209
1000000 | uniform | 0 | 4096 | 32 | 512 | uncached | 256 | 8 | 2 | 293.6830000000000000 | 716.3800000000000000 | 2.4392967928004004
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 4 | uncached | 0 | 128 | 2 | 46.9810000000000000 | 112.9930000000000000 | 2.4050786488154786
1000000 | uniform | 0 | 4096 | 16 | 128 | uncached | 256 | 8 | 2 | 157.6140000000000000 | 377.7630000000000000 | 2.3967604400624310
1000000 | cyclic_fuzz | 4 | 4096 | 32 | 2 | uncached | 0 | 8 | 2 | 19.3970000000000000 | 46.3440000000000000 | 2.3892354487807393
1000000 | uniform | 0 | 65536 | 8 | 1024 | uncached | 0 | 128 | 2 | 548.1220000000000000 | 1307.4100000000000000 | 2.3852536479105017
1000000 | cyclic | 4 | 128 | 32 | 2 | uncached | 0 | 8 | 2 | 17.1170000000000000 | 40.8000000000000000 | 2.3835952561780686
1000000 | uniform | 4 | 65536 | 16 | 128 | uncached | 256 | 8 | 2 | 154.9780000000000000 | 366.8740000000000000 | 2.3672650311657138
1000000 | cyclic | 0 | 128 | 16 | 4 | uncached | 256 | 8 | 2 | 30.4870000000000000 | 70.9150000000000000 | 2.3260734083379801
1000000 | uniform | 4 | 65536 | 32 | 2048 | uncached | 0 | 128 | 2 | 264.8540000000000000 | 615.1720000000000000 | 2.3226834406880772
1000000 | cyclic_fuzz | 0 | 65536 | 8 | 4 | uncached | 0 | 128 | 2 | 50.5090000000000000 | 116.0020000000000000 | 2.2966600011879071
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 256 | 8 | 2 | 29.7680000000000000 | 67.9860000000000000 | 2.2838618650900296
1000000 | cyclic | 4 | 65536 | 32 | 2 | uncached | 0 | 8 | 2 | 18.4270000000000000 | 42.0330000000000000 | 2.2810549736799262
1000000 | uniform | 4 | 4096 | 32 | 2048 | uncached | 0 | 128 | 2 | 260.3540000000000000 | 592.9440000000000000 | 2.2774530062914340
1000000 | uniform | 4 | 4096 | 32 | 64 | uncached | 256 | 8 | 2 | 69.7780000000000000 | 158.7150000000000000 | 2.2745707816217146
1000000 | uniform | 0 | 4096 | 8 | 1024 | uncached | 0 | 128 | 2 | 579.3350000000000000 | 1299.1340000000000000 | 2.2424573001803792
1000000 | uniform_pages | 0 | 4096 | 32 | 64 | uncached | 256 | 8 | 2 | 85.3150000000000000 | 188.8470000000000000 | 2.2135263435503722
1000000 | uniform | 0 | 4096 | 32 | 32 | uncached | 0 | 8 | 2 | 38.1710000000000000 | 84.1300000000000000 | 2.2040292368552042
1000000 | cyclic_fuzz | 0 | 4096 | 8 | 2 | uncached | 256 | 8 | 2 | 29.6710000000000000 | 64.5820000000000000 | 2.1766034174783459
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 4 | uncached | 256 | 8 | 2 | 33.0140000000000000 | 71.8290000000000000 | 2.1757133337372024
1000000 | uniform | 0 | 4096 | 32 | 64 | uncached | 256 | 8 | 2 | 70.5350000000000000 | 152.9810000000000000 | 2.1688665201672928
1000000 | uniform | 4 | 65536 | 16 | 32 | uncached | 0 | 8 | 2 | 40.9560000000000000 | 86.4240000000000000 | 2.1101670084969235
1000000 | cyclic | 4 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 20.2450000000000000 | 42.5380000000000000 | 2.1011607804396147
1000000 | uniform | 0 | 4096 | 0 | 512 | uncached | 256 | 128 | 2 | 594.5600000000000000 | 1247.3030000000000000 | 2.0978589208826695
1000000 | cyclic_fuzz | 4 | 128 | 32 | 4 | uncached | 0 | 8 | 2 | 25.7620000000000000 | 54.0310000000000000 | 2.0973138731464948
1000000 | cyclic | 0 | 65536 | 8 | 4 | uncached | 0 | 128 | 2 | 40.6250000000000000 | 84.8690000000000000 | 2.0890830769230769
1000000 | cyclic_fuzz | 4 | 4096 | 16 | 2 | uncached | 0 | 8 | 2 | 22.1670000000000000 | 46.0590000000000000 | 2.0778183786709974
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 4 | uncached | 256 | 8 | 2 | 35.3820000000000000 | 73.3000000000000000 | 2.0716748629246510
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 256 | 8 | 2 | 35.2720000000000000 | 73.0640000000000000 | 2.0714447720571558
1000000 | cyclic_fuzz | 4 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 26.9920000000000000 | 55.7600000000000000 | 2.0657972732661529
1000000 | uniform_pages | 0 | 65536 | 32 | 32 | uncached | 0 | 8 | 2 | 51.6610000000000000 | 105.8500000000000000 | 2.0489343992566927
1000000 | cyclic_fuzz | 0 | 4096 | 1 | 4 | uncached | 0 | 8 | 2 | 118.4810000000000000 | 241.3620000000000000 | 2.0371367561043543
1000000 | uniform | 0 | 65536 | 16 | 512 | uncached | 256 | 8 | 2 | 371.4000000000000000 | 754.8360000000000000 | 2.0324071082390953
1000000 | cyclic_fuzz | 4 | 65536 | 16 | 2 | uncached | 0 | 8 | 2 | 22.3170000000000000 | 45.1480000000000000 | 2.0230317695030694
1000000 | cyclic | 0 | 4096 | 8 | 2 | uncached | 256 | 8 | 2 | 26.1960000000000000 | 52.8250000000000000 | 2.0165292411055123
1000000 | cyclic | 4 | 65536 | 32 | 4 | uncached | 0 | 8 | 2 | 23.1710000000000000 | 46.7180000000000000 | 2.0162271805273834
1000000 | cyclic | 4 | 128 | 32 | 4 | uncached | 0 | 8 | 2 | 23.1320000000000000 | 46.4200000000000000 | 2.0067439045478126
1000000 | uniform | 0 | 65536 | 32 | 32 | uncached | 0 | 8 | 2 | 37.3090000000000000 | 74.6180000000000000 | 2.0000000000000000
1000000 | cyclic_fuzz | 4 | 128 | 16 | 2 | uncached | 0 | 8 | 2 | 21.8060000000000000 | 43.5620000000000000 | 1.9977070531046501
1000000 | cyclic_fuzz | 0 | 4096 | 32 | 4 | uncached | 256 | 8 | 2 | 36.1660000000000000 | 71.9780000000000000 | 1.9902118011391915
1000000 | uniform | 4 | 4096 | 8 | 64 | uncached | 0 | 8 | 2 | 128.0980000000000000 | 253.8370000000000000 | 1.9815844119346126
1000000 | uniform | 0 | 4096 | 8 | 256 | uncached | 256 | 8 | 2 | 419.2600000000000000 | 823.3580000000000000 | 1.9638362829747651
1000000 | uniform | 4 | 65536 | 8 | 64 | uncached | 0 | 8 | 2 | 126.1750000000000000 | 247.4340000000000000 | 1.9610382405389340
1000000 | uniform | 4 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 42.8650000000000000 | 83.2560000000000000 | 1.9422839146156538
1000000 | uniform | 0 | 65536 | 8 | 256 | uncached | 256 | 8 | 2 | 423.4570000000000000 | 817.4440000000000000 | 1.9304061569415549
1000000 | uniform | 0 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 41.7190000000000000 | 80.1330000000000000 | 1.9207795009468108
1000000 | uniform | 0 | 65536 | 0 | 512 | uncached | 256 | 128 | 2 | 649.1030000000000000 | 1244.8640000000000000 | 1.9178219789463306
1000000 | uniform_pages | 0 | 4096 | 16 | 64 | uncached | 256 | 8 | 2 | 95.7390000000000000 | 183.5780000000000000 | 1.9174839929391366
1000000 | cyclic | 4 | 128 | 16 | 2 | uncached | 0 | 8 | 2 | 20.7090000000000000 | 39.5300000000000000 | 1.9088319088319088
1000000 | cyclic | 4 | 65536 | 16 | 2 | uncached | 0 | 8 | 2 | 19.6800000000000000 | 37.3750000000000000 | 1.8991361788617886
1000000 | uniform_pages | 0 | 65536 | 16 | 32 | uncached | 0 | 8 | 2 | 54.5230000000000000 | 102.9430000000000000 | 1.8880655869999817
1000000 | cyclic_fuzz | 0 | 4096 | 16 | 2 | uncached | 0 | 128 | 2 | 24.0170000000000000 | 44.3860000000000000 | 1.8481075904567598
1000000 | cyclic | 0 | 128 | 8 | 4 | uncached | 256 | 8 | 2 | 38.9400000000000000 | 71.9120000000000000 | 1.8467385721623010
1000000 | cyclic_fuzz | 4 | 65536 | 32 | 8 | uncached | 0 | 8 | 2 | 41.2820000000000000 | 76.0580000000000000 | 1.8424010464609273
1000000 | uniform | 0 | 65536 | 16 | 64 | uncached | 256 | 8 | 2 | 79.0730000000000000 | 145.6630000000000000 | 1.8421332186713543
1000000 | uniform | 4 | 65536 | 32 | 32 | uncached | 256 | 8 | 2 | 36.1090000000000000 | 66.3960000000000000 | 1.8387659586252735
1000000 | uniform | 4 | 4096 | 16 | 2048 | uncached | 0 | 128 | 2 | 332.7450000000000000 | 607.7430000000000000 | 1.8264526889960781
1000000 | cyclic | 4 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 38.8040000000000000 | 70.6930000000000000 | 1.8217967219874240
1000000 | uniform_pages | 0 | 65536 | 8 | 128 | uncached | 256 | 8 | 2 | 304.1250000000000000 | 534.8750000000000000 | 1.7587340731607069
1000000 | uniform_pages | 0 | 4096 | 8 | 128 | uncached | 256 | 8 | 2 | 302.8000000000000000 | 531.6400000000000000 | 1.7557463672391017
1000000 | uniform_pages | 0 | 4096 | 16 | 32 | uncached | 0 | 8 | 2 | 53.4580000000000000 | 93.3620000000000000 | 1.7464551610610199
1000000 | uniform_pages | 0 | 65536 | 0 | 211 | uncached | 256 | 128 | 2 | 823.1190000000000000 | 1431.1640000000000000 | 1.7387084977992247
1000000 | uniform | 4 | 65536 | 8 | 128 | uncached | 256 | 8 | 2 | 241.5110000000000000 | 408.9400000000000000 | 1.6932562077917776
1000000 | cyclic_fuzz | 4 | 4096 | 32 | 8 | uncached | 0 | 8 | 2 | 39.9330000000000000 | 67.5440000000000000 | 1.6914331505271329
1000000 | uniform_pages | 0 | 4096 | 0 | 211 | uncached | 256 | 128 | 2 | 835.8970000000000000 | 1407.5400000000000000 | 1.6838677492561883
1000000 | uniform | 4 | 4096 | 8 | 128 | uncached | 256 | 8 | 2 | 250.1520000000000000 | 420.7920000000000000 | 1.6821452556845438
1000000 | cyclic_fuzz | 0 | 4096 | 1 | 32 | uncached | 0 | 8 | 2 | 757.9950000000000000 | 1274.2050000000000000 | 1.6810203233530564
1000000 | cyclic | 0 | 128 | 16 | 4 | uncached | 256 | 128 | 2 | 28.6620000000000000 | 47.9570000000000000 | 1.6731909845788849
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 16 | uncached | 0 | 8 | 2 | 420.6100000000000000 | 695.4390000000000000 | 1.6534057678134139
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 256 | 8 | 2 | 51.8940000000000000 | 85.7260000000000000 | 1.6519443480941920
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 8 | uncached | 0 | 8 | 2 | 243.8420000000000000 | 402.3430000000000000 | 1.6500151737600577
1000000 | uniform | 4 | 4096 | 16 | 16 | uncached | 0 | 8 | 2 | 21.1470000000000000 | 34.8840000000000000 | 1.6495956873315364
1000000 | cyclic | 0 | 4096 | 32 | 4 | uncached | 256 | 128 | 2 | 28.4450000000000000 | 46.7740000000000000 | 1.6443663209702935
1000000 | uniform | 0 | 65536 | 8 | 32 | uncached | 0 | 8 | 2 | 58.8180000000000000 | 96.5620000000000000 | 1.6417083205821347
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 256 | 128 | 2 | 48.6230000000000000 | 79.5400000000000000 | 1.6358513460707895
1000000 | uniform | 4 | 4096 | 32 | 4 | cached | 256 | 128 | 2 | 0.76200000000000000000 | 1.24600000000000000000 | 1.63517060367454068241
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 2 | uncached | 0 | 8 | 2 | 104.5580000000000000 | 170.5210000000000000 | 1.6308747298150309
1000000 | uniform | 4 | 65536 | 32 | 16 | uncached | 0 | 8 | 2 | 20.9970000000000000 | 34.0890000000000000 | 1.6235176453779111
1000000 | cyclic_fuzz | 0 | 4096 | 1 | 16 | uncached | 0 | 8 | 2 | 428.8660000000000000 | 695.0810000000000000 | 1.6207416768874194
1000000 | cyclic_fuzz | 0 | 65536 | 16 | 8 | uncached | 256 | 8 | 2 | 56.5290000000000000 | 90.8090000000000000 | 1.6064144067646695
1000000 | cyclic_fuzz | 0 | 65536 | 1 | 4 | uncached | 0 | 8 | 2 | 151.9910000000000000 | 243.1920000000000000 | 1.6000421077563803
1000000 | uniform | 0 | 4096 | 8 | 32 | uncached | 0 | 8 | 2 | 60.7930000000000000 | 96.3030000000000000 | 1.5841133025183821
1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 256 | 8 | 2 | 55.6950000000000000 | 87.8090000000000000 | 1.5766047221474100
1000000 | uniform | 4 | 65536 | 1 | 1024 | uncached | 256 | 128 | 2 | 655.7590000000000000 | 1029.9260000000000000 | 1.5705861452149341
1000000 | cyclic_fuzz | 0 | 65536 | 32 | 4 | uncached | 256 | 128 | 2 | 32.6060000000000000 | 51.1150000000000000 | 1.5676562595841256
1000000 | cyclic | 0 | 4096 | 16 | 4 | uncached | 256 | 128 | 2 | 30.7160000000000000 | 48.0980000000000000 | 1.5658939966141425
1000000 | cyclic | 0 | 65536 | 32 | 8 | uncached | 256 | 8 | 2 | 55.8840000000000000 | 87.4530000000000000 | 1.5649022976164913
1000000 | cyclic | 0 | 4096 | 1 | 32 | uncached | 0 | 8 | 2 | 807.6740000000000000 | 1259.1210000000000000 | 1.5589470504188571
1000000 | uniform | 4 | 4096 | 1 | 1024 | uncached | 256 | 128 | 2 | 657.6370000000000000 | 1024.9000000000000000 | 1.5584585417183036
1000000 | cyclic | 0 | 65536 | 32 | 4 | uncached | 256 | 128 | 2 | 29.7980000000000000 | 46.4080000000000000 | 1.5574199610712128
1000000 | uniform | 4 | 4096 | 32 | 16 | uncached | 0 | 8 | 2 | 19.9800000000000000 | 31.0990000000000000 | 1.5565065065065065
1000000 | uniform | 4 | 4096 | 1 | 2048 | uncached | 256 | 128 | 2 | 599.4480000000000000 | 930.6370000000000000 | 1.5524899574275000
1000000 | cyclic | 0 | 65536 | 1 | 32 | uncached | 0 | 8 | 2 | 813.1960000000000000 | 1248.6470000000000000 | 1.5354809910525876
1000000 | cyclic | 0 | 4096 | 32 | 8 | uncached | 256 | 128 | 2 | 52.5890000000000000 | 80.6930000000000000 | 1.5344083363441024
1000000 | cyclic | 0 | 65536 | 1 | 16 | uncached | 0 | 8 | 2 | 438.0680000000000000 | 671.1000000000000000 | 1.5319539432234265
1000000 | uniform | 4 | 65536 | 1 | 2048 | uncached | 256 | 128 | 2 | 610.4480000000000000 | 933.1770000000000000 | 1.5286756611537756
1000000 | uniform | 0 | 65536 | 1 | 2048 | uncached | 0 | 8 | 2 | 2372.3930000000000000 | 3624.7540000000000000 | 1.5278893505418369
1000000 | cyclic | 0 | 4096 | 1 | 52 | uncached | 0 | 8 | 2 | 1314.9080000000000000 | 2004.4420000000000000 | 1.5243971441347988
1000000 | uniform | 0 | 65536 | 16 | 32 | uncached | 256 | 8 | 2 | 40.2340000000000000 | 61.3310000000000000 | 1.5243575085748372
1000000 | cyclic | 0 | 4096 | 16 | 8 | uncached | 256 | 128 | 2 | 48.6230000000000000 | 73.9910000000000000 | 1.5217284001398515
1000000 | uniform | 0 | 65536 | 16 | 8 | uncached | 0 | 8 | 2 | 12.8680000000000000 | 19.5690000000000000 | 1.5207491451663040
1000000 | cyclic | 0 | 4096 | 1 | 8 | uncached | 0 | 8 | 2 | 238.0610000000000000 | 361.4180000000000000 | 1.5181739134087482
1000000 | uniform | 4 | 65536 | 8 | 32 | uncached | 0 | 8 | 2 | 63.2790000000000000 | 96.0280000000000000 | 1.5175334629181877
1000000 | uniform_pages | 4 | 4096 | 0 | 4 | cached | 256 | 8 | 2 | 1.74500000000000000000 | 2.6380000000000000 | 1.51174785100286532951
1000000 | cyclic | 0 | 4096 | 1 | 16 | uncached | 0 | 8 | 2 | 450.1170000000000000 | 679.8840000000000000 | 1.5104606135738041
1000000 | uniform | 0 | 65536 | 1 | 4048 | uncached | 0 | 8 | 2 | 2384.6790000000000000 | 3601.4130000000000000 | 1.5102296787114744
1000000 | cyclic | 0 | 65536 | 1 | 8 | uncached | 0 | 8 | 2 | 232.1420000000000000 | 350.5240000000000000 | 1.5099551136804197
1000000 | cyclic | 0 | 4096 | 1 | 4 | uncached | 0 | 8 | 2 | 143.2900000000000000 | 215.2850000000000000 | 1.5024425989252565
1000000 | cyclic | 0 | 128 | 1 | 4 | uncached | 0 | 8 | 2 | 141.2470000000000000 | 211.7170000000000000 | 1.4989132512548939
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 11:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 13:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 15:52 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
@ 2024-03-29 21:39 ` Thomas Munro <[email protected]>
2024-03-29 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Munro @ 2024-03-29 21:39 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Sat, Mar 30, 2024 at 4:53 AM Tomas Vondra
<[email protected]> wrote:
> Two observations:
>
> * The combine limit seems to have negligible impact. There's no visible
> difference between combine_limit=8kB and 128kB.
>
> * Parallel queries seem to work about the same as master (especially for
> optimal cases, but even for not optimal ones).
>
>
> The optimal plans with kernel readahead (two charts in the first row)
> look fairly good. There are a couple regressed cases, but a bunch of
> faster ones too.
Thanks for doing this!
> The optimal plans without kernel read ahead (two charts in the second
> row) perform pretty poorly - there are massive regressions. But I think
> the obvious reason is that the streaming read API skips prefetches for
> sequential access patterns, relying on kernel to do the readahead. But
> if the kernel readahead is disabled for the device, that obviously can't
> happen ...
Right, it does seem that this whole concept is sensitive on the
'borderline' between sequential and random, and this patch changes
that a bit and we lose some. It's becoming much clearer to me that
master is already exposing weird kinks, and the streaming version is
mostly better, certainly on low IOPS systems. I suspect that there
must be queries in the wild that would run much faster with eic=0 than
eic=1 today due to that, and while the streaming version also loses in
some cases, it seems that it mostly loses because of not triggering
RA, which can at least be improved by increasing the RA window. On
the flip side, master is more prone to running out of IOPS and there
is no way to tune your way out of that.
> I think the question is how much we can (want to) rely on the readahead
> to be done by the kernel. ...
We already rely on it everywhere, for basic things like sequential scan.
> ... Maybe there should be some flag to force
> issuing fadvise even for sequential patterns, perhaps at the tablespace
> level? ...
Yeah, I've wondered about trying harder to "second guess" the Linux
RA. At the moment, read_stream.c detects *exactly* sequential reads
(see seq_blocknum) to suppress advice, but if we knew/guessed the RA
window size, we could (1) detect it with the same window that Linux
will use to detect it, and (2) [new realisation from yesterday's
testing] we could even "tickle" it to wake it up in certain cases
where it otherwise wouldn't, by temporarily using a smaller
io_combine_limit if certain patterns come along. I think that sounds
like madness (I suspect that any place where the latter would help is
a place where you could turn RA up a bit higher for the same effect
without weird kludges), or another way to put it would be to call it
"overfitting" to the pre-existing quirks; but maybe it's a future
research idea...
> I don't recall seeing a system with disabled readahead, but I'm
> sure there are cases where it may not really work - it clearly can't
> work with direct I/O, ...
Right, for direct I/O everything is slow right now including seq scan.
We need to start asynchronous reads in the background (imagine
literally just a bunch of background "I/O workers" running preadv() on
your behalf to get your future buffers ready for you, or equivalently
Linux io_uring). That's the real goal of this project: restructuring
so we have the information we need to do that, ie teach every part of
PostgreSQL to predict the future in a standard and centralised way.
Should work out better than RA heuristics, because we're not just
driving in a straight line, we can turn corners too.
> ... but I've also not been very successful with
> prefetching on ZFS.
posix_favise() did not do anything in OpenZFS before 2.2, maybe you
have an older version?
> I certainly admit the data sets are synthetic and perhaps adversarial.
> My intent was to cover a wide range of data sets, to trigger even less
> common cases. It's certainly up to debate how serious the regressions on
> those data sets are in practice, I'm not suggesting "this strange data
> set makes it slower than master, so we can't commit this".
Right, yeah. Thanks! Your initial results seemed discouraging, but
looking closer I'm starting to feel a lot more positive about
streaming BHS.
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 11:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 13:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 15:52 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
@ 2024-03-29 23:40 ` Tomas Vondra <[email protected]>
2024-03-29 23:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Tomas Vondra @ 2024-03-29 23:40 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On 3/29/24 22:39, Thomas Munro wrote:
> ...
>
>> I don't recall seeing a system with disabled readahead, but I'm
>> sure there are cases where it may not really work - it clearly can't
>> work with direct I/O, ...
>
> Right, for direct I/O everything is slow right now including seq scan.
> We need to start asynchronous reads in the background (imagine
> literally just a bunch of background "I/O workers" running preadv() on
> your behalf to get your future buffers ready for you, or equivalently
> Linux io_uring). That's the real goal of this project: restructuring
> so we have the information we need to do that, ie teach every part of
> PostgreSQL to predict the future in a standard and centralised way.
> Should work out better than RA heuristics, because we're not just
> driving in a straight line, we can turn corners too.
>
>> ... but I've also not been very successful with
>> prefetching on ZFS.
>
> posix_favise() did not do anything in OpenZFS before 2.2, maybe you
> have an older version?
>
Sorry, I meant the prefetch (readahead) built into ZFS. I may be wrong
but I don't think the regular RA (in linux kernel) works for ZFS, right?
I was wondering if we could use this (posix_fadvise) to improve that,
essentially by issuing fadvise even for sequential patterns. But now
that I think about that, if posix_fadvise works since 2.2, maybe RA
works too now?)
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 11:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 13:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 15:52 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
@ 2024-03-29 23:56 ` Thomas Munro <[email protected]>
0 siblings, 0 replies; 21+ messages in thread
From: Thomas Munro @ 2024-03-29 23:56 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Melanie Plageman <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Sat, Mar 30, 2024 at 12:40 PM Tomas Vondra
<[email protected]> wrote:
> Sorry, I meant the prefetch (readahead) built into ZFS. I may be wrong
> but I don't think the regular RA (in linux kernel) works for ZFS, right?
Right, it separate page cache ("ARC") and prefetch settings:
https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html
That's probably why Linux posix_fadvise didn't affect it, well that
and the fact, at a wild guess, that Solaris didn't have that system
call...
> I was wondering if we could use this (posix_fadvise) to improve that,
> essentially by issuing fadvise even for sequential patterns. But now
> that I think about that, if posix_fadvise works since 2.2, maybe RA
> works too now?)
It should work fine. I am planning to look into this a bit some day
soon -- I think there may be some interesting interactions between
systems with big pages/records like ZFS/BTRFS/... and io_combine_limit
that might offer interesting optimisation tweak opportunities, but
first things first...
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
@ 2024-03-31 15:45 ` Melanie Plageman <[email protected]>
2024-04-03 22:57 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
1 sibling, 1 reply; 21+ messages in thread
From: Melanie Plageman @ 2024-03-31 15:45 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Thomas Munro <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Fri, Mar 29, 2024 at 12:05:15PM +0100, Tomas Vondra wrote:
>
>
> On 3/29/24 02:12, Thomas Munro wrote:
> > On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
> > <[email protected]> wrote:
> >> I think there's some sort of bug, triggering this assert in heapam
> >>
> >> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
> >
> > Thanks for the repro. I can't seem to reproduce it (still trying) but
> > I assume this is with Melanie's v11 patch set which had
> > v11-0016-v10-Read-Stream-API.patch.
> >
> > Would you mind removing that commit and instead applying the v13
> > stream_read.c patches[1]? v10 stream_read.c was a little confused
> > about random I/O combining, which I fixed with a small adjustment to
> > the conditions for the "if" statement right at the end of
> > read_stream_look_ahead(). Sorry about that. The fixed version, with
> > eic=4, with your test query using WHERE a < a, ends its scan with:
> >
>
> I'll give that a try. Unfortunately unfortunately the v11 still has the
> problem I reported about a week ago:
>
> ERROR: prefetch and main iterators are out of sync
>
> So I can't run the full benchmarks :-( but master vs. streaming read API
> should work, I think.
Odd, I didn't notice you reporting this ERROR popping up. Now that I
take a look, v11 (at least, maybe also v10) had this very sill mistake:
if (scan->bm_parallel == NULL &&
scan->rs_pf_bhs_iterator &&
hscan->pfblockno > hscan->rs_base.blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
It errors out if the prefetch block is ahead of the current block --
which is the opposite of what we want. I've fixed this in attached v12.
This version also has v13 of the streaming read API. I noticed one
mistake in my bitmapheap scan streaming read user -- it freed the
streaming read object at the wrong time. I don't know if this was
causing any other issues, but it at least is fixed in this version.
- Melanie
Attachments:
[text/x-diff] v12-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240331154551.lty4mundyoyhtixw@liskov/2-v12-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From d406f48e2de3cf1627b2947ceae741113e9bf454 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v12 01/17] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 805d222cebc..b0f61c65f36 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aabe..93fdcd226bf 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v12-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240331154551.lty4mundyoyhtixw@liskov/3-v12-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 07b5447bc106b9aebeeac1e1143b57cff218bcf0 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v12 02/17] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226bf..c64530674bd 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v12-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (15.0K, ../../20240331154551.lty4mundyoyhtixw@liskov/4-v12-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From 415995a3fcd92e5055b30d0cf9213894f78deeb9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v12 03/17] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b661d9811eb..27e9d05f14b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,6 +948,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1036,6 +1038,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1055,6 +1063,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 41a4bb0981d..4c5fe8f090d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -28,6 +28,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2209,6 +2210,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2321,6 +2340,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674bd..83d9db8f39b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 32a3fbce961..49f212eb9a6 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -72,6 +72,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index cf76fc29d4b..e8d04b4dec6 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -63,6 +63,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -959,9 +966,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e7ff8e4992f..4880f346bf1 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,10 +1794,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1817,9 +1814,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v12-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.2K, ../../20240331154551.lty4mundyoyhtixw@liskov/5-v12-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From c7f5e03f51ea387a20a22b2ecb969d58eca3b5fc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v12 04/17] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39b..5df3b5ca46d 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v12-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.3K, ../../20240331154551.lty4mundyoyhtixw@liskov/6-v12-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 48d53f4a2204e35991e7ee8dad115252aace37d2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v12 05/17] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46d..404de0595ec 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v12-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch (4.4K, ../../20240331154551.lty4mundyoyhtixw@liskov/7-v12-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch)
download | inline diff:
From 7a3992b21b174a3a644a3eb82d982c77a64be7dd Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v12 06/17] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 4c5fe8f090d..973734e9ffa 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2199,7 +2199,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2327,6 +2328,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595ec..c95e3412da8 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e8d04b4dec6..19e99c81092 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -811,6 +811,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -826,7 +829,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2013,14 +2017,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2031,7 +2037,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v12-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (2.9K, ../../20240331154551.lty4mundyoyhtixw@liskov/8-v12-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From bf96c0a51ad2d88d2077418740d1e57696534625 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v12 07/17] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da8..49938c9ed41 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v12-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240331154551.lty4mundyoyhtixw@liskov/9-v12-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From 6bb0d8733eee92db27152ea40fab632759d34ba5 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v12 08/17] Remove table_scan_bitmap_next_tuple parameter
tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 973734e9ffa..2cb5fb18675 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2335,7 +2335,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed41..282dcb97919 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 19e99c81092..ff3e8bcdd6f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -802,10 +802,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -836,15 +833,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2050,7 +2042,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2062,7 +2053,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v12-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (23.0K, ../../20240331154551.lty4mundyoyhtixw@liskov/10-v12-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 521548fd0b97f1d7356e225984cfb12b2dc05bde Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v12 09/17] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 2cb5fb18675..916d7a6e2e8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2199,18 +2199,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2229,16 +2262,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2330,7 +2354,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb97919..90cb10bc819 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno < node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..92b829cebc7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index ff3e8bcdd6f..ad1805b55ed 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -22,6 +22,7 @@
#include "access/xact.h"
#include "commands/vacuum.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -795,19 +796,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -826,8 +822,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -964,9 +960,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -1013,6 +1013,21 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1023,6 +1038,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -2006,19 +2036,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2028,8 +2056,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 4880f346bf1..8e344155679 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1792,8 +1792,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1802,9 +1800,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1812,8 +1812,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1822,9 +1820,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v12-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.0K, ../../20240331154551.lty4mundyoyhtixw@liskov/11-v12-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From 551cdba973c43816ad40a17b23451d280c8db19e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v12 10/17] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext(). Now, on the first scan, the the
iterator is created after the scan descriptor is created.
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 163 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 29 +---
src/include/executor/nodeBitmapHeapscan.h | 10 ++
src/include/nodes/execnodes.h | 8 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 916d7a6e2e8..38c4fd5011c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2218,10 +2218,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 90cb10bc819..8b6f22bc3b6 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,6 +56,56 @@ static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ dsa_pointer shared_area,
+ dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(shared_area))
+ result->parallel = tbm_attach_shared_iterate(personal_area, shared_area);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -97,43 +147,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
+ if (init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -154,21 +184,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +216,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ dsa);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ node->pf_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
+ dsa);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -280,7 +311,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ node->pf_iterator &&
node->pfblockno < node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -321,12 +352,11 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
@@ -335,7 +365,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -348,8 +378,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -371,7 +399,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -431,23 +459,22 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (prefetch_iterator)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
node->prefetch_pages++;
@@ -475,8 +502,6 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
if (prefetch_iterator)
{
while (1)
@@ -500,12 +525,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
@@ -572,18 +597,17 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator)
+ {
+ bhs_end_iterate(node->pf_iterator);
+ node->pf_iterator = NULL;
+ }
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -628,12 +652,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (node->pf_iterator)
+ bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -671,11 +693,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
+ scanstate->pf_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc7..fb22f305bf6 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,12 +20,12 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
+struct BitmapHeapIterator;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -44,8 +44,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ struct BitmapHeapIterator *rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index ad1805b55ed..592a8421bdf 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -964,8 +964,7 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
+ result->rs_bhs_iterator = NULL;
return result;
}
@@ -1015,17 +1014,8 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1040,17 +1030,8 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caae..cb56d20dc6f 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,5 +28,15 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8e344155679..cf8b4995f0d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1787,6 +1787,8 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+struct BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1795,12 +1797,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1815,12 +1816,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a8d7bed411f..0fb2dad2053 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v12-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.2K, ../../20240331154551.lty4mundyoyhtixw@liskov/12-v12-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From 2ee3891dc6362dc484bbb4ea6778a268bd345469 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v12 11/17] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 38c4fd5011c..46f1a374252 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2199,7 +2199,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2349,7 +2350,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 8b6f22bc3b6..fb79f57d7a6 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -119,7 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -298,14 +297,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 592a8421bdf..a06ed271eb6 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -804,8 +804,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -822,8 +822,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -2019,15 +2021,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -2038,7 +2042,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v12-0012-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240331154551.lty4mundyoyhtixw@liskov/13-v12-0012-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From 393fa25bbff39b997f9f21d68ec9dd24c0add612 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v12 12/17] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fcc..1dc4c99bf99 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v12-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch (20.7K, ../../20240331154551.lty4mundyoyhtixw@liskov/14-v12-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch)
download | inline diff:
From 82b1fab2acccba1cdb9295f1e2b8897b12d33860 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v12 13/17] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 30 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++-------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/executor/nodeBitmapHeapscan.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
8 files changed, 113 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3aa457a29e1 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 46f1a374252..bd630e38fa8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2207,7 +2207,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2219,9 +2219,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(scan->rs_bhs_iterator);
+ bhs_iterate(scan->rs_bhs_iterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2236,11 +2236,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2248,19 +2248,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2289,7 +2289,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2298,9 +2298,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2350,7 +2350,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index fb79f57d7a6..d61965a2761 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -77,15 +77,16 @@ bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_ar
return result;
}
-TBMIterateResult *
-bhs_iterate(BitmapHeapIterator *iterator)
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result)
{
Assert(iterator);
+ Assert(result);
if (iterator->serial)
- return tbm_iterate(iterator->serial);
+ tbm_iterate(iterator->serial, result);
else
- return tbm_shared_iterate(iterator->parallel);
+ tbm_shared_iterate(iterator->parallel, result);
}
void
@@ -348,7 +349,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -360,8 +361,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -394,8 +395,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -462,10 +463,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -473,7 +476,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -482,13 +485,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -501,7 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -520,8 +523,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -529,17 +532,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf99..309a44bdb84 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae1..3b432263bb0 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index cb56d20dc6f..3c330f86e62 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -34,7 +34,7 @@ typedef struct BitmapHeapIterator
struct TBMSharedIterator *parallel;
} BitmapHeapIterator;
-extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result);
extern void bhs_end_iterate(BitmapHeapIterator *iterator);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v12-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (31.5K, ../../20240331154551.lty4mundyoyhtixw@liskov/15-v12-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From 1bc468541c8c3de890c5c4acc374f8cb33ac1764 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v12 14/17] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 262 +++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 341 ++--------------------
src/include/access/heapam.h | 17 ++
src/include/access/relscan.h | 8 +
src/include/access/tableam.h | 26 +-
src/include/nodes/execnodes.h | 14 -
7 files changed, 355 insertions(+), 339 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 27e9d05f14b..086dae59668 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,8 +948,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1032,6 +1040,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1044,6 +1058,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1069,6 +1089,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd630e38fa8..36c6fb021c3 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -61,6 +61,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2197,6 +2200,73 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2215,6 +2285,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2355,6 +2427,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->rs_pf_bhs_iterator &&
+ hscan->pfblockno < hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2365,6 +2449,154 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+
+ if (pstate == NULL)
+ {
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2390,6 +2622,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d61965a2761..187b288e688 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
dsa_pointer shared_area,
@@ -122,7 +118,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -142,7 +137,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
@@ -154,7 +149,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -169,23 +173,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
@@ -216,19 +213,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
- node->pf_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
- dsa);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
+ scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
+ scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
+ dsa);
}
#endif /* USE_PREFETCH */
@@ -243,36 +240,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -296,23 +263,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->pf_iterator &&
- node->pfblockno < node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,219 +289,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -594,22 +334,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator)
- {
- bhs_end_iterate(node->pf_iterator);
- node->pf_iterator = NULL;
- }
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -648,14 +378,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->pf_iterator)
- bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -688,17 +414,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->pf_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -738,13 +458,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -828,7 +541,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 49f212eb9a6..accbc1749cd 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -82,6 +82,23 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used in serial BHS
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fb22f305bf6..7938b741d66 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@
struct ParallelTableScanDescData;
struct BitmapHeapIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,6 +46,13 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
+ struct BitmapHeapIterator *rs_pf_bhs_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+ /* used to validate BHS prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a06ed271eb6..a30c0412901 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -807,17 +807,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -967,6 +956,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
+ result->rs_pf_bhs_iterator = NULL;
+ result->prefetch_maximum = 0;
+ result->bm_parallel = NULL;
return result;
}
@@ -1018,6 +1010,12 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1034,6 +1032,12 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cf8b4995f0d..592215d5ee7 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,18 +1794,11 @@ struct BitmapHeapIterator;
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1813,18 +1806,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v12-0015-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../20240331154551.lty4mundyoyhtixw@liskov/16-v12-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 441e94149ab349bc7008575e16f8cae5556f092a Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v12 15/17] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 086dae59668..8dc1eab5348 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -319,6 +319,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 36c6fb021c3..717bcd08175 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2194,12 +2194,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2233,8 +2227,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2597,30 +2591,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -3002,7 +3009,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index d9e23ef3175..1e89e3dfffa 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -92,8 +92,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 187b288e688..2f9387e51a2 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -230,44 +230,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
#endif /* USE_PREFETCH */
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346f..cf56cc572fe 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a30c0412901..8665e90fbc7 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -795,36 +795,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -2023,44 +2007,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2071,7 +2024,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v12-0016-v13-Streaming-Read-API.patch (70.5K, ../../20240331154551.lty4mundyoyhtixw@liskov/17-v12-0016-v13-Streaming-Read-API.patch)
download | inline diff:
From 5235e68513f32ffef2a896037296199d9d90039c Mon Sep 17 00:00:00 2001
From: Thomas Munro <[email protected]>
Date: Mon, 26 Feb 2024 23:48:31 +1300
Subject: [PATCH v12 16/17] v13 Streaming Read API
Part 1:
Provide vectored variant of ReadBuffer().
Break ReadBuffer() up into two steps: StartReadBuffers() and
WaitReadBuffers(). This has two advantages:
1. Multiple consecutive blocks can be read with one system call.
2. Advice (hints of future reads) can optionally be issued to the kernel.
The traditional ReadBuffer() function is now implemented in terms of
those functions, to avoid duplication. For now we still only read a
block at a time so there is no change to generated system calls yet, but
later commits will provide infrastructure to help build up larger calls.
Callers should respect the new GUC io_combine_limit, and the limit on
per-backend pins which is now exposed as a public interface.
With some more infrastructure in later work, StartReadBuffers() could
be extended to start real asynchronous I/O instead of advice.
Part 2:
Provide API for streaming relation data.
Introduce an abstraction where relation data can be accessed as a
stream of buffers, with an implementation that is more efficient than
the equivalent sequence of ReadBuffer() calls.
Client code supplies a callback that can say which block number is
wanted next, and then consumes individual buffers one at a time from the
stream. This division allows read_stream.c to build up large calls to
StartReadBuffers() up to io_combine_limit, and issue posix_fadvise()
advice ahead of time in a systematic way when random access is detected.
This API is based on an idea from Andres Freund to pave the way for
asynchronous I/O in future work as required to support direct I/O. The
goal is to have an abstraction that insulates client code from future
changes to the I/O subsystem that would benefit from information about
future needs.
An extended API may be necessary in future for more complicated cases
(for example recovery, whose LsnReadQueue device in xlogprefetcher.c is
a distant cousin of this code and should eventually be replaced by
this), but this basic API is sufficient for many common usage patterns
involving predictable access to a single relation fork.
---
doc/src/sgml/config.sgml | 14 +
src/backend/storage/Makefile | 2 +-
src/backend/storage/aio/Makefile | 14 +
src/backend/storage/aio/meson.build | 5 +
src/backend/storage/aio/read_stream.c | 750 ++++++++++++++++++
src/backend/storage/buffer/bufmgr.c | 701 +++++++++++-----
src/backend/storage/buffer/localbuf.c | 14 +-
src/backend/storage/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 14 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/storage/bufmgr.h | 41 +-
src/include/storage/read_stream.h | 63 ++
src/tools/pgindent/typedefs.list | 3 +
13 files changed, 1400 insertions(+), 223 deletions(-)
create mode 100644 src/backend/storage/aio/Makefile
create mode 100644 src/backend/storage/aio/meson.build
create mode 100644 src/backend/storage/aio/read_stream.c
create mode 100644 src/include/storage/read_stream.h
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index f65c17e5ae4..241a6079688 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2720,6 +2720,20 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-io-combine-limit" xreflabel="io_combine_limit">
+ <term><varname>io_combine_limit</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>io_combine_limit</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Controls the largest I/O size in operations that combine I/O.
+ The default is 128kB.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-max-worker-processes" xreflabel="max_worker_processes">
<term><varname>max_worker_processes</varname> (<type>integer</type>)
<indexterm>
diff --git a/src/backend/storage/Makefile b/src/backend/storage/Makefile
index 8376cdfca20..eec03f6f2b4 100644
--- a/src/backend/storage/Makefile
+++ b/src/backend/storage/Makefile
@@ -8,6 +8,6 @@ subdir = src/backend/storage
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = buffer file freespace ipc large_object lmgr page smgr sync
+SUBDIRS = aio buffer file freespace ipc large_object lmgr page smgr sync
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/Makefile b/src/backend/storage/aio/Makefile
new file mode 100644
index 00000000000..2f29a9ec4d1
--- /dev/null
+++ b/src/backend/storage/aio/Makefile
@@ -0,0 +1,14 @@
+#
+# Makefile for storage/aio
+#
+# src/backend/storage/aio/Makefile
+#
+
+subdir = src/backend/storage/aio
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = \
+ read_stream.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/aio/meson.build b/src/backend/storage/aio/meson.build
new file mode 100644
index 00000000000..10e1aa3b20b
--- /dev/null
+++ b/src/backend/storage/aio/meson.build
@@ -0,0 +1,5 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+backend_sources += files(
+ 'read_stream.c',
+)
diff --git a/src/backend/storage/aio/read_stream.c b/src/backend/storage/aio/read_stream.c
new file mode 100644
index 00000000000..6b4b97372c7
--- /dev/null
+++ b/src/backend/storage/aio/read_stream.c
@@ -0,0 +1,750 @@
+/*-------------------------------------------------------------------------
+ *
+ * read_stream.c
+ * Mechanism for accessing buffered relation data with look-ahead
+ *
+ * Code that needs to access relation data typically pins blocks one at a
+ * time, often in a predictable order that might be sequential or data-driven.
+ * Calling the simple ReadBuffer() function for each block is inefficient,
+ * because blocks that are not yet in the buffer pool require I/O operations
+ * that are small and might stall waiting for storage. This mechanism looks
+ * into the future and calls StartReadBuffers() and WaitReadBuffers() to read
+ * neighboring blocks together and ahead of time, with an adaptive look-ahead
+ * distance.
+ *
+ * A user-provided callback generates a stream of block numbers that is used
+ * to form reads of up to io_combine_limit, by attempting to merge them with a
+ * pending read. When that isn't possible, the existing pending read is sent
+ * to StartReadBuffers() so that a new one can begin to form.
+ *
+ * The algorithm for controlling the look-ahead distance tries to classify the
+ * stream into three ideal behaviors:
+ *
+ * A) No I/O is necessary, because the requested blocks are fully cached
+ * already. There is no benefit to looking ahead more than one block, so
+ * distance is 1. This is the default initial assumption.
+ *
+ * B) I/O is necessary, but fadvise is undesirable because the access is
+ * sequential, or impossible because direct I/O is enabled or the system
+ * doesn't support advice. There is no benefit in looking ahead more than
+ * io_combine_limit, because in this case only goal is larger read system
+ * calls. Looking further ahead would pin many buffers and perform
+ * speculative work looking ahead for no benefit.
+ *
+ * C) I/O is necesssary, it appears random, and this system supports fadvise.
+ * We'll look further ahead in order to reach the configured level of I/O
+ * concurrency.
+ *
+ * The distance increases rapidly and decays slowly, so that it moves towards
+ * those levels as different I/O patterns are discovered. For example, a
+ * sequential scan of fully cached data doesn't bother looking ahead, but a
+ * sequential scan that hits a region of uncached blocks will start issuing
+ * increasingly wide read calls until it plateaus at io_combine_limit.
+ *
+ * The main data structure is a circular queue of buffers of size
+ * max_pinned_buffers plus some extra space for technical reasons, ready to be
+ * returned by read_stream_next_buffer(). Each buffer also has an optional
+ * variable sized object that is passed from the callback to the consumer of
+ * buffers.
+ *
+ * Parallel to the queue of buffers, there is a circular queue of in-progress
+ * I/Os that have been started with StartReadBuffers(), and for which
+ * WaitReadBuffers() must be called before returning the buffer.
+ *
+ * For example, if the callback return block numbers 10, 42, 43, 60 in
+ * successive calls, then these data structures might appear as follows:
+ *
+ * buffers buf/data ios
+ *
+ * +----+ +-----+ +--------+
+ * | | | | +----+ 42..44 | <- oldest_io_index
+ * +----+ +-----+ | +--------+
+ * oldest_buffer_index -> | 10 | | ? | | +--+ 60..60 |
+ * +----+ +-----+ | | +--------+
+ * | 42 | | ? |<-+ | | | <- next_io_index
+ * +----+ +-----+ | +--------+
+ * | 43 | | ? | | | |
+ * +----+ +-----+ | +--------+
+ * | 44 | | ? | | | |
+ * +----+ +-----+ | +--------+
+ * | 60 | | ? |<---+
+ * +----+ +-----+
+ * next_buffer_index -> | | | |
+ * +----+ +-----+
+ *
+ * In the example, 5 buffers are pinned, and the next buffer to be streamed to
+ * the client is block 10. Block 10 was a hit and has no associated I/O, but
+ * the range 42..44 requires an I/O wait before its buffers are returned, as
+ * does block 60.
+ *
+ *
+ * Portions Copyright (c) 2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/storage/aio/read_stream.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "catalog/pg_tablespace.h"
+#include "miscadmin.h"
+#include "storage/fd.h"
+#include "storage/smgr.h"
+#include "storage/read_stream.h"
+#include "utils/memdebug.h"
+#include "utils/rel.h"
+#include "utils/spccache.h"
+
+typedef struct InProgressIO
+{
+ int16 buffer_index;
+ ReadBuffersOperation op;
+} InProgressIO;
+
+/*
+ * State for managing a stream of reads.
+ */
+struct ReadStream
+{
+ int16 max_ios;
+ int16 ios_in_progress;
+ int16 queue_size;
+ int16 max_pinned_buffers;
+ int16 pinned_buffers;
+ int16 distance;
+ bool advice_enabled;
+
+ /*
+ * Sometimes we need to be able to 'unget' a block number to resolve a
+ * flow control problem when I/Os are split.
+ */
+ BlockNumber unget_blocknum;
+ bool have_unget_blocknum;
+
+ /*
+ * The callback that will tell us which block numbers to read, and an
+ * opaque pointer that will be pass to it for its own purposes.
+ */
+ ReadStreamBlockNumberCB callback;
+ void *callback_private_data;
+
+ /* Next expected block, for detecting sequential access. */
+ BlockNumber seq_blocknum;
+
+ /* The read operation we are currently preparing. */
+ BlockNumber pending_read_blocknum;
+ int16 pending_read_nblocks;
+
+ /* Space for buffers and optional per-buffer private data. */
+ size_t per_buffer_data_size;
+ void *per_buffer_data;
+
+ /* Read operations that have been started but not waited for yet. */
+ InProgressIO *ios;
+ int16 oldest_io_index;
+ int16 next_io_index;
+
+ /* Circular queue of buffers. */
+ int16 oldest_buffer_index; /* Next pinned buffer to return */
+ int16 next_buffer_index; /* Index of next buffer to pin */
+ Buffer buffers[FLEXIBLE_ARRAY_MEMBER];
+};
+
+/*
+ * Return a pointer to the per-buffer data by index.
+ */
+static inline void *
+get_per_buffer_data(ReadStream *stream, int16 buffer_index)
+{
+ return (char *) stream->per_buffer_data +
+ stream->per_buffer_data_size * buffer_index;
+}
+
+/*
+ * Ask the callback which block it would like us to read next, with a small
+ * buffer in front to allow read_stream_unget_block() to work.
+ */
+static inline BlockNumber
+read_stream_get_block(ReadStream *stream, void *per_buffer_data)
+{
+ if (!stream->have_unget_blocknum)
+ return stream->callback(stream,
+ stream->callback_private_data,
+ per_buffer_data);
+
+ /*
+ * You can only unget one block, and next_buffer_index can't change across
+ * a get, unget, get sequence, so the callback's per_buffer_data, if any,
+ * is still present in the correct slot. We just have to return the
+ * previous block number.
+ */
+ stream->have_unget_blocknum = false;
+ return stream->unget_blocknum;
+}
+
+/*
+ * In order to deal with short reads in StartReadBuffers(), we sometimes need
+ * to defer handling of a block until later.
+ */
+static inline void
+read_stream_unget_block(ReadStream *stream, BlockNumber blocknum)
+{
+ Assert(!stream->have_unget_blocknum);
+ stream->have_unget_blocknum = true;
+ stream->unget_blocknum = blocknum;
+}
+
+static void
+read_stream_start_pending_read(ReadStream *stream, bool suppress_advice)
+{
+ bool need_wait;
+ int nblocks;
+ int flags;
+ int16 io_index;
+ int16 overflow;
+ int16 buffer_index;
+
+ /* This should only be called with a pending read. */
+ Assert(stream->pending_read_nblocks > 0);
+ Assert(stream->pending_read_nblocks <= io_combine_limit);
+
+ /* We had better not exceed the pin limit by starting this read. */
+ Assert(stream->pinned_buffers + stream->pending_read_nblocks <=
+ stream->max_pinned_buffers);
+
+ /* We had better not be overwriting an existing pinned buffer. */
+ if (stream->pinned_buffers > 0)
+ Assert(stream->next_buffer_index != stream->oldest_buffer_index);
+ else
+ Assert(stream->next_buffer_index == stream->oldest_buffer_index);
+
+ /*
+ * If advice hasn't been suppressed, this system supports it, and this
+ * isn't a strictly sequential pattern, then we'll issue advice.
+ */
+ if (!suppress_advice &&
+ stream->advice_enabled &&
+ stream->pending_read_blocknum != stream->seq_blocknum)
+ flags = READ_BUFFERS_ISSUE_ADVICE;
+ else
+ flags = 0;
+
+ /* We say how many blocks we want to read, but may be smaller on return. */
+ buffer_index = stream->next_buffer_index;
+ io_index = stream->next_io_index;
+ nblocks = stream->pending_read_nblocks;
+ need_wait = StartReadBuffers(&stream->ios[io_index].op,
+ &stream->buffers[buffer_index],
+ stream->pending_read_blocknum,
+ &nblocks,
+ flags);
+ stream->pinned_buffers += nblocks;
+
+ /* Remember whether we need to wait before returning this buffer. */
+ if (!need_wait)
+ {
+ /* Look-ahead distance decays, no I/O necessary (behavior A). */
+ if (stream->distance > 1)
+ stream->distance--;
+ }
+ else
+ {
+ /*
+ * Remember to call WaitReadBuffers() before returning head buffer.
+ * Look-ahead distance will be adjusted after waiting.
+ */
+ stream->ios[io_index].buffer_index = buffer_index;
+ if (++stream->next_io_index == stream->max_ios)
+ stream->next_io_index = 0;
+ Assert(stream->ios_in_progress < stream->max_ios);
+ stream->ios_in_progress++;
+ stream->seq_blocknum = stream->pending_read_blocknum + nblocks;
+ }
+
+ /*
+ * We gave a contiguous range of buffer space to StartReadBuffers(), but
+ * we want it to wrap around at queue_size. Slide overflowing buffers to
+ * the front of the array.
+ */
+ overflow = (buffer_index + nblocks) - stream->queue_size;
+ if (overflow > 0)
+ memmove(&stream->buffers[0],
+ &stream->buffers[stream->queue_size],
+ sizeof(stream->buffers[0]) * overflow);
+
+ /* Compute location of start of next read, without using % operator. */
+ buffer_index += nblocks;
+ if (buffer_index >= stream->queue_size)
+ buffer_index -= stream->queue_size;
+ Assert(buffer_index >= 0 && buffer_index < stream->queue_size);
+ stream->next_buffer_index = buffer_index;
+
+ /* Adjust the pending read to cover the remaining portion, if any. */
+ stream->pending_read_blocknum += nblocks;
+ stream->pending_read_nblocks -= nblocks;
+}
+
+static void
+read_stream_look_ahead(ReadStream *stream, bool suppress_advice)
+{
+ while (stream->ios_in_progress < stream->max_ios &&
+ stream->pinned_buffers + stream->pending_read_nblocks < stream->distance)
+ {
+ BlockNumber blocknum;
+ int16 buffer_index;
+ void *per_buffer_data;
+
+ if (stream->pending_read_nblocks == io_combine_limit)
+ {
+ read_stream_start_pending_read(stream, suppress_advice);
+ suppress_advice = false;
+ continue;
+ }
+
+ /*
+ * See which block the callback wants next in the stream. We need to
+ * compute the index of the Nth block of the pending read including
+ * wrap-around, but we don't want to use the expensive % operator.
+ */
+ buffer_index = stream->next_buffer_index + stream->pending_read_nblocks;
+ if (buffer_index >= stream->queue_size)
+ buffer_index -= stream->queue_size;
+ Assert(buffer_index >= 0 && buffer_index < stream->queue_size);
+ per_buffer_data = get_per_buffer_data(stream, buffer_index);
+ blocknum = read_stream_get_block(stream, per_buffer_data);
+ if (blocknum == InvalidBlockNumber)
+ {
+ /* End of stream. */
+ stream->distance = 0;
+ break;
+ }
+
+ /* Can we merge it with the pending read? */
+ if (stream->pending_read_nblocks > 0 &&
+ stream->pending_read_blocknum + stream->pending_read_nblocks == blocknum)
+ {
+ stream->pending_read_nblocks++;
+ continue;
+ }
+
+ /* We have to start the pending read before we can build another. */
+ if (stream->pending_read_nblocks > 0)
+ {
+ read_stream_start_pending_read(stream, suppress_advice);
+ suppress_advice = false;
+ if (stream->ios_in_progress == stream->max_ios)
+ {
+ /* And we've hit the limit. Rewind, and stop here. */
+ read_stream_unget_block(stream, blocknum);
+ return;
+ }
+ }
+
+ /* This is the start of a new pending read. */
+ stream->pending_read_blocknum = blocknum;
+ stream->pending_read_nblocks = 1;
+ }
+
+ /*
+ * We don't start the pending read just because we've hit the distance
+ * limit, preferring to give it another chance to grow to full
+ * io_combine_limit size once more buffers have been consumed. However,
+ * if we've already reached io_combine_limit, or we've reached the
+ * distance limit and there isn't anything pinned yet, or the callback has
+ * signaled end-of-stream, we start the read immediately.
+ */
+ if (stream->pending_read_nblocks > 0 &&
+ (stream->pending_read_nblocks == io_combine_limit ||
+ (stream->pending_read_nblocks == stream->distance &&
+ stream->pinned_buffers == 0) ||
+ stream->distance == 0) &&
+ stream->ios_in_progress < stream->max_ios)
+ read_stream_start_pending_read(stream, suppress_advice);
+}
+
+/*
+ * Create a new read stream object that can be used to perform the equivalent
+ * of a series of ReadBuffer() calls for one fork of one relation.
+ * Internally, it generates larger vectored reads where possible by looking
+ * ahead. The callback should return block numbers or InvalidBlockNumber to
+ * signal end-of-stream, and if per_buffer_data_size is non-zero, it may also
+ * write extra data for each block into the space provided to it. It will
+ * also receive callback_private_data for its own purposes.
+ */
+ReadStream *
+read_stream_begin_relation(int flags,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ ReadStreamBlockNumberCB callback,
+ void *callback_private_data,
+ size_t per_buffer_data_size)
+{
+ ReadStream *stream;
+ size_t size;
+ int16 queue_size;
+ int16 max_ios;
+ uint32 max_pinned_buffers;
+ Oid tablespace_id;
+
+ /* Make sure our bmr's smgr and persistent are populated. */
+ if (bmr.smgr == NULL)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ /*
+ * Decide how many I/Os we will allow to run at the same time. That
+ * currently means advice to the kernel to tell it that we will soon read.
+ * This number also affects how far we look ahead for opportunities to
+ * start more I/Os.
+ */
+ tablespace_id = bmr.smgr->smgr_rlocator.locator.spcOid;
+ if (!OidIsValid(MyDatabaseId) ||
+ (bmr.rel && IsCatalogRelation(bmr.rel)) ||
+ IsCatalogRelationOid(bmr.smgr->smgr_rlocator.locator.relNumber))
+ {
+ /*
+ * Avoid circularity while trying to look up tablespace settings or
+ * before spccache.c is ready.
+ */
+ max_ios = effective_io_concurrency;
+ }
+ else if (flags & READ_STREAM_MAINTENANCE)
+ max_ios = get_tablespace_maintenance_io_concurrency(tablespace_id);
+ else
+ max_ios = get_tablespace_io_concurrency(tablespace_id);
+ max_ios = Min(max_ios, PG_INT16_MAX);
+
+ /*
+ * Choose the maximum number of buffers we're prepared to pin. We try to
+ * pin fewer if we can, though. We clamp it to at least io_combine_limit
+ * so that we can have a chance to build up a full io_combine_limit sized
+ * read, even when max_ios is zero. Be careful not to allow int16 to
+ * overflow (even though that's not possible with the current GUC range
+ * limits), allowing also for the spare entry and the overflow space.
+ */
+ max_pinned_buffers = Max(max_ios * 4, io_combine_limit);
+ max_pinned_buffers = Min(max_pinned_buffers,
+ PG_INT16_MAX - io_combine_limit - 1);
+
+ /* Don't allow this backend to pin more than its share of buffers. */
+ if (SmgrIsTemp(bmr.smgr))
+ LimitAdditionalLocalPins(&max_pinned_buffers);
+ else
+ LimitAdditionalPins(&max_pinned_buffers);
+ Assert(max_pinned_buffers > 0);
+
+ /*
+ * We need one extra entry for buffers and per-buffer data, because users
+ * of per-buffer data have access to the object until the next call to
+ * read_stream_next_buffer(), so we need a gap between the head and tail
+ * of the queue so that we don't clobber it.
+ */
+ queue_size = max_pinned_buffers + 1;
+
+ /*
+ * Allocate the object, the buffers, the ios and per_data_data space in
+ * one big chunk. Though we have queue_size buffers, we want to be able
+ * to assume that all the buffers for a single read are contiguous (i.e.
+ * don't wrap around halfway through), so we allow temporary overflows of
+ * up to the maximum possible read size by allocating an extra
+ * io_combine_limit - 1 elements.
+ */
+ size = offsetof(ReadStream, buffers);
+ size += sizeof(Buffer) * (queue_size + io_combine_limit - 1);
+ size += sizeof(InProgressIO) * Max(1, max_ios);
+ size += per_buffer_data_size * queue_size;
+ size += MAXIMUM_ALIGNOF * 2;
+ stream = (ReadStream *) palloc(size);
+ memset(stream, 0, offsetof(ReadStream, buffers));
+ stream->ios = (InProgressIO *)
+ MAXALIGN(&stream->buffers[queue_size + io_combine_limit - 1]);
+ if (per_buffer_data_size > 0)
+ stream->per_buffer_data = (void *)
+ MAXALIGN(&stream->ios[Max(1, max_ios)]);
+
+#ifdef USE_PREFETCH
+
+ /*
+ * This system supports prefetching advice. We can use it as long as
+ * direct I/O isn't enabled, the caller hasn't promised sequential access
+ * (overriding our detection heuristics), and max_ios hasn't been set to
+ * zero.
+ */
+ if ((io_direct_flags & IO_DIRECT_DATA) == 0 &&
+ (flags & READ_STREAM_SEQUENTIAL) == 0 &&
+ max_ios > 0)
+ stream->advice_enabled = true;
+#endif
+
+ /*
+ * For now, max_ios = 0 is interpreted as max_ios = 1 with advice disabled
+ * above. If we had real asynchronous I/O we might need a slightly
+ * different definition.
+ */
+ if (max_ios == 0)
+ max_ios = 1;
+
+ stream->max_ios = max_ios;
+ stream->per_buffer_data_size = per_buffer_data_size;
+ stream->max_pinned_buffers = max_pinned_buffers;
+ stream->queue_size = queue_size;
+
+ if (!bmr.smgr)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+ stream->callback = callback;
+ stream->callback_private_data = callback_private_data;
+
+ /*
+ * Skip the initial ramp-up phase if the caller says we're going to be
+ * reading the whole relation. This way we start out assuming we'll be
+ * doing full io_combine_limit sized reads (behavior B).
+ */
+ if (flags & READ_STREAM_FULL)
+ stream->distance = Min(max_pinned_buffers, io_combine_limit);
+ else
+ stream->distance = 1;
+
+ /*
+ * Since we always currently always access the same relation, we can
+ * initialize parts of the ReadBuffersOperation objects and leave them
+ * that way, to avoid wasting CPU cycles writing to them for each read.
+ */
+ for (int i = 0; i < max_ios; ++i)
+ {
+ stream->ios[i].op.bmr = bmr;
+ stream->ios[i].op.forknum = forknum;
+ stream->ios[i].op.strategy = strategy;
+ }
+
+ return stream;
+}
+
+/*
+ * Pull one pinned buffer out of a stream. Each call returns successive
+ * blocks in the order specified by the callback. If per_buffer_data_size was
+ * set to a non-zero size, *per_buffer_data receives a pointer to the extra
+ * per-buffer data that the callback had a chance to populate, which remains
+ * valid until the next call to read_stream_next_buffer(). When the stream
+ * runs out of data, InvalidBuffer is returned. The caller may decide to end
+ * the stream early at any time by calling read_stream_end().
+ */
+Buffer
+read_stream_next_buffer(ReadStream *stream, void **per_buffer_data)
+{
+ Buffer buffer;
+ int16 oldest_buffer_index;
+
+ /*
+ * A fast path for all-cached scans (behavior A). This is the same as the
+ * usual algorithm, but it is specialized for no I/O and no per-buffer
+ * data, so we can skip the queue management code, stay in the same buffer
+ * slot and use singular StartReadBuffer().
+ */
+ if (likely(per_buffer_data == NULL &&
+ stream->ios_in_progress == 0 &&
+ stream->pinned_buffers == 1 &&
+ stream->distance == 1))
+ {
+ BlockNumber next_blocknum;
+
+ /*
+ * We have a pinned buffer that we need to serve up, but we also want
+ * to probe the next one before we return, just in case we need to
+ * start an I/O. We can re-use the same buffer slot, and an arbitrary
+ * I/O slot since they're all free.
+ */
+ oldest_buffer_index = stream->oldest_buffer_index;
+ Assert((oldest_buffer_index + 1) % stream->queue_size ==
+ stream->next_buffer_index);
+ buffer = stream->buffers[oldest_buffer_index];
+ Assert(buffer != InvalidBuffer);
+ Assert(stream->pending_read_nblocks <= 1);
+ if (unlikely(stream->pending_read_nblocks == 1))
+ {
+ next_blocknum = stream->pending_read_blocknum;
+ stream->pending_read_nblocks = 0;
+ }
+ else
+ next_blocknum = read_stream_get_block(stream, NULL);
+ if (unlikely(next_blocknum == InvalidBlockNumber))
+ {
+ /* End of stream. */
+ stream->distance = 0;
+ stream->next_buffer_index = oldest_buffer_index;
+ /* Pin transferred to caller. */
+ stream->pinned_buffers = 0;
+ return buffer;
+ }
+ /* Call the special single block version, which is marginally faster. */
+ if (unlikely(StartReadBuffer(&stream->ios[0].op,
+ &stream->buffers[oldest_buffer_index],
+ next_blocknum,
+ stream->advice_enabled ?
+ READ_BUFFERS_ISSUE_ADVICE : 0)))
+ {
+ /* I/O needed. We'll take the general path next time. */
+ stream->oldest_io_index = 0;
+ stream->next_io_index = stream->max_ios > 1 ? 1 : 0;
+ stream->ios_in_progress = 1;
+ stream->ios[0].buffer_index = oldest_buffer_index;
+ stream->seq_blocknum = next_blocknum + 1;
+ /* Increase look ahead distance (move towards behavior B/C). */
+ stream->distance = Min(2, stream->max_pinned_buffers);
+ }
+ /* Pin transferred to caller, got another one, no net change. */
+ Assert(stream->pinned_buffers == 1);
+ return buffer;
+ }
+
+ if (stream->pinned_buffers == 0)
+ {
+ Assert(stream->oldest_buffer_index == stream->next_buffer_index);
+
+ /* End of stream reached? */
+ if (stream->distance == 0)
+ return InvalidBuffer;
+
+ /*
+ * The usual order of operations is that we look ahead at the bottom
+ * of this function after potentially finishing an I/O and making
+ * space for more, but if we're just starting up we'll need to crank
+ * the handle to get started.
+ */
+ read_stream_look_ahead(stream, true);
+
+ /* End of stream reached? */
+ if (stream->pinned_buffers == 0)
+ {
+ Assert(stream->distance == 0);
+ return InvalidBuffer;
+ }
+ }
+
+ /* Grab the oldest pinned buffer and associated per-buffer data. */
+ Assert(stream->pinned_buffers > 0);
+ oldest_buffer_index = stream->oldest_buffer_index;
+ Assert(oldest_buffer_index >= 0 &&
+ oldest_buffer_index < stream->queue_size);
+ buffer = stream->buffers[oldest_buffer_index];
+ if (per_buffer_data)
+ *per_buffer_data = get_per_buffer_data(stream, oldest_buffer_index);
+
+ Assert(BufferIsValid(buffer));
+
+ /* Do we have to wait for an associated I/O first? */
+ if (stream->ios_in_progress > 0 &&
+ stream->ios[stream->oldest_io_index].buffer_index == oldest_buffer_index)
+ {
+ int16 io_index = stream->oldest_io_index;
+ int16 distance;
+
+ /* Sanity check that we still agree on the buffers. */
+ Assert(stream->ios[io_index].op.buffers ==
+ &stream->buffers[oldest_buffer_index]);
+
+ WaitReadBuffers(&stream->ios[io_index].op);
+
+ Assert(stream->ios_in_progress > 0);
+ stream->ios_in_progress--;
+ if (++stream->oldest_io_index == stream->max_ios)
+ stream->oldest_io_index = 0;
+
+ if (stream->ios[io_index].op.flags & READ_BUFFERS_ISSUE_ADVICE)
+ {
+ /* Distance ramps up fast (behavior C). */
+ distance = stream->distance * 2;
+ distance = Min(distance, stream->max_pinned_buffers);
+ stream->distance = distance;
+ }
+ else
+ {
+ /* No advice; move towards io_combine_limit (behavior B). */
+ if (stream->distance > io_combine_limit)
+ {
+ stream->distance--;
+ }
+ else
+ {
+ distance = stream->distance * 2;
+ distance = Min(distance, io_combine_limit);
+ distance = Min(distance, stream->max_pinned_buffers);
+ stream->distance = distance;
+ }
+ }
+ }
+
+#ifdef CLOBBER_FREED_MEMORY
+ /* Clobber old buffer and per-buffer data for debugging purposes. */
+ stream->buffers[oldest_buffer_index] = InvalidBuffer;
+
+ /*
+ * The caller will get access to the per-buffer data, until the next call.
+ * We wipe the one before, which is never occupied because queue_size
+ * allowed one extra element. This will hopefully trip up client code
+ * that is holding a dangling pointer to it.
+ */
+ if (stream->per_buffer_data)
+ wipe_mem(get_per_buffer_data(stream,
+ oldest_buffer_index == 0 ?
+ stream->queue_size - 1 :
+ oldest_buffer_index - 1),
+ stream->per_buffer_data_size);
+#endif
+
+ /* Pin transferred to caller. */
+ Assert(stream->pinned_buffers > 0);
+ stream->pinned_buffers--;
+
+ /* Advance oldest buffer, with wrap-around. */
+ stream->oldest_buffer_index++;
+ if (stream->oldest_buffer_index == stream->queue_size)
+ stream->oldest_buffer_index = 0;
+
+ /* Prepare for the next call. */
+ read_stream_look_ahead(stream, false);
+
+ return buffer;
+}
+
+/*
+ * Reset a read stream by releasing any queued up buffers, allowing the stream
+ * to be used again for different blocks. This can be used to clear an
+ * end-of-stream condition and start again, or to throw away blocks that were
+ * speculatively read and read some different blocks instead.
+ */
+void
+read_stream_reset(ReadStream *stream)
+{
+ Buffer buffer;
+
+ /* Stop looking ahead. */
+ stream->distance = 0;
+
+ /* Unpin anything that wasn't consumed. */
+ while ((buffer = read_stream_next_buffer(stream, NULL)) != InvalidBuffer)
+ ReleaseBuffer(buffer);
+
+ Assert(stream->pinned_buffers == 0);
+ Assert(stream->ios_in_progress == 0);
+
+ /* Start off assuming data is cached. */
+ stream->distance = 1;
+}
+
+/*
+ * Release and free a read stream.
+ */
+void
+read_stream_end(ReadStream *stream)
+{
+ read_stream_reset(stream);
+ pfree(stream);
+}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index f0f8d4259c5..7123cbbaa2a 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -19,6 +19,11 @@
* and pin it so that no one can destroy it while this process
* is using it.
*
+ * StartReadBuffers() -- as above, but for multiple contiguous blocks in
+ * two steps.
+ *
+ * WaitReadBuffers() -- second step of StartReadBuffers().
+ *
* ReleaseBuffer() -- unpin a buffer
*
* MarkBufferDirty() -- mark a pinned buffer's contents as "dirty".
@@ -160,6 +165,9 @@ int checkpoint_flush_after = DEFAULT_CHECKPOINT_FLUSH_AFTER;
int bgwriter_flush_after = DEFAULT_BGWRITER_FLUSH_AFTER;
int backend_flush_after = DEFAULT_BACKEND_FLUSH_AFTER;
+/* Limit on how many blocks should be handled in single I/O operations. */
+int io_combine_limit = DEFAULT_IO_COMBINE_LIMIT;
+
/* local state for LockBufferForCleanup */
static BufferDesc *PinCountWaitBuf = NULL;
@@ -471,10 +479,9 @@ ForgetPrivateRefCountEntry(PrivateRefCountEntry *ref)
)
-static Buffer ReadBuffer_common(SMgrRelation smgr, char relpersistence,
+static Buffer ReadBuffer_common(BufferManagerRelation bmr,
ForkNumber forkNum, BlockNumber blockNum,
- ReadBufferMode mode, BufferAccessStrategy strategy,
- bool *hit);
+ ReadBufferMode mode, BufferAccessStrategy strategy);
static BlockNumber ExtendBufferedRelCommon(BufferManagerRelation bmr,
ForkNumber fork,
BufferAccessStrategy strategy,
@@ -500,7 +507,7 @@ static uint32 WaitBufHdrUnlocked(BufferDesc *buf);
static int SyncOneBuffer(int buf_id, bool skip_recently_used,
WritebackContext *wb_context);
static void WaitIO(BufferDesc *buf);
-static bool StartBufferIO(BufferDesc *buf, bool forInput);
+static bool StartBufferIO(BufferDesc *buf, bool forInput, bool nowait);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
uint32 set_flag_bits, bool forget_owner);
static void AbortBufferIO(Buffer buffer);
@@ -781,7 +788,6 @@ Buffer
ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy)
{
- bool hit;
Buffer buf;
/*
@@ -794,15 +800,9 @@ ReadBufferExtended(Relation reln, ForkNumber forkNum, BlockNumber blockNum,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
- /*
- * Read the buffer, and update pgstat counters to reflect a cache hit or
- * miss.
- */
- pgstat_count_buffer_read(reln);
- buf = ReadBuffer_common(RelationGetSmgr(reln), reln->rd_rel->relpersistence,
- forkNum, blockNum, mode, strategy, &hit);
- if (hit)
- pgstat_count_buffer_hit(reln);
+ buf = ReadBuffer_common(BMR_REL(reln),
+ forkNum, blockNum, mode, strategy);
+
return buf;
}
@@ -822,13 +822,12 @@ ReadBufferWithoutRelcache(RelFileLocator rlocator, ForkNumber forkNum,
BlockNumber blockNum, ReadBufferMode mode,
BufferAccessStrategy strategy, bool permanent)
{
- bool hit;
-
SMgrRelation smgr = smgropen(rlocator, INVALID_PROC_NUMBER);
- return ReadBuffer_common(smgr, permanent ? RELPERSISTENCE_PERMANENT :
- RELPERSISTENCE_UNLOGGED, forkNum, blockNum,
- mode, strategy, &hit);
+ return ReadBuffer_common(BMR_SMGR(smgr, permanent ? RELPERSISTENCE_PERMANENT :
+ RELPERSISTENCE_UNLOGGED),
+ forkNum, blockNum,
+ mode, strategy);
}
/*
@@ -994,35 +993,146 @@ ExtendBufferedRelTo(BufferManagerRelation bmr,
*/
if (buffer == InvalidBuffer)
{
- bool hit;
-
Assert(extended_by == 0);
- buffer = ReadBuffer_common(bmr.smgr, bmr.relpersistence,
- fork, extend_to - 1, mode, strategy,
- &hit);
+ buffer = ReadBuffer_common(bmr, fork, extend_to - 1, mode, strategy);
}
return buffer;
}
/*
- * ReadBuffer_common -- common logic for all ReadBuffer variants
- *
- * *hit is set to true if the request was satisfied from shared buffer cache.
+ * Zero a buffer and lock it, as part of the implementation of
+ * RBM_ZERO_AND_LOCK or RBM_ZERO_AND_CLEANUP_LOCK. The buffer must be already
+ * pinned. It does not have to be valid, but it is valid and locked on
+ * return.
*/
-static Buffer
-ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
- BlockNumber blockNum, ReadBufferMode mode,
- BufferAccessStrategy strategy, bool *hit)
+static void
+ZeroBuffer(Buffer buffer, ReadBufferMode mode)
{
BufferDesc *bufHdr;
- Block bufBlock;
- bool found;
+ uint32 buf_state;
+
+ Assert(mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK);
+
+ if (BufferIsLocal(buffer))
+ bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+ else
+ {
+ bufHdr = GetBufferDescriptor(buffer - 1);
+ if (mode == RBM_ZERO_AND_LOCK)
+ LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE);
+ else
+ LockBufferForCleanup(buffer);
+ }
+
+ memset(BufferGetPage(buffer), 0, BLCKSZ);
+
+ if (BufferIsLocal(buffer))
+ {
+ buf_state = pg_atomic_read_u32(&bufHdr->state);
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ buf_state = LockBufHdr(bufHdr);
+ buf_state |= BM_VALID;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
+
+/*
+ * Pin a buffer for a given block. *foundPtr is set to true if the block was
+ * already present, or false if more work is required to either read it in or
+ * zero it.
+ */
+static inline Buffer
+PinBufferForBlock(BufferManagerRelation bmr,
+ ForkNumber forkNum,
+ BlockNumber blockNum,
+ BufferAccessStrategy strategy,
+ bool *foundPtr)
+{
+ BufferDesc *bufHdr;
+ bool isLocalBuf;
IOContext io_context;
IOObject io_object;
- bool isLocalBuf = SmgrIsTemp(smgr);
- *hit = false;
+ Assert(blockNum != P_NEW);
+
+ Assert(bmr.smgr);
+
+ isLocalBuf = bmr.relpersistence == RELPERSISTENCE_TEMP;
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(strategy);
+ io_object = IOOBJECT_RELATION;
+ }
+
+ TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend);
+
+ if (isLocalBuf)
+ {
+ bufHdr = LocalBufferAlloc(bmr.smgr, forkNum, blockNum, foundPtr);
+ if (*foundPtr)
+ pgBufferUsage.local_blks_hit++;
+ }
+ else
+ {
+ bufHdr = BufferAlloc(bmr.smgr, bmr.relpersistence, forkNum, blockNum,
+ strategy, foundPtr, io_context);
+ if (*foundPtr)
+ pgBufferUsage.shared_blks_hit++;
+ }
+ if (bmr.rel)
+ {
+ /*
+ * While pgBufferUsage's "read" counter isn't bumped unless we reach
+ * WaitReadBuffers() (so, not for hits, and not for buffers that are
+ * zeroed instead), the per-relation stats always count them.
+ */
+ pgstat_count_buffer_read(bmr.rel);
+ if (*foundPtr)
+ pgstat_count_buffer_hit(bmr.rel);
+ }
+ if (*foundPtr)
+ {
+ VacuumPageHit++;
+ pgstat_count_io_op(io_object, io_context, IOOP_HIT);
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageHit;
+
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
+ bmr.smgr->smgr_rlocator.locator.spcOid,
+ bmr.smgr->smgr_rlocator.locator.dbOid,
+ bmr.smgr->smgr_rlocator.locator.relNumber,
+ bmr.smgr->smgr_rlocator.backend,
+ true);
+ }
+
+ return BufferDescriptorGetBuffer(bufHdr);
+}
+
+/*
+ * ReadBuffer_common -- common logic for all ReadBuffer variants
+ */
+static inline Buffer
+ReadBuffer_common(BufferManagerRelation bmr, ForkNumber forkNum,
+ BlockNumber blockNum, ReadBufferMode mode,
+ BufferAccessStrategy strategy)
+{
+ ReadBuffersOperation operation;
+ Buffer buffer;
+ int flags;
/*
* Backward compatibility path, most code should use ExtendBufferedRel()
@@ -1041,181 +1151,359 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
flags |= EB_LOCK_FIRST;
- return ExtendBufferedRel(BMR_SMGR(smgr, relpersistence),
- forkNum, strategy, flags);
+ return ExtendBufferedRel(bmr, forkNum, strategy, flags);
}
- TRACE_POSTGRESQL_BUFFER_READ_START(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend);
-
- if (isLocalBuf)
+ if (unlikely(mode == RBM_ZERO_AND_CLEANUP_LOCK ||
+ mode == RBM_ZERO_AND_LOCK))
{
- /*
- * We do not use a BufferAccessStrategy for I/O of temporary tables.
- * However, in some cases, the "strategy" may not be NULL, so we can't
- * rely on IOContextForStrategy() to set the right IOContext for us.
- * This may happen in cases like CREATE TEMPORARY TABLE AS...
- */
- io_context = IOCONTEXT_NORMAL;
- io_object = IOOBJECT_TEMP_RELATION;
- bufHdr = LocalBufferAlloc(smgr, forkNum, blockNum, &found);
- if (found)
- pgBufferUsage.local_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.local_blks_read++;
+ bool found;
+
+ if (bmr.smgr == NULL)
+ {
+ bmr.smgr = RelationGetSmgr(bmr.rel);
+ bmr.relpersistence = bmr.rel->rd_rel->relpersistence;
+ }
+
+ buffer = PinBufferForBlock(bmr, forkNum, blockNum, strategy, &found);
+ ZeroBuffer(buffer, mode);
+ return buffer;
}
+
+ if (mode == RBM_ZERO_ON_ERROR)
+ flags = READ_BUFFERS_ZERO_ON_ERROR;
else
+ flags = 0;
+ operation.bmr = bmr;
+ operation.forknum = forkNum;
+ operation.strategy = strategy;
+ if (StartReadBuffer(&operation,
+ &buffer,
+ blockNum,
+ flags))
+ WaitReadBuffers(&operation);
+
+ return buffer;
+}
+
+/*
+ * Single block version of the StartReadBuffers(). This might save a few
+ * instructions when called from another translation unit, if the compiler
+ * inlines the code and specializes for nblocks == 1.
+ */
+bool
+StartReadBuffer(ReadBuffersOperation *operation,
+ Buffer *buffer,
+ BlockNumber blocknum,
+ int flags)
+{
+ int nblocks = 1;
+ bool result;
+
+ result = StartReadBuffers(operation, buffer, blocknum, &nblocks, flags);
+ Assert(nblocks == 1); /* single block can't be short */
+
+ return result;
+}
+
+/*
+ * Begin reading a range of blocks beginning at blockNum and extending for
+ * *nblocks. On return, up to *nblocks pinned buffers holding those blocks
+ * are written into the buffers array, and *nblocks is updated to contain the
+ * actual number, which may be fewer than requested. Caller sets some of the
+ * members of operation; see struct definition.
+ *
+ * If false is returned, no I/O is necessary. If true is returned, one I/O
+ * has been started, and WaitReadBuffers() must be called with the same
+ * operation object before the buffers are accessed. Along with the operation
+ * object, the caller-supplied array of buffers must remain valid until
+ * WaitReadBuffers() is called.
+ *
+ * Currently the I/O is only started with optional operating system advice,
+ * and the real I/O happens in WaitReadBuffers(). In future work, true I/O
+ * could be initiated here.
+ */
+inline bool
+StartReadBuffers(ReadBuffersOperation *operation,
+ Buffer *buffers,
+ BlockNumber blockNum,
+ int *nblocks,
+ int flags)
+{
+ int actual_nblocks = *nblocks;
+ int io_buffers_len = 0;
+
+ Assert(*nblocks > 0);
+ Assert(*nblocks <= MAX_IO_COMBINE_LIMIT);
+
+ if (!operation->bmr.smgr)
{
- /*
- * lookup the buffer. IO_IN_PROGRESS is set if the requested block is
- * not currently in memory.
- */
- io_context = IOContextForStrategy(strategy);
- io_object = IOOBJECT_RELATION;
- bufHdr = BufferAlloc(smgr, relpersistence, forkNum, blockNum,
- strategy, &found, io_context);
- if (found)
- pgBufferUsage.shared_blks_hit++;
- else if (mode == RBM_NORMAL || mode == RBM_NORMAL_NO_LOG ||
- mode == RBM_ZERO_ON_ERROR)
- pgBufferUsage.shared_blks_read++;
+ operation->bmr.smgr = RelationGetSmgr(operation->bmr.rel);
+ operation->bmr.relpersistence = operation->bmr.rel->rd_rel->relpersistence;
}
- /* At this point we do NOT hold any locks. */
-
- /* if it was already in the buffer pool, we're done */
- if (found)
+ for (int i = 0; i < actual_nblocks; ++i)
{
- /* Just need to update stats before we exit */
- *hit = true;
- VacuumPageHit++;
- pgstat_count_io_op(io_object, io_context, IOOP_HIT);
+ bool found;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageHit;
+ buffers[i] = PinBufferForBlock(operation->bmr,
+ operation->forknum,
+ blockNum + i,
+ operation->strategy,
+ &found);
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ if (found)
+ {
+ /*
+ * Terminate the read as soon as we get a hit. It could be a
+ * single buffer hit, or it could be a hit that follows a readable
+ * range. We don't want to create more than one readable range,
+ * so we stop here.
+ */
+ actual_nblocks = i + 1;
+ break;
+ }
+ else
+ {
+ /* Extend the readable range to cover this block. */
+ io_buffers_len++;
+ }
+ }
+ *nblocks = actual_nblocks;
- /*
- * In RBM_ZERO_AND_LOCK mode the caller expects the page to be locked
- * on return.
- */
- if (!isLocalBuf)
+ if (io_buffers_len > 0)
+ {
+ /* Populate information needed for I/O. */
+ operation->buffers = buffers;
+ operation->blocknum = blockNum;
+ operation->flags = flags;
+ operation->nblocks = actual_nblocks;
+ operation->io_buffers_len = io_buffers_len;
+
+ if (flags & READ_BUFFERS_ISSUE_ADVICE)
{
- if (mode == RBM_ZERO_AND_LOCK)
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr),
- LW_EXCLUSIVE);
- else if (mode == RBM_ZERO_AND_CLEANUP_LOCK)
- LockBufferForCleanup(BufferDescriptorGetBuffer(bufHdr));
+ /*
+ * In theory we should only do this if PinBufferForBlock() had to
+ * allocate new buffers above. That way, if two calls to
+ * StartReadBuffers() were made for the same blocks before
+ * WaitReadBuffers(), only the first would issue the advice.
+ * That'd be a better simulation of true asynchronous I/O, which
+ * would only start the I/O once, but isn't done here for
+ * simplicity. Note also that the following call might actually
+ * issue two advice calls if we cross a segment boundary; in a
+ * true asynchronous version we might choose to process only one
+ * real I/O at a time in that case.
+ */
+ smgrprefetch(operation->bmr.smgr,
+ operation->forknum,
+ blockNum,
+ operation->io_buffers_len);
}
- return BufferDescriptorGetBuffer(bufHdr);
+ /* Indicate that WaitReadBuffers() should be called. */
+ return true;
+ }
+ else
+ {
+ return false;
}
+}
+
+static inline bool
+WaitReadBuffersCanStartIO(Buffer buffer, bool nowait)
+{
+ if (BufferIsLocal(buffer))
+ {
+ BufferDesc *bufHdr = GetLocalBufferDescriptor(-buffer - 1);
+
+ return (pg_atomic_read_u32(&bufHdr->state) & BM_VALID) == 0;
+ }
+ else
+ return StartBufferIO(GetBufferDescriptor(buffer - 1), true, nowait);
+}
+
+void
+WaitReadBuffers(ReadBuffersOperation *operation)
+{
+ Buffer *buffers;
+ int nblocks;
+ BlockNumber blocknum;
+ ForkNumber forknum;
+ bool isLocalBuf;
+ IOContext io_context;
+ IOObject io_object;
/*
- * if we have gotten to this point, we have allocated a buffer for the
- * page but its contents are not yet valid. IO_IN_PROGRESS is set for it,
- * if it's a shared buffer.
+ * Currently operations are only allowed to include a read of some range,
+ * with an optional extra buffer that is already pinned at the end. So
+ * nblocks can be at most one more than io_buffers_len.
*/
- Assert(!(pg_atomic_read_u32(&bufHdr->state) & BM_VALID)); /* spinlock not needed */
+ Assert((operation->nblocks == operation->io_buffers_len) ||
+ (operation->nblocks == operation->io_buffers_len + 1));
- bufBlock = isLocalBuf ? LocalBufHdrGetBlock(bufHdr) : BufHdrGetBlock(bufHdr);
+ /* Find the range of the physical read we need to perform. */
+ nblocks = operation->io_buffers_len;
+ if (nblocks == 0)
+ return; /* nothing to do */
+
+ buffers = &operation->buffers[0];
+ blocknum = operation->blocknum;
+ forknum = operation->forknum;
+
+ isLocalBuf = operation->bmr.relpersistence == RELPERSISTENCE_TEMP;
+ if (isLocalBuf)
+ {
+ io_context = IOCONTEXT_NORMAL;
+ io_object = IOOBJECT_TEMP_RELATION;
+ }
+ else
+ {
+ io_context = IOContextForStrategy(operation->strategy);
+ io_object = IOOBJECT_RELATION;
+ }
/*
- * Read in the page, unless the caller intends to overwrite it and just
- * wants us to allocate a buffer.
+ * We count all these blocks as read by this backend. This is traditional
+ * behavior, but might turn out to be not true if we find that someone
+ * else has beaten us and completed the read of some of these blocks. In
+ * that case the system globally double-counts, but we traditionally don't
+ * count this as a "hit", and we don't have a separate counter for "miss,
+ * but another backend completed the read".
*/
- if (mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK)
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ if (isLocalBuf)
+ pgBufferUsage.local_blks_read += nblocks;
else
+ pgBufferUsage.shared_blks_read += nblocks;
+
+ for (int i = 0; i < nblocks; ++i)
{
- instr_time io_start = pgstat_prepare_io_time(track_io_timing);
+ int io_buffers_len;
+ Buffer io_buffers[MAX_IO_COMBINE_LIMIT];
+ void *io_pages[MAX_IO_COMBINE_LIMIT];
+ instr_time io_start;
+ BlockNumber io_first_block;
+
+ /*
+ * Skip this block if someone else has already completed it. If an
+ * I/O is already in progress in another backend, this will wait for
+ * the outcome: either done, or something went wrong and we will
+ * retry.
+ */
+ if (!WaitReadBuffersCanStartIO(buffers[i], false))
+ {
+ /*
+ * Report this as a 'hit' for this backend, even though it must
+ * have started out as a miss in PinBufferForBlock().
+ */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, blocknum + i,
+ operation->bmr.smgr->smgr_rlocator.locator.spcOid,
+ operation->bmr.smgr->smgr_rlocator.locator.dbOid,
+ operation->bmr.smgr->smgr_rlocator.locator.relNumber,
+ operation->bmr.smgr->smgr_rlocator.backend,
+ true);
+ continue;
+ }
+
+ /* We found a buffer that we need to read in. */
+ io_buffers[0] = buffers[i];
+ io_pages[0] = BufferGetBlock(buffers[i]);
+ io_first_block = blocknum + i;
+ io_buffers_len = 1;
- smgrread(smgr, forkNum, blockNum, bufBlock);
+ /*
+ * How many neighboring-on-disk blocks can we can scatter-read into
+ * other buffers at the same time? In this case we don't wait if we
+ * see an I/O already in progress. We already hold BM_IO_IN_PROGRESS
+ * for the head block, so we should get on with that I/O as soon as
+ * possible. We'll come back to this block again, above.
+ */
+ while ((i + 1) < nblocks &&
+ WaitReadBuffersCanStartIO(buffers[i + 1], true))
+ {
+ /* Must be consecutive block numbers. */
+ Assert(BufferGetBlockNumber(buffers[i + 1]) ==
+ BufferGetBlockNumber(buffers[i]) + 1);
+
+ io_buffers[io_buffers_len] = buffers[++i];
+ io_pages[io_buffers_len++] = BufferGetBlock(buffers[i]);
+ }
- pgstat_count_io_op_time(io_object, io_context,
- IOOP_READ, io_start, 1);
+ io_start = pgstat_prepare_io_time(track_io_timing);
+ smgrreadv(operation->bmr.smgr, forknum, io_first_block, io_pages, io_buffers_len);
+ pgstat_count_io_op_time(io_object, io_context, IOOP_READ, io_start,
+ io_buffers_len);
- /* check for garbage data */
- if (!PageIsVerifiedExtended((Page) bufBlock, blockNum,
- PIV_LOG_WARNING | PIV_REPORT_STAT))
+ /* Verify each block we read, and terminate the I/O. */
+ for (int j = 0; j < io_buffers_len; ++j)
{
- if (mode == RBM_ZERO_ON_ERROR || zero_damaged_pages)
+ BufferDesc *bufHdr;
+ Block bufBlock;
+
+ if (isLocalBuf)
{
- ereport(WARNING,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s; zeroing out page",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- MemSet((char *) bufBlock, 0, BLCKSZ);
+ bufHdr = GetLocalBufferDescriptor(-io_buffers[j] - 1);
+ bufBlock = LocalBufHdrGetBlock(bufHdr);
}
else
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("invalid page in block %u of relation %s",
- blockNum,
- relpath(smgr->smgr_rlocator, forkNum))));
- }
- }
-
- /*
- * In RBM_ZERO_AND_LOCK / RBM_ZERO_AND_CLEANUP_LOCK mode, grab the buffer
- * content lock before marking the page as valid, to make sure that no
- * other backend sees the zeroed page before the caller has had a chance
- * to initialize it.
- *
- * Since no-one else can be looking at the page contents yet, there is no
- * difference between an exclusive lock and a cleanup-strength lock. (Note
- * that we cannot use LockBuffer() or LockBufferForCleanup() here, because
- * they assert that the buffer is already valid.)
- */
- if ((mode == RBM_ZERO_AND_LOCK || mode == RBM_ZERO_AND_CLEANUP_LOCK) &&
- !isLocalBuf)
- {
- LWLockAcquire(BufferDescriptorGetContentLock(bufHdr), LW_EXCLUSIVE);
- }
+ {
+ bufHdr = GetBufferDescriptor(io_buffers[j] - 1);
+ bufBlock = BufHdrGetBlock(bufHdr);
+ }
- if (isLocalBuf)
- {
- /* Only need to adjust flags */
- uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
+ /* check for garbage data */
+ if (!PageIsVerifiedExtended((Page) bufBlock, io_first_block + j,
+ PIV_LOG_WARNING | PIV_REPORT_STAT))
+ {
+ if ((operation->flags & READ_BUFFERS_ZERO_ON_ERROR) || zero_damaged_pages)
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s; zeroing out page",
+ io_first_block + j,
+ relpath(operation->bmr.smgr->smgr_rlocator, forknum))));
+ memset(bufBlock, 0, BLCKSZ);
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("invalid page in block %u of relation %s",
+ io_first_block + j,
+ relpath(operation->bmr.smgr->smgr_rlocator, forknum))));
+ }
- buf_state |= BM_VALID;
- pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
- }
- else
- {
- /* Set BM_VALID, terminate IO, and wake up any waiters */
- TerminateBufferIO(bufHdr, false, BM_VALID, true);
- }
+ /* Terminate I/O and set BM_VALID. */
+ if (isLocalBuf)
+ {
+ uint32 buf_state = pg_atomic_read_u32(&bufHdr->state);
- VacuumPageMiss++;
- if (VacuumCostActive)
- VacuumCostBalance += VacuumCostPageMiss;
+ buf_state |= BM_VALID;
+ pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
+ }
+ else
+ {
+ /* Set BM_VALID, terminate IO, and wake up any waiters */
+ TerminateBufferIO(bufHdr, false, BM_VALID, true);
+ }
- TRACE_POSTGRESQL_BUFFER_READ_DONE(forkNum, blockNum,
- smgr->smgr_rlocator.locator.spcOid,
- smgr->smgr_rlocator.locator.dbOid,
- smgr->smgr_rlocator.locator.relNumber,
- smgr->smgr_rlocator.backend,
- found);
+ /* Report I/Os as completing individually. */
+ TRACE_POSTGRESQL_BUFFER_READ_DONE(forknum, io_first_block + j,
+ operation->bmr.smgr->smgr_rlocator.locator.spcOid,
+ operation->bmr.smgr->smgr_rlocator.locator.dbOid,
+ operation->bmr.smgr->smgr_rlocator.locator.relNumber,
+ operation->bmr.smgr->smgr_rlocator.backend,
+ false);
+ }
- return BufferDescriptorGetBuffer(bufHdr);
+ VacuumPageMiss += io_buffers_len;
+ if (VacuumCostActive)
+ VacuumCostBalance += VacuumCostPageMiss * io_buffers_len;
+ }
}
/*
- * BufferAlloc -- subroutine for ReadBuffer. Handles lookup of a shared
- * buffer. If no buffer exists already, selects a replacement
- * victim and evicts the old page, but does NOT read in new page.
+ * BufferAlloc -- subroutine for PinBufferForBlock. Handles lookup of a shared
+ * buffer. If no buffer exists already, selects a replacement victim and
+ * evicts the old page, but does NOT read in new page.
*
* "strategy" can be a buffer replacement strategy object, or NULL for
* the default strategy. The selected buffer's usage_count is advanced when
@@ -1223,11 +1511,7 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
*
* The returned buffer is pinned and is already marked as holding the
* desired page. If it already did have the desired page, *foundPtr is
- * set true. Otherwise, *foundPtr is set false and the buffer is marked
- * as IO_IN_PROGRESS; ReadBuffer will now need to do I/O to fill it.
- *
- * *foundPtr is actually redundant with the buffer's BM_VALID flag, but
- * we keep it for simplicity in ReadBuffer.
+ * set true. Otherwise, *foundPtr is set false.
*
* io_context is passed as an output parameter to avoid calling
* IOContextForStrategy() when there is a shared buffers hit and no IO
@@ -1286,19 +1570,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(buf, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return buf;
@@ -1363,19 +1638,10 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
{
/*
* We can only get here if (a) someone else is still reading in
- * the page, or (b) a previous read attempt failed. We have to
- * wait for any active read attempt to finish, and then set up our
- * own read attempt if the page is still not BM_VALID.
- * StartBufferIO does it all.
+ * the page, (b) a previous read attempt failed, or (c) someone
+ * called StartReadBuffers() but not yet WaitReadBuffers().
*/
- if (StartBufferIO(existing_buf_hdr, true))
- {
- /*
- * If we get here, previous attempts to read the buffer must
- * have failed ... but we shall bravely try again.
- */
- *foundPtr = false;
- }
+ *foundPtr = false;
}
return existing_buf_hdr;
@@ -1407,15 +1673,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
LWLockRelease(newPartitionLock);
/*
- * Buffer contents are currently invalid. Try to obtain the right to
- * start I/O. If StartBufferIO returns false, then someone else managed
- * to read it before we did, so there's nothing left for BufferAlloc() to
- * do.
+ * Buffer contents are currently invalid.
*/
- if (StartBufferIO(victim_buf_hdr, true))
- *foundPtr = false;
- else
- *foundPtr = true;
+ *foundPtr = false;
return victim_buf_hdr;
}
@@ -1769,7 +2029,7 @@ again:
* pessimistic, but outside of toy-sized shared_buffers it should allow
* sufficient pins.
*/
-static void
+void
LimitAdditionalPins(uint32 *additional_pins)
{
uint32 max_backends;
@@ -2034,7 +2294,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
buf_state &= ~BM_VALID;
UnlockBufHdr(existing_hdr, buf_state);
- } while (!StartBufferIO(existing_hdr, true));
+ } while (!StartBufferIO(existing_hdr, true, false));
}
else
{
@@ -2057,7 +2317,7 @@ ExtendBufferedRelShared(BufferManagerRelation bmr,
LWLockRelease(partition_lock);
/* XXX: could combine the locked operations in it with the above */
- StartBufferIO(victim_buf_hdr, true);
+ StartBufferIO(victim_buf_hdr, true, false);
}
}
@@ -2372,7 +2632,12 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
else
{
/*
- * If we previously pinned the buffer, it must surely be valid.
+ * If we previously pinned the buffer, it is likely to be valid, but
+ * it may not be if StartReadBuffers() was called and
+ * WaitReadBuffers() hasn't been called yet. We'll check by loading
+ * the flags without locking. This is racy, but it's OK to return
+ * false spuriously: when WaitReadBuffers() calls StartBufferIO(),
+ * it'll see that it's now valid.
*
* Note: We deliberately avoid a Valgrind client request here.
* Individual access methods can optionally superimpose buffer page
@@ -2381,7 +2646,7 @@ PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy)
* that the buffer page is legitimately non-accessible here. We
* cannot meddle with that.
*/
- result = true;
+ result = (pg_atomic_read_u32(&buf->state) & BM_VALID) != 0;
}
ref->refcount++;
@@ -3449,7 +3714,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln, IOObject io_object,
* someone else flushed the buffer before we could, so we need not do
* anything.
*/
- if (!StartBufferIO(buf, false))
+ if (!StartBufferIO(buf, false, false))
return;
/* Setup error traceback support for ereport() */
@@ -5184,9 +5449,15 @@ WaitIO(BufferDesc *buf)
*
* Returns true if we successfully marked the buffer as I/O busy,
* false if someone else already did the work.
+ *
+ * If nowait is true, then we don't wait for an I/O to be finished by another
+ * backend. In that case, false indicates either that the I/O was already
+ * finished, or is still in progress. This is useful for callers that want to
+ * find out if they can perform the I/O as part of a larger operation, without
+ * waiting for the answer or distinguishing the reasons why not.
*/
static bool
-StartBufferIO(BufferDesc *buf, bool forInput)
+StartBufferIO(BufferDesc *buf, bool forInput, bool nowait)
{
uint32 buf_state;
@@ -5199,6 +5470,8 @@ StartBufferIO(BufferDesc *buf, bool forInput)
if (!(buf_state & BM_IO_IN_PROGRESS))
break;
UnlockBufHdr(buf, buf_state);
+ if (nowait)
+ return false;
WaitIO(buf);
}
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index fcfac335a57..985a2c7049c 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -108,10 +108,9 @@ PrefetchLocalBuffer(SMgrRelation smgr, ForkNumber forkNum,
* LocalBufferAlloc -
* Find or create a local buffer for the given page of the given relation.
*
- * API is similar to bufmgr.c's BufferAlloc, except that we do not need
- * to do any locking since this is all local. Also, IO_IN_PROGRESS
- * does not get set. Lastly, we support only default access strategy
- * (hence, usage_count is always advanced).
+ * API is similar to bufmgr.c's BufferAlloc, except that we do not need to do
+ * any locking since this is all local. We support only default access
+ * strategy (hence, usage_count is always advanced).
*/
BufferDesc *
LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
@@ -287,7 +286,7 @@ GetLocalVictimBuffer(void)
}
/* see LimitAdditionalPins() */
-static void
+void
LimitAdditionalLocalPins(uint32 *additional_pins)
{
uint32 max_pins;
@@ -297,9 +296,10 @@ LimitAdditionalLocalPins(uint32 *additional_pins)
/*
* In contrast to LimitAdditionalPins() other backends don't play a role
- * here. We can allow up to NLocBuffer pins in total.
+ * here. We can allow up to NLocBuffer pins in total, but it might not be
+ * initialized yet so read num_temp_buffers.
*/
- max_pins = (NLocBuffer - NLocalPinnedBuffers);
+ max_pins = (num_temp_buffers - NLocalPinnedBuffers);
if (*additional_pins >= max_pins)
*additional_pins = max_pins;
diff --git a/src/backend/storage/meson.build b/src/backend/storage/meson.build
index 40345bdca27..739d13293fb 100644
--- a/src/backend/storage/meson.build
+++ b/src/backend/storage/meson.build
@@ -1,5 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+subdir('aio')
subdir('buffer')
subdir('file')
subdir('freespace')
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 92fcd5fa4d5..c12784cbec8 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3129,6 +3129,20 @@ struct config_int ConfigureNamesInt[] =
NULL
},
+ {
+ {"io_combine_limit",
+ PGC_USERSET,
+ RESOURCES_ASYNCHRONOUS,
+ gettext_noop("Limit on the size of data reads and writes."),
+ NULL,
+ GUC_UNIT_BLOCKS
+ },
+ &io_combine_limit,
+ DEFAULT_IO_COMBINE_LIMIT,
+ 1, MAX_IO_COMBINE_LIMIT,
+ NULL, NULL, NULL
+ },
+
{
{"backend_flush_after", PGC_USERSET, RESOURCES_ASYNCHRONOUS,
gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index adcc0257f91..baecde28410 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -203,6 +203,7 @@
#backend_flush_after = 0 # measured in pages, 0 disables
#effective_io_concurrency = 1 # 1-1000; 0 disables prefetching
#maintenance_io_concurrency = 10 # 1-1000; 0 disables prefetching
+#io_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
#max_worker_processes = 8 # (change requires restart)
#max_parallel_workers_per_gather = 2 # limited by max_parallel_workers
#max_parallel_maintenance_workers = 2 # limited by max_parallel_workers
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d51d46d3353..241f68c45e1 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -14,6 +14,7 @@
#ifndef BUFMGR_H
#define BUFMGR_H
+#include "port/pg_iovec.h"
#include "storage/block.h"
#include "storage/buf.h"
#include "storage/bufpage.h"
@@ -133,6 +134,10 @@ extern PGDLLIMPORT bool track_io_timing;
extern PGDLLIMPORT int effective_io_concurrency;
extern PGDLLIMPORT int maintenance_io_concurrency;
+#define MAX_IO_COMBINE_LIMIT PG_IOV_MAX
+#define DEFAULT_IO_COMBINE_LIMIT Min(MAX_IO_COMBINE_LIMIT, (128 * 1024) / BLCKSZ)
+extern PGDLLIMPORT int io_combine_limit;
+
extern PGDLLIMPORT int checkpoint_flush_after;
extern PGDLLIMPORT int backend_flush_after;
extern PGDLLIMPORT int bgwriter_flush_after;
@@ -158,7 +163,6 @@ extern PGDLLIMPORT int32 *LocalRefCount;
#define BUFFER_LOCK_SHARE 1
#define BUFFER_LOCK_EXCLUSIVE 2
-
/*
* prototypes for functions in bufmgr.c
*/
@@ -177,6 +181,38 @@ extern Buffer ReadBufferWithoutRelcache(RelFileLocator rlocator,
ForkNumber forkNum, BlockNumber blockNum,
ReadBufferMode mode, BufferAccessStrategy strategy,
bool permanent);
+
+#define READ_BUFFERS_ZERO_ON_ERROR 0x01
+#define READ_BUFFERS_ISSUE_ADVICE 0x02
+
+struct ReadBuffersOperation
+{
+ /* The following members should be set by the caller. */
+ BufferManagerRelation bmr;
+ ForkNumber forknum;
+ BufferAccessStrategy strategy;
+
+ /* The following private members should not be accessed directly. */
+ Buffer *buffers;
+ BlockNumber blocknum;
+ int flags;
+ int16 nblocks;
+ int16 io_buffers_len;
+};
+
+typedef struct ReadBuffersOperation ReadBuffersOperation;
+
+extern bool StartReadBuffer(ReadBuffersOperation *operation,
+ Buffer *buffer,
+ BlockNumber blocknum,
+ int flags);
+extern bool StartReadBuffers(ReadBuffersOperation *operation,
+ Buffer *buffers,
+ BlockNumber blocknum,
+ int *nblocks,
+ int flags);
+extern void WaitReadBuffers(ReadBuffersOperation *operation);
+
extern void ReleaseBuffer(Buffer buffer);
extern void UnlockReleaseBuffer(Buffer buffer);
extern bool BufferIsExclusiveLocked(Buffer buffer);
@@ -250,6 +286,9 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern bool BgBufferSync(struct WritebackContext *wb_context);
+extern void LimitAdditionalPins(uint32 *additional_pins);
+extern void LimitAdditionalLocalPins(uint32 *additional_pins);
+
/* in buf_init.c */
extern void InitBufferPool(void);
extern Size BufferShmemSize(void);
diff --git a/src/include/storage/read_stream.h b/src/include/storage/read_stream.h
new file mode 100644
index 00000000000..f5dbc087b0b
--- /dev/null
+++ b/src/include/storage/read_stream.h
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * read_stream.h
+ * Mechanism for accessing buffered relation data with look-ahead
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/read_stream.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef READ_STREAM_H
+#define READ_STREAM_H
+
+#include "storage/bufmgr.h"
+
+/* Default tuning, reasonable for many users. */
+#define READ_STREAM_DEFAULT 0x00
+
+/*
+ * I/O streams that are performing maintenance work on behalf of potentially
+ * many users, and thus should be governed by maintenance_io_concurrency
+ * instead of effective_io_concurrency. For example, VACUUM or CREATE INDEX.
+ */
+#define READ_STREAM_MAINTENANCE 0x01
+
+/*
+ * We usually avoid issuing prefetch advice automatically when sequential
+ * access is detected, but this flag explicitly disables it, for cases that
+ * might not be correctly detected. Explicit advice is known to perform worse
+ * than letting the kernel (at least Linux) detect sequential access.
+ */
+#define READ_STREAM_SEQUENTIAL 0x02
+
+/*
+ * We usually ramp up from smaller reads to larger ones, to support users who
+ * don't know if it's worth reading lots of buffers yet. This flag disables
+ * that, declaring ahead of time that we'll be reading all available buffers.
+ */
+#define READ_STREAM_FULL 0x04
+
+struct ReadStream;
+typedef struct ReadStream ReadStream;
+
+/* Callback that returns the next block number to read. */
+typedef BlockNumber (*ReadStreamBlockNumberCB) (ReadStream *stream,
+ void *callback_private_data,
+ void *per_buffer_data);
+
+extern ReadStream *read_stream_begin_relation(int flags,
+ BufferAccessStrategy strategy,
+ BufferManagerRelation bmr,
+ ForkNumber forknum,
+ ReadStreamBlockNumberCB callback,
+ void *callback_private_data,
+ size_t per_buffer_data_size);
+extern Buffer read_stream_next_buffer(ReadStream *stream, void **per_buffer_private);
+extern void read_stream_reset(ReadStream *stream);
+extern void read_stream_end(ReadStream *stream);
+
+#endif /* READ_STREAM_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0fb2dad2053..e652b5b8141 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1215,6 +1215,7 @@ InjectionPointCacheEntry
InjectionPointEntry
InjectionPointSharedState
InlineCodeBlock
+InProgressIO
InsertStmt
Instrumentation
Int128AggState
@@ -2288,11 +2289,13 @@ ReInitializeDSMForeignScan_function
ReScanForeignScan_function
ReadBufPtrType
ReadBufferMode
+ReadBuffersOperation
ReadBytePtrType
ReadExtraTocPtrType
ReadFunc
ReadLocalXLogPageNoWaitPrivate
ReadReplicationSlotCmd
+ReadStream
ReassignOwnedStmt
RecheckForeignScan_function
RecordCacheArrayEntry
--
2.40.1
[text/x-diff] v12-0017-BitmapHeapScan-uses-read-stream-API.patch (26.5K, ../../20240331154551.lty4mundyoyhtixw@liskov/18-v12-0017-BitmapHeapScan-uses-read-stream-API.patch)
download | inline diff:
From 8f207aa0d17355228fb8bf4df5c1c82c3be935bf Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v12 17/17] BitmapHeapScan uses read stream API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the read stream API prefetching. Heap table AM implements a read
stream callback which uses the iterator to get the next valid block that
needs to be fetched for the read stream API.
---
src/backend/access/heap/heapam.c | 99 ++++--
src/backend/access/heap/heapam_handler.c | 347 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 43 +--
src/include/access/heapam.h | 21 +-
src/include/access/relscan.h | 6 -
src/include/access/tableam.h | 14 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 121 insertions(+), 418 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 8dc1eab5348..2a9547249a0 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -108,6 +108,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_stream_read_next(ReadStream *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -324,6 +326,23 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
/* page-at-a-time fields are always invalid when not rs_inited */
+ /*
+ * Make the read stream object for BitmapHeapScan. If this is a rescan, we
+ * expect that the previous read stream was ended in heap_rescan() before
+ * invoking initscan(). Do not create the read stream until after creating
+ * the BufferAccessStrategy object.
+ */
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ scan->rs_read_stream = read_stream_begin_relation(READ_STREAM_DEFAULT,
+ scan->rs_strategy,
+ BMR_REL(scan->rs_base.rs_rd),
+ MAIN_FORKNUM,
+ bitmapheap_stream_read_next,
+ scan,
+ sizeof(TBMIterateResult));
+ }
+
/*
* copy the scan key, if appropriate
*/
@@ -950,16 +969,9 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
+ scan->rs_read_stream = NULL;
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1042,12 +1054,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1060,10 +1066,13 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
+ /* Free the old read stream before reinitializing the scan descriptor */
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
{
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
+ if (scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
+
+ scan->rs_read_stream = NULL;
}
/*
@@ -1091,11 +1100,9 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
+ /* We must free the read stream before the BufferAccessStrategy object */
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
/*
* decrement relation reference count and free scan descriptor storage
@@ -10132,3 +10139,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_stream_read_next(ReadStream *pgsr, void *private_data,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) private_data;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_bhs_iterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 717bcd08175..859bc4603d0 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -61,9 +61,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2194,147 +2191,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *per_buffer_data;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_read_stream);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_bhs_iterator, &tbmres);
+ hscan->rs_cbuf = read_stream_next_buffer(hscan->rs_read_stream, &per_buffer_data);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(per_buffer_data);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = per_buffer_data;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2355,7 +2273,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2364,9 +2282,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2416,23 +2334,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_bhs_iterator &&
- hscan->pfblockno < hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2443,153 +2349,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2624,41 +2383,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2f9387e51a2..f2662ea542c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -131,14 +131,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -149,15 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,13 +157,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -213,22 +190,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- {
- scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
- scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
- dsa);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
}
@@ -525,14 +492,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -563,11 +526,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index accbc1749cd..c4943af9e4d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/read_stream.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -72,6 +73,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ ReadStream *rs_read_stream;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -82,23 +86,6 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /*
- * These fields only used for prefetching in bitmap table scans
- */
-
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used in serial BHS
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
-
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 7938b741d66..02893bf99bc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -46,13 +46,7 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
- struct BitmapHeapIterator *rs_pf_bhs_iterator;
-
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate BHS prefetch and current block stay in sync */
- BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 8665e90fbc7..559ea9c248d 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -940,8 +940,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
- result->rs_pf_bhs_iterator = NULL;
- result->prefetch_maximum = 0;
result->bm_parallel = NULL;
return result;
}
@@ -994,12 +992,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1016,12 +1008,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 592215d5ee7..8d86b900a0b 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1767,11 +1767,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1779,10 +1775,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-31 15:45 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
@ 2024-04-03 22:57 ` Melanie Plageman <[email protected]>
2024-04-04 14:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Melanie Plageman @ 2024-04-03 22:57 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Thomas Munro <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Sun, Mar 31, 2024 at 11:45:51AM -0400, Melanie Plageman wrote:
> On Fri, Mar 29, 2024 at 12:05:15PM +0100, Tomas Vondra wrote:
> >
> >
> > On 3/29/24 02:12, Thomas Munro wrote:
> > > On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
> > > <[email protected]> wrote:
> > >> I think there's some sort of bug, triggering this assert in heapam
> > >>
> > >> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
> > >
> > > Thanks for the repro. I can't seem to reproduce it (still trying) but
> > > I assume this is with Melanie's v11 patch set which had
> > > v11-0016-v10-Read-Stream-API.patch.
> > >
> > > Would you mind removing that commit and instead applying the v13
> > > stream_read.c patches[1]? v10 stream_read.c was a little confused
> > > about random I/O combining, which I fixed with a small adjustment to
> > > the conditions for the "if" statement right at the end of
> > > read_stream_look_ahead(). Sorry about that. The fixed version, with
> > > eic=4, with your test query using WHERE a < a, ends its scan with:
> > >
> >
> > I'll give that a try. Unfortunately unfortunately the v11 still has the
> > problem I reported about a week ago:
> >
> > ERROR: prefetch and main iterators are out of sync
> >
> > So I can't run the full benchmarks :-( but master vs. streaming read API
> > should work, I think.
>
> Odd, I didn't notice you reporting this ERROR popping up. Now that I
> take a look, v11 (at least, maybe also v10) had this very sill mistake:
>
> if (scan->bm_parallel == NULL &&
> scan->rs_pf_bhs_iterator &&
> hscan->pfblockno > hscan->rs_base.blockno)
> elog(ERROR, "prefetch and main iterators are out of sync");
>
> It errors out if the prefetch block is ahead of the current block --
> which is the opposite of what we want. I've fixed this in attached v12.
>
> This version also has v13 of the streaming read API. I noticed one
> mistake in my bitmapheap scan streaming read user -- it freed the
> streaming read object at the wrong time. I don't know if this was
> causing any other issues, but it at least is fixed in this version.
Attached v13 is rebased over master (which includes the streaming read
API now). I also reset the streaming read object on rescan instead of
creating a new one each time.
I don't know how much chance any of this has of going in to 17 now, but
I thought I would start looking into the regression repro Tomas provided
in [1].
I'm also not sure if I should try and group the commits into fewer
commits now or wait until I have some idea of whether or not the
approach in 0013 and 0014 is worth pursuing.
- Melanie
[1] https://www.postgresql.org/message-id/5d5954ed-6f43-4f1a-8e19-ece75b2b7362%40enterprisedb.com
Attachments:
[text/x-diff] v13-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../20240403225759.sq5hmxdmbalroosy@liskov/2-v13-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 3a3cffe2adb8b43ae0298bb23633856cc4045299 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v13 01/16] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 805d222ceb..b0f61c65f3 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..93fdcd226b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v13-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.2K, ../../20240403225759.sq5hmxdmbalroosy@liskov/3-v13-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 8244da026228e82333eb501d0d9f96540d2282be Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v13 02/16] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226b..c64530674b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -742,16 +752,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v13-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (15.0K, ../../20240403225759.sq5hmxdmbalroosy@liskov/4-v13-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From 378bf4147ec506c32d01fa94335299804db42f18 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 20:15:05 -0500
Subject: [PATCH v13 03/16] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index a9d5b109a5..50b4bc1475 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,6 +948,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1036,6 +1038,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1055,6 +1063,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 0952d4a98e..ae894a7573 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2198,6 +2199,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2310,6 +2329,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c64530674b..83d9db8f39 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,11 +185,25 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
}
node->initialized = true;
@@ -207,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -228,37 +232,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -301,52 +282,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -518,7 +480,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -569,7 +531,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -639,8 +601,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -650,7 +610,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -695,8 +654,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -740,8 +697,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -752,7 +707,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index a307fb5f24..a527e1b99f 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -76,6 +76,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e7eeb75409..55f1139757 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -63,6 +63,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -937,9 +944,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e7ff8e4992..4880f346bf 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,10 +1794,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1817,9 +1814,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v13-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.2K, ../../20240403225759.sq5hmxdmbalroosy@liskov/5-v13-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From c2fd41ee5afbd8bd643bc6cd3c8529f912e83219 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v13 04/16] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 83d9db8f39..5df3b5ca46 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -474,14 +474,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -532,7 +527,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v13-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.3K, ../../20240403225759.sq5hmxdmbalroosy@liskov/6-v13-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 48d65097497466d8c23a1bfe38b74c2942900f24 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v13 05/16] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5df3b5ca46..404de0595e 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -230,7 +230,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -341,7 +341,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -360,7 +360,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v13-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch (4.4K, ../../20240403225759.sq5hmxdmbalroosy@liskov/7-v13-0006-table_scan_bitmap_next_block-returns-lossy-or-ex.patch)
download | inline diff:
From 34ca369410bf7c1161cf34b5eafb74099a8d3811 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v13 06/16] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ae894a7573..48b8359786 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,7 +2188,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2316,6 +2317,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 404de0595e..c95e3412da 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,7 +211,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -232,12 +232,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 55f1139757..75c5ee956d 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -789,6 +789,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -804,7 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1971,14 +1975,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1989,7 +1995,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.40.1
[text/x-diff] v13-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (2.9K, ../../20240403225759.sq5hmxdmbalroosy@liskov/8-v13-0007-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From fe6ac36bbd1dadebc9c2e7a2958c7009029f20b1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v13 07/16] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c95e3412da..49938c9ed4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -206,6 +202,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -221,9 +219,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v13-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240403225759.sq5hmxdmbalroosy@liskov/9-v13-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From 39c219ddccb054e5fbcd3cec9e4f3356847af8bd Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v13 08/16] Remove table_scan_bitmap_next_tuple parameter
tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 48b8359786..265f33f000 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2324,7 +2324,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 49938c9ed4..282dcb9791 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -286,7 +286,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 75c5ee956d..1b545f2b67 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -780,10 +780,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -814,15 +811,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2008,7 +2000,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2020,7 +2011,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v13-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (23.0K, ../../20240403225759.sq5hmxdmbalroosy@liskov/10-v13-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 5ff565b545d1ff1e08aaa9c610fff3d8c7d0d7b5 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v13 09/16] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 198 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 149 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 265f33f000..8a9d2a9c34 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,18 +2188,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2218,16 +2251,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2319,7 +2343,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 282dcb9791..90cb10bc81 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +109,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +161,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -202,55 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid, lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -272,45 +232,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno < node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,13 +311,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -356,14 +335,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -388,7 +370,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -466,6 +451,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -523,6 +509,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -584,12 +572,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -597,13 +581,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -634,28 +618,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -688,8 +668,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -697,9 +675,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1b545f2b67..74ddd5d66d 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -22,6 +22,7 @@
#include "access/xact.h"
#include "commands/vacuum.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -773,19 +774,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -804,8 +800,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -942,9 +938,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -991,6 +991,21 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1001,6 +1016,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1964,19 +1994,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1986,8 +2014,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 4880f346bf..8e34415567 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1792,8 +1792,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1802,9 +1800,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1812,8 +1812,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1822,9 +1820,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v13-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.0K, ../../20240403225759.sq5hmxdmbalroosy@liskov/11-v13-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From 89f594870e40601de33778f7157c8ee3c1b7b370 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v13 10/16] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext(). Now, on the first scan, the the
iterator is created after the scan descriptor is created.
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 163 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 29 +---
src/include/executor/nodeBitmapHeapscan.h | 10 ++
src/include/nodes/execnodes.h | 8 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 8a9d2a9c34..b2e52e3fc1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2207,10 +2207,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 90cb10bc81..8b6f22bc3b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,6 +56,56 @@ static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ dsa_pointer shared_area,
+ dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(shared_area))
+ result->parallel = tbm_attach_shared_iterate(personal_area, shared_area);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -97,43 +147,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
+ if (init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -154,21 +184,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +216,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ dsa);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ node->pf_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
+ dsa);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -280,7 +311,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ node->pf_iterator &&
node->pfblockno < node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -321,12 +352,11 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
@@ -335,7 +365,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -348,8 +378,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -371,7 +399,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -431,23 +459,22 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (prefetch_iterator)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
node->prefetch_pages++;
@@ -475,8 +502,6 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
if (prefetch_iterator)
{
while (1)
@@ -500,12 +525,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
@@ -572,18 +597,17 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator)
+ {
+ bhs_end_iterate(node->pf_iterator);
+ node->pf_iterator = NULL;
+ }
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -628,12 +652,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (node->pf_iterator)
+ bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -671,11 +693,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
+ scanstate->pf_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..fb22f305bf 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,12 +20,12 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
+struct BitmapHeapIterator;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -44,8 +44,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ struct BitmapHeapIterator *rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 74ddd5d66d..d4fbf0d889 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -942,8 +942,7 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
+ result->rs_bhs_iterator = NULL;
return result;
}
@@ -993,17 +992,8 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1018,17 +1008,8 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caa..cb56d20dc6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,5 +28,15 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8e34415567..cf8b4995f0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1787,6 +1787,8 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+struct BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1795,12 +1797,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1815,12 +1816,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8d08386d65..825454c90c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v13-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.2K, ../../20240403225759.sq5hmxdmbalroosy@liskov/12-v13-0011-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From 63f90ed6dcac0c65c56b292a8c78ad87561c1550 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v13 11/16] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index b2e52e3fc1..b08837efd0 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,7 +2188,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2338,7 +2339,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 8b6f22bc3b..fb79f57d7a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -119,7 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -298,14 +297,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index d4fbf0d889..70e538b76b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -782,8 +782,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -800,8 +800,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1977,15 +1979,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1996,7 +2000,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v13-0012-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../20240403225759.sq5hmxdmbalroosy@liskov/13-v13-0012-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From 27eb60345e14bba33420198c55bf3c2041592097 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v13 12/16] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fc..1dc4c99bf9 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639b..432fae5296 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.40.1
[text/x-diff] v13-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch (20.7K, ../../20240403225759.sq5hmxdmbalroosy@liskov/14-v13-0013-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch)
download | inline diff:
From 05f7e86cb1276a6141d0213d40bcd7252e8af31e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v13 13/16] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 30 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++-------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/executor/nodeBitmapHeapscan.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
8 files changed, 113 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb..3aa457a29e 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544..033d525339 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index b08837efd0..d9ceb4b848 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2196,7 +2196,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2208,9 +2208,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(scan->rs_bhs_iterator);
+ bhs_iterate(scan->rs_bhs_iterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2225,11 +2225,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2237,19 +2237,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2278,7 +2278,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2287,9 +2287,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2339,7 +2339,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index fb79f57d7a..d61965a276 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -77,15 +77,16 @@ bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_ar
return result;
}
-TBMIterateResult *
-bhs_iterate(BitmapHeapIterator *iterator)
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result)
{
Assert(iterator);
+ Assert(result);
if (iterator->serial)
- return tbm_iterate(iterator->serial);
+ tbm_iterate(iterator->serial, result);
else
- return tbm_shared_iterate(iterator->parallel);
+ tbm_shared_iterate(iterator->parallel, result);
}
void
@@ -348,7 +349,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -360,8 +361,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -394,8 +395,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -462,10 +463,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -473,7 +476,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -482,13 +485,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -501,7 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -520,8 +523,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -529,17 +532,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf9..309a44bdb8 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae..3b432263bb 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index cb56d20dc6..3c330f86e6 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -34,7 +34,7 @@ typedef struct BitmapHeapIterator
struct TBMSharedIterator *parallel;
} BitmapHeapIterator;
-extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result);
extern void bhs_end_iterate(BitmapHeapIterator *iterator);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae5296..f000c1af28 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.40.1
[text/x-diff] v13-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (31.5K, ../../20240403225759.sq5hmxdmbalroosy@liskov/15-v13-0014-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From e1a29efe65346205cc324178d72b01c8c9325d12 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v13 14/16] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 262 +++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 341 ++--------------------
src/include/access/heapam.h | 17 ++
src/include/access/relscan.h | 8 +
src/include/access/tableam.h | 26 +-
src/include/nodes/execnodes.h | 14 -
7 files changed, 355 insertions(+), 339 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 50b4bc1475..263c728543 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -948,8 +948,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1032,6 +1040,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1044,6 +1058,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1069,6 +1089,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index d9ceb4b848..42d5b749de 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -60,6 +60,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2186,6 +2189,73 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2204,6 +2274,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2344,6 +2416,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->rs_pf_bhs_iterator &&
+ hscan->pfblockno < hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2354,6 +2438,154 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+
+ if (pstate == NULL)
+ {
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2379,6 +2611,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d61965a276..187b288e68 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
dsa_pointer shared_area,
@@ -122,7 +118,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -142,7 +137,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
@@ -154,7 +149,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -169,23 +173,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
@@ -216,19 +213,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
- node->pf_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
- dsa);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
+ scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
+ scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
+ dsa);
}
#endif /* USE_PREFETCH */
@@ -243,36 +240,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -296,23 +263,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->pf_iterator &&
- node->pfblockno < node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -336,219 +289,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -594,22 +334,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator)
- {
- bhs_end_iterate(node->pf_iterator);
- node->pf_iterator = NULL;
- }
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -648,14 +378,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->pf_iterator)
- bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -688,17 +414,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->pf_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -738,13 +458,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -828,7 +541,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index a527e1b99f..bf212f577f 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -86,6 +86,23 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used in serial BHS
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fb22f305bf..7938b741d6 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@
struct ParallelTableScanDescData;
struct BitmapHeapIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,6 +46,13 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
+ struct BitmapHeapIterator *rs_pf_bhs_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+ /* used to validate BHS prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 70e538b76b..a2b229fb87 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -785,17 +785,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -945,6 +934,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
+ result->rs_pf_bhs_iterator = NULL;
+ result->prefetch_maximum = 0;
+ result->bm_parallel = NULL;
return result;
}
@@ -996,6 +988,12 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1012,6 +1010,12 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cf8b4995f0..592215d5ee 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,18 +1794,11 @@ struct BitmapHeapIterator;
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1813,18 +1806,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v13-0015-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../20240403225759.sq5hmxdmbalroosy@liskov/16-v13-0015-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 8b0607dbdfbc6d9f05153f9ada673cd11526c9aa Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v13 15/16] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 263c728543..92f050272b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -319,6 +319,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 42d5b749de..a04774c7dd 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2183,12 +2183,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2222,8 +2216,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2586,30 +2580,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -2990,7 +2997,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index 55b8caeadf..0b87530a9c 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -90,8 +90,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 187b288e68..2f9387e51a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -230,44 +230,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
#endif /* USE_PREFETCH */
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a2b229fb87..79689ec377 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -773,36 +773,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -1981,44 +1965,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2029,7 +1982,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v13-0016-BitmapHeapScan-uses-read-stream-API.patch (26.2K, ../../20240403225759.sq5hmxdmbalroosy@liskov/17-v13-0016-BitmapHeapScan-uses-read-stream-API.patch)
download | inline diff:
From b44830bbc4e60ea867b09fa009e2e476e989773f Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v13 16/16] BitmapHeapScan uses read stream API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the read stream API prefetching. Heap table AM implements a read
stream callback which uses the iterator to get the next valid block that
needs to be fetched for the read stream API.
---
src/backend/access/heap/heapam.c | 95 ++++--
src/backend/access/heap/heapam_handler.c | 347 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 43 +--
src/include/access/heapam.h | 21 +-
src/include/access/relscan.h | 6 -
src/include/access/tableam.h | 14 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 115 insertions(+), 420 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 92f050272b..4a74398e54 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -108,6 +108,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_stream_read_next(ReadStream *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -950,16 +952,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1014,6 +1008,23 @@ heap_beginscan(Relation relation, Snapshot snapshot,
initscan(scan, key, false);
+ /*
+ * Make the read stream object for BitmapHeapScan. BitmapHeapScan does not
+ * use a BufferAccessStrategy, so we could do this before initscan(), but
+ * wait until after so the scan descriptor is fully set up. On rescan, the
+ * stream will be reset.
+ */
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ scan->rs_read_stream = read_stream_begin_relation(READ_STREAM_DEFAULT,
+ scan->rs_strategy,
+ scan->rs_base.rs_rd,
+ MAIN_FORKNUM,
+ bitmapheap_stream_read_next,
+ scan,
+ sizeof(TBMIterateResult));
+ }
+
return (TableScanDesc) scan;
}
@@ -1042,12 +1053,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1060,11 +1065,8 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_read_stream)
+ read_stream_reset(scan->rs_read_stream);
/*
* reinitialize scan descriptor
@@ -1091,11 +1093,8 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
/*
* decrement relation reference count and free scan descriptor storage
@@ -10111,3 +10110,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_stream_read_next(ReadStream *pgsr, void *private_data,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) private_data;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_bhs_iterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index a04774c7dd..77ce10327a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -60,9 +60,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2183,147 +2180,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *per_buffer_data;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_read_stream);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_bhs_iterator, &tbmres);
+ hscan->rs_cbuf = read_stream_next_buffer(hscan->rs_read_stream, &per_buffer_data);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(per_buffer_data);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = per_buffer_data;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2344,7 +2262,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2353,9 +2271,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2405,23 +2323,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_bhs_iterator &&
- hscan->pfblockno < hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2432,153 +2338,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2613,41 +2372,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2f9387e51a..f2662ea542 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -131,14 +131,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -149,15 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,13 +157,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -213,22 +190,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
extra_flags);
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- {
- scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
- scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
- dsa);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
}
@@ -525,14 +492,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -563,11 +526,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index bf212f577f..deccd5c237 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/read_stream.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -76,6 +77,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ ReadStream *rs_read_stream;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -86,23 +90,6 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /*
- * These fields only used for prefetching in bitmap table scans
- */
-
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used in serial BHS
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
-
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 7938b741d6..02893bf99b 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -46,13 +46,7 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
- struct BitmapHeapIterator *rs_pf_bhs_iterator;
-
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate BHS prefetch and current block stay in sync */
- BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 79689ec377..aead772e6f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -918,8 +918,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
- result->rs_pf_bhs_iterator = NULL;
- result->prefetch_maximum = 0;
result->bm_parallel = NULL;
return result;
}
@@ -972,12 +970,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -994,12 +986,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 592215d5ee..8d86b900a0 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1767,11 +1767,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1779,10 +1775,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.40.1
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-31 15:45 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-03 22:57 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
@ 2024-04-04 14:35 ` Tomas Vondra <[email protected]>
2024-04-05 08:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Tomas Vondra @ 2024-04-04 14:35 UTC (permalink / raw)
To: Melanie Plageman <[email protected]>; +Cc: Thomas Munro <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On 4/4/24 00:57, Melanie Plageman wrote:
> On Sun, Mar 31, 2024 at 11:45:51AM -0400, Melanie Plageman wrote:
>> On Fri, Mar 29, 2024 at 12:05:15PM +0100, Tomas Vondra wrote:
>>>
>>>
>>> On 3/29/24 02:12, Thomas Munro wrote:
>>>> On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
>>>> <[email protected]> wrote:
>>>>> I think there's some sort of bug, triggering this assert in heapam
>>>>>
>>>>> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
>>>>
>>>> Thanks for the repro. I can't seem to reproduce it (still trying) but
>>>> I assume this is with Melanie's v11 patch set which had
>>>> v11-0016-v10-Read-Stream-API.patch.
>>>>
>>>> Would you mind removing that commit and instead applying the v13
>>>> stream_read.c patches[1]? v10 stream_read.c was a little confused
>>>> about random I/O combining, which I fixed with a small adjustment to
>>>> the conditions for the "if" statement right at the end of
>>>> read_stream_look_ahead(). Sorry about that. The fixed version, with
>>>> eic=4, with your test query using WHERE a < a, ends its scan with:
>>>>
>>>
>>> I'll give that a try. Unfortunately unfortunately the v11 still has the
>>> problem I reported about a week ago:
>>>
>>> ERROR: prefetch and main iterators are out of sync
>>>
>>> So I can't run the full benchmarks :-( but master vs. streaming read API
>>> should work, I think.
>>
>> Odd, I didn't notice you reporting this ERROR popping up. Now that I
>> take a look, v11 (at least, maybe also v10) had this very sill mistake:
>>
>> if (scan->bm_parallel == NULL &&
>> scan->rs_pf_bhs_iterator &&
>> hscan->pfblockno > hscan->rs_base.blockno)
>> elog(ERROR, "prefetch and main iterators are out of sync");
>>
>> It errors out if the prefetch block is ahead of the current block --
>> which is the opposite of what we want. I've fixed this in attached v12.
>>
>> This version also has v13 of the streaming read API. I noticed one
>> mistake in my bitmapheap scan streaming read user -- it freed the
>> streaming read object at the wrong time. I don't know if this was
>> causing any other issues, but it at least is fixed in this version.
>
> Attached v13 is rebased over master (which includes the streaming read
> API now). I also reset the streaming read object on rescan instead of
> creating a new one each time.
>
> I don't know how much chance any of this has of going in to 17 now, but
> I thought I would start looking into the regression repro Tomas provided
> in [1].
>
My personal opinion is that we should try to get in as many of the the
refactoring patches as possible, but I think it's probably too late for
the actual switch to the streaming API.
If someone else feels like committing that part, I won't stand in the
way, but I'm not quite convinced it won't cause regressions. Maybe it's
OK but I'd need more time to do more tests, collect data, and so on. And
I don't think we have that, especially considering we'd still need to
commit the other parts first.
> I'm also not sure if I should try and group the commits into fewer
> commits now or wait until I have some idea of whether or not the
> approach in 0013 and 0014 is worth pursuing.
>
You mean whether to pursue the approach in general, or for v17? I think
it looks like the right approach, but for v17 see above :-(
As for merging, I wouldn't do that. I looked at the commits and while
some of them seem somewhat "trivial", I really like how you organized
the commits, and kept those that just "move" code around, and those that
actually change stuff. It's much easier to understand, IMO.
I went through the first ~10 commits, and added some review - either as
a separate commit, when possible, in the code as XXX comment, and also
in the commit message. The code tweaks are utterly trivial (whitespace
or indentation to make the line shorter). It shouldn't take much time to
deal with those, I think.
I think the main focus should be updating the commit messages. If it was
only a single patch, I'd probably try to write the messages myself, but
with this many patches it'd be great if you could update those and I'll
review that before commit.
I always struggle with writing commit messages myself, and it takes me
ages to write a good one (well, I think the message is good, but who
knows ...). But I think a good message should be concise enough to
explain what and why it's done. It may reference a thread for all the
gory details, but the basic reasoning should be in the commit message.
For example the message for "BitmapPrefetch use prefetch block recheck
for skip fetch" now says that it "makes more sense to do X" but does not
really say why that's the case. The linked message does, but it'd be
good to have that in the message (because how would I know how much of
the thread to read?).
Also, it'd be very helpful if you could update the author & reviewed-by
fields. I'll review those before commit, ofc, but I admit I lost track
of who reviewed which part.
I'd focus on the first ~8-9 commits or so for now, we can commit more if
things go reasonably well.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[text/x-patch] v14-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.8K, ../../[email protected]/2-v14-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From 8d42c0e61cda5f7a8b574fc6a775bec84370ba26 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v14 01/19] BitmapHeapScan begin scan after bitmap creation
There is no reason for a BitmapHeapScan to begin the scan of the
underlying table in ExecInitBitmapHeapScan(). Instead, do so after
completing the index scan and building the bitmap.
---
src/backend/access/table/tableam.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 26 +++++++++++++++++------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index 805d222cebc..b0f61c65f36 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,7 +120,6 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
-
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aabe..93fdcd226bf 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,20 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+ }
+
node->initialized = true;
}
@@ -601,7 +615,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +693,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +800,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.44.0
[text/x-patch] v14-0002-review.patch (1.9K, ../../[email protected]/3-v14-0002-review.patch)
download | inline diff:
From d2160a9958f8185ab4803e9e778b370a6a6dde32 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 15:15:18 +0200
Subject: [PATCH v14 02/19] review
so is this an optimization, bugfix, or what? what difference does
it make in practice? or is it just cosmetic change?
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/table/tableam.c | 1 +
src/backend/executor/nodeBitmapHeapscan.c | 11 ++++++-----
2 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/table/tableam.c b/src/backend/access/table/tableam.c
index b0f61c65f36..805d222cebc 100644
--- a/src/backend/access/table/tableam.c
+++ b/src/backend/access/table/tableam.c
@@ -120,6 +120,7 @@ table_beginscan_catalog(Relation relation, int nkeys, struct ScanKeyData *key)
NULL, flags);
}
+
/* ----------------------------------------------------------------------------
* Parallel table scan related functions.
* ----------------------------------------------------------------------------
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 93fdcd226bf..c8c466e3c5c 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -185,11 +185,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
- scan = node->ss.ss_currentScanDesc = table_beginscan_bm(
- node->ss.ss_currentRelation,
- node->ss.ps.state->es_snapshot,
- 0,
- NULL);
+ scan = table_beginscan_bm(node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+
+ node->ss.ss_currentScanDesc = scan;
}
node->initialized = true;
--
2.44.0
[text/x-patch] v14-0003-BitmapHeapScan-set-can_skip_fetch-later.patch (2.4K, ../../[email protected]/4-v14-0003-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From ee72a2b5c846408ba9549d9d4c8b6d419fe3c311 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v14 03/19] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() when
!BitmapHeapScanState->initialized instead of in
ExecInitBitmapHeapScan(). This is a preliminary step to removing
can_skip_fetch from BitmapHeapScanState and setting it in table AM
specific code.
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c8c466e3c5c..2148a21531a 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -743,16 +753,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.44.0
[text/x-patch] v14-0004-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (15.1K, ../../[email protected]/5-v14-0004-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From ee62f42859a69cab1039155e2f15d0239a3f3dc1 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 15:34:25 +0200
Subject: [PATCH v14 04/19] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block and implement it for the heap
table AM by moving the vmbuffer and other fields needed for the
optimization from the BitmapHeapScanState into the HeapScanDescData.
heapam_scan_bitmap_next_block() now decides whether or not to skip
fetching the block before reading it in and
heapam_scan_bitmap_next_tuple() returns NULL-filled tuples for skipped
blocks.
The layering violation is still present in BitmapHeapScans's prefetching
code. However, this will be eliminated when prefetching is implemented
using the upcoming streaming read API discussed in [1].
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index dada2ecd1e3..10f2faaa60b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -967,6 +967,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1055,6 +1057,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1074,6 +1082,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3e7a6b5548b..929b2cf8e71 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2198,6 +2199,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2310,6 +2329,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2148a21531a..a8bc5dec53d 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,10 +185,24 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
node->ss.ss_currentScanDesc = scan;
}
@@ -208,7 +212,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -229,37 +233,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -302,52 +283,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -519,7 +481,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -570,7 +532,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -640,8 +602,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -651,7 +611,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -696,8 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -741,8 +698,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -753,7 +708,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2765efc4e5e..750ea30852e 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -76,6 +76,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e7eeb754098..55f11397576 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -63,6 +63,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -937,9 +944,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e7ff8e4992f..4880f346bf1 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,10 +1794,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1817,9 +1814,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.44.0
[text/x-patch] v14-0005-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.5K, ../../[email protected]/6-v14-0005-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From e3fa2f25b5631bb9a5e27515be8161ef89418c40 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v14 05/19] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. It makes more sense for it to use the
recheck flag from the TBMIterateResult for the prefetch block instead.
See this [1] thread on hackers reporting the issue.
XXX I think this commit message should "distill" the reasoning from [1]
to justify why this "makes more sense". The details can be left to the
thread, but the basics should be in this message I think.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a8bc5dec53d..2e2cec8b3b5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -475,14 +475,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -533,7 +528,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.44.0
[text/x-patch] v14-0006-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.4K, ../../[email protected]/7-v14-0006-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 9b66215da19d8b6dc1e135204d9b427eb47c99d5 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v14 06/19] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
Author: Melanie Plageman
Reviewed-by: ???????
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2e2cec8b3b5..795a893035d 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -231,7 +231,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -342,7 +342,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -361,7 +361,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.44.0
[text/x-patch] v14-0007-table_scan_bitmap_next_block-returns-lossy-or-ex.patch (4.4K, ../../[email protected]/8-v14-0007-table_scan_bitmap_next_block-returns-lossy-or-ex.patch)
download | inline diff:
From 742490b1d4ef3b41eaece6b9fc0475d52b60ca39 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v14 07/19] table_scan_bitmap_next_block() returns lossy or
exact
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So, the table AM must inform
BitmapHeapNext() whether or not the current block is lossy or exact for
the purposes of the counters used in EXPLAIN.
---
src/backend/access/heap/heapam_handler.c | 5 ++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 +++++-----
src/include/access/tableam.h | 14 ++++++++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 929b2cf8e71..a511bbfa88d 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,7 +2188,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ bool *lossy)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2316,6 +2317,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ *lossy = tbmres->ntuples < 0;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 795a893035d..56943b211fa 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -212,7 +212,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
+ bool valid, lossy;
CHECK_FOR_INTERRUPTS();
@@ -233,12 +233,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
+ if (lossy)
node->lossy_pages++;
+ else
+ node->exact_pages++;
if (!valid)
{
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 55f11397576..75c5ee956db 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -789,6 +789,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy indicates whether or not the block's representation in the bitmap
+ * is lossy or exact.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -804,7 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ bool *lossy);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1971,14 +1975,16 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
* Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
* a bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy is set to true if bitmap is lossy for the
+ * selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ bool *lossy)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1989,7 +1995,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres, lossy);
}
/*
--
2.44.0
[text/x-patch] v14-0008-review.patch (841B, ../../[email protected]/9-v14-0008-review.patch)
download | inline diff:
From ae9308ef69e04e1231bfd718c59482320cc9dde3 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 15:51:03 +0200
Subject: [PATCH v14 08/19] review
I wonder if returning "exact" instead of lossy would be better, for the
silly reason that the if branching in BitmapHeapNext would be the same.
---
src/backend/executor/nodeBitmapHeapscan.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 56943b211fa..9b2034c2f83 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -212,7 +212,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid, lossy;
+ bool valid,
+ lossy;
CHECK_FOR_INTERRUPTS();
--
2.44.0
[text/x-patch] v14-0009-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (2.9K, ../../[email protected]/10-v14-0009-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From 597254156f86cb57bb0340b68d52e8a3aa7d981e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v14 09/19] Reduce scope of BitmapHeapScan tbmiterator local
variables
To simplify the diff of a future commit which will move the TBMIterators
into the scan descriptor, define them in a narrower scope now.
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 9b2034c2f83..cf778f61d52 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -207,6 +203,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -223,9 +221,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.44.0
[text/x-patch] v14-0010-review.patch (838B, ../../[email protected]/11-v14-0010-review.patch)
download | inline diff:
From 7e4e7dfcc334198b83ccc86d15dc6aa6f8189a36 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 15:58:57 +0200
Subject: [PATCH v14 10/19] review
---
src/backend/executor/nodeBitmapHeapscan.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cf778f61d52..6787547b9ff 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -99,6 +99,10 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * XXX why don't we assign directly to the node-> fields? Then we
+ * would not need these local variables at all.
+ */
TBMIterator *tbmiterator = NULL;
TBMSharedIterator *shared_tbmiterator = NULL;
--
2.44.0
[text/x-patch] v14-0011-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.4K, ../../[email protected]/12-v14-0011-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From 3fb2c24d79f8cdc8e33007697db362ad5b50b4fc Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v14 11/19] Remove table_scan_bitmap_next_tuple parameter
tbmres
With the addition of the proposed streaming read API [1],
table_scan_bitmap_next_block() will no longer take a TBMIterateResult as
an input. Instead table AMs will be responsible for implementing a
callback for the streaming read API which specifies which blocks should
be prefetched and read.
Thus, it no longer makes sense to use the TBMIterateResult as a means of
communication between table_scan_bitmap_next_tuple() and
table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
XXX Does it make sense to merge this without the switch to the new API?
Imagine an AM chose to do the work in table_scan_bitmap_next_tuple. Now
it would have to do that in the other callback. Is that OK, or would it
make it harder/less efficient? Ultimately the AM will have to do that
anyway, so maybe it doesn't mattter.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index a511bbfa88d..02bab2e93f4 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2324,7 +2324,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 6787547b9ff..14a5af30087 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -292,7 +292,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 75c5ee956db..1b545f2b678 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -780,10 +780,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -814,15 +811,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2008,7 +2000,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2020,7 +2011,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.44.0
[text/x-patch] v14-0012-Make-table_scan_bitmap_next_block-async-friendly.patch (23.0K, ../../[email protected]/13-v14-0012-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 7dff0fdde3016536244a78efe0803ddcb9fdb8ee Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 16:05:04 +0200
Subject: [PATCH v14 12/19] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to
the table AM.
These changes will enable bitmapheapscan to use the future streaming
read API [1]. Table AMs will implement a streaming read API callback
returning the next block to fetch. In heap AM's case, the callback will
use the iterator to identify the next block to fetch. Since choosing the
next block will no longer the responsibility of BitmapHeapNext(), the
streaming read control flow requires these changes to
table_scan_bitmap_next_block().
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 59 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 199 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 195 insertions(+), 150 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 02bab2e93f4..eea3b7f149e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,18 +2188,51 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2218,16 +2251,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2319,7 +2343,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*lossy = tbmres->ntuples < 0;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 14a5af30087..5f987b40ede 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -70,8 +69,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
+ bool lossy;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +82,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -115,7 +113,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -168,7 +165,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -207,56 +203,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- bool valid,
- lossy;
-
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- valid = table_scan_bitmap_next_block(scan, tbmres, &lossy);
-
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
- if (!valid)
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -278,45 +237,60 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ break;
+
+ if (lossy)
+ node->lossy_pages++;
+ else
+ node->exact_pages++;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno < node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -342,13 +316,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -362,14 +340,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -394,7 +375,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -472,6 +456,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -529,6 +514,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -590,12 +577,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -603,13 +586,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -640,28 +623,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -694,8 +673,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -703,9 +680,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304ab..92b829cebc7 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 1b545f2b678..74ddd5d66d3 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -22,6 +22,7 @@
#include "access/xact.h"
#include "commands/vacuum.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -773,19 +774,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy indicates whether or not the block's representation in the bitmap
* is lossy or exact.
*
@@ -804,8 +800,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy);
+ bool *recheck, bool *lossy,
+ BlockNumber *blockno);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -942,9 +938,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -991,6 +991,21 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1001,6 +1016,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1964,19 +1994,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy is set to true if bitmap is lossy for the
- * selected block and false otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy is set to true
+ * if bitmap is lossy for the selected block and false otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
- bool *lossy)
+ bool *recheck, bool *lossy, BlockNumber *blockno)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1986,8 +2014,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres, lossy);
+ return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
+ lossy, blockno);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 4880f346bf1..8e344155679 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1792,8 +1792,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1802,9 +1800,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1812,8 +1812,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1822,9 +1820,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.44.0
[text/x-patch] v14-0013-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.1K, ../../[email protected]/14-v14-0013-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From 53362e02e069747a413aa11cf25df918a94b1cc1 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 16:05:53 +0200
Subject: [PATCH v14 13/19] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext(). Now, on the first scan, the the
iterator is created after the scan descriptor is created.
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 163 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 29 +---
src/include/executor/nodeBitmapHeapscan.h | 10 ++
src/include/nodes/execnodes.h | 8 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 114 insertions(+), 109 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index eea3b7f149e..e086cbd2cfd 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2207,10 +2207,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 5f987b40ede..a1f9acc2cfb 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -56,6 +56,56 @@ static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
+ dsa_pointer shared_area,
+ dsa_area *personal_area);
+
+BitmapHeapIterator *
+bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_area)
+{
+ BitmapHeapIterator *result = palloc(sizeof(BitmapHeapIterator));
+
+ result->serial = NULL;
+ result->parallel = NULL;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(shared_area))
+ result->parallel = tbm_attach_shared_iterate(personal_area, shared_area);
+ else
+ result->serial = tbm_begin_iterate(tbm);
+
+ return result;
+}
+
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ {
+ tbm_end_iterate(iterator->serial);
+ iterator->serial = NULL;
+ }
+ else
+ {
+ tbm_end_shared_iterate(iterator->parallel);
+ iterator->parallel = NULL;
+ }
+
+ pfree(iterator);
+}
/* ----------------------------------------------------------------
@@ -98,46 +148,22 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (!node->initialized)
{
/*
- * XXX why don't we assign directly to the node-> fields? Then we
- * would not need these local variables at all.
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
*/
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
+ if (init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -158,21 +184,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -203,8 +217,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ dsa);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ node->pf_iterator = bhs_begin_iterate(tbm,
+ pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
+ dsa);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -285,7 +312,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ node->pf_iterator &&
node->pfblockno < node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -326,12 +353,11 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
@@ -340,7 +366,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -353,8 +379,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -376,7 +400,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -436,23 +460,22 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (prefetch_iterator)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
node->prefetch_pages++;
@@ -480,8 +503,6 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
if (prefetch_iterator)
{
while (1)
@@ -505,12 +526,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
+ node->pf_iterator = NULL;
break;
}
@@ -577,18 +598,17 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator)
+ {
+ bhs_end_iterate(node->pf_iterator);
+ node->pf_iterator = NULL;
+ }
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -633,12 +653,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (node->pf_iterator)
+ bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -676,11 +694,10 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
+ scanstate->pf_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc7..fb22f305bf6 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,12 +20,12 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
+struct BitmapHeapIterator;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -44,8 +44,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ struct BitmapHeapIterator *rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 74ddd5d66d3..d4fbf0d889b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -942,8 +942,7 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
+ result->rs_bhs_iterator = NULL;
return result;
}
@@ -993,17 +992,8 @@ table_endscan(TableScanDesc scan)
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1018,17 +1008,8 @@ table_rescan(TableScanDesc scan,
{
if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
{
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
+ bhs_end_iterate(scan->rs_bhs_iterator);
+ scan->rs_bhs_iterator = NULL;
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caae..cb56d20dc6f 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,5 +28,15 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+} BitmapHeapIterator;
+
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 8e344155679..cf8b4995f0d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1787,6 +1787,8 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+struct BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1795,12 +1797,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1815,12 +1816,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b98e330e3ed..bcee567b65b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.44.0
[text/x-patch] v14-0014-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.2K, ../../[email protected]/15-v14-0014-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From f1290c0c5127abc0755a5ead9d1d77549bd01cc7 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 17:09:12 -0400
Subject: [PATCH v14 14/19] table_scan_bitmap_next_block counts lossy and exact
pages
Now that the table_scan_bitmap_next_block() callback only returns false
when the bitmap is exhausted, it is simpler to move the management of
the lossy and exact page counters into it. We will eventually remove
this callback and table_scan_bitmap_next_tuple() will update those
counters when a new block is read in.
---
src/backend/access/heap/heapam_handler.c | 8 ++++++--
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
src/include/access/tableam.h | 21 +++++++++++++--------
3 files changed, 21 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e086cbd2cfd..fe31b0efee4 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,7 +2188,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block;
@@ -2338,7 +2339,10 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- *lossy = tbmres->ntuples < 0;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
/*
* Return true to indicate that a valid block was found and the bitmap is
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a1f9acc2cfb..261e00849a7 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -119,7 +119,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- bool lossy;
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -299,14 +298,10 @@ new_page:
BitmapAdjustPrefetchIterator(node);
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &lossy, &node->blockno))
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ &node->lossy_pages, &node->exact_pages))
break;
- if (lossy)
- node->lossy_pages++;
- else
- node->exact_pages++;
-
/*
* If serial, we can error out if the the prefetch block doesn't stay
* ahead of the current block.
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index d4fbf0d889b..70e538b76ba 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -782,8 +782,8 @@ typedef struct TableAmRoutine
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * lossy indicates whether or not the block's representation in the bitmap
- * is lossy or exact.
+ * lossy_pages is incremented if the block's representation in the bitmap
+ * is lossy, otherwise, exact_pages is incremented.
*
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
@@ -800,8 +800,10 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck, bool *lossy,
- BlockNumber *blockno);
+ bool *recheck,
+ BlockNumber *blockno,
+ long *lossy_pages,
+ long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1977,15 +1979,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
/*
* Prepare to fetch / check / return tuples as part of a bitmap table scan.
* `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy is set to true
- * if bitmap is lossy for the selected block and false otherwise.
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented if bitmap is lossy for the selected block and exact_pages is
+ * incremented otherwise.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, bool *lossy, BlockNumber *blockno)
+ bool *recheck, BlockNumber *blockno,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1996,7 +2000,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- lossy, blockno);
+ blockno, lossy_pages,
+ exact_pages);
}
/*
--
2.44.0
[text/x-patch] v14-0015-Hard-code-TBMIterateResult-offsets-array-size.patch (5.4K, ../../[email protected]/16-v14-0015-Hard-code-TBMIterateResult-offsets-array-size.patch)
download | inline diff:
From 0e379ec9ce12ecc1fc1b9b0d93ea99a174a63966 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 20:13:43 -0500
Subject: [PATCH v14 15/19] Hard-code TBMIterateResult offsets array size
TIDBitmap's TBMIterateResult had a flexible sized array of tuple offsets
but the API always allocated MaxHeapTuplesPerPage OffsetNumbers.
Creating a fixed-size aray of size MaxHeapTuplesPerPage is more clear
for the API user.
---
src/backend/nodes/tidbitmap.c | 33 +++++++--------------------------
src/include/nodes/tidbitmap.h | 12 ++++++++++--
2 files changed, 17 insertions(+), 28 deletions(-)
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index e8ab5d78fcc..1dc4c99bf99 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -40,7 +40,6 @@
#include <limits.h>
-#include "access/htup_details.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "nodes/bitmapset.h"
@@ -48,14 +47,6 @@
#include "storage/lwlock.h"
#include "utils/dsa.h"
-/*
- * The maximum number of tuples per page is not large (typically 256 with
- * 8K pages, or 1024 with 32K pages). So there's not much point in making
- * the per-page bitmaps variable size. We just legislate that the size
- * is this:
- */
-#define MAX_TUPLES_PER_PAGE MaxHeapTuplesPerPage
-
/*
* When we have to switch over to lossy storage, we use a data structure
* with one bit per page, where all pages having the same number DIV
@@ -67,7 +58,7 @@
* table, using identical data structures. (This is because the memory
* management for hashtables doesn't easily/efficiently allow space to be
* transferred easily from one hashtable to another.) Therefore it's best
- * if PAGES_PER_CHUNK is the same as MAX_TUPLES_PER_PAGE, or at least not
+ * if PAGES_PER_CHUNK is the same as MaxHeapTuplesPerPage, or at least not
* too different. But we also want PAGES_PER_CHUNK to be a power of 2 to
* avoid expensive integer remainder operations. So, define it like this:
*/
@@ -79,7 +70,7 @@
#define BITNUM(x) ((x) % BITS_PER_BITMAPWORD)
/* number of active words for an exact page: */
-#define WORDS_PER_PAGE ((MAX_TUPLES_PER_PAGE - 1) / BITS_PER_BITMAPWORD + 1)
+#define WORDS_PER_PAGE ((MaxHeapTuplesPerPage - 1) / BITS_PER_BITMAPWORD + 1)
/* number of active words for a lossy chunk: */
#define WORDS_PER_CHUNK ((PAGES_PER_CHUNK - 1) / BITS_PER_BITMAPWORD + 1)
@@ -181,7 +172,7 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/*
@@ -222,7 +213,7 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output; /* MUST BE LAST (because variable-size) */
+ TBMIterateResult output;
};
/* Local function prototypes */
@@ -390,7 +381,7 @@ tbm_add_tuples(TIDBitmap *tbm, const ItemPointer tids, int ntids,
bitnum;
/* safety check to ensure we don't overrun bit array bounds */
- if (off < 1 || off > MAX_TUPLES_PER_PAGE)
+ if (off < 1 || off > MaxHeapTuplesPerPage)
elog(ERROR, "tuple offset out of range: %u", off);
/*
@@ -692,12 +683,7 @@ tbm_begin_iterate(TIDBitmap *tbm)
Assert(tbm->iterating != TBM_ITERATING_SHARED);
- /*
- * Create the TBMIterator struct, with enough trailing space to serve the
- * needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMIterator *) palloc(sizeof(TBMIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = palloc(sizeof(TBMIterator));
iterator->tbm = tbm;
/*
@@ -1463,12 +1449,7 @@ tbm_attach_shared_iterate(dsa_area *dsa, dsa_pointer dp)
TBMSharedIterator *iterator;
TBMSharedIteratorState *istate;
- /*
- * Create the TBMSharedIterator struct, with enough trailing space to
- * serve the needs of the TBMIterateResult sub-struct.
- */
- iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator) +
- MAX_TUPLES_PER_PAGE * sizeof(OffsetNumber));
+ iterator = (TBMSharedIterator *) palloc0(sizeof(TBMSharedIterator));
istate = (TBMSharedIteratorState *) dsa_get_address(dsa, dp);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 1945f0639bf..432fae52962 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -22,6 +22,7 @@
#ifndef TIDBITMAP_H
#define TIDBITMAP_H
+#include "access/htup_details.h"
#include "storage/itemptr.h"
#include "utils/dsa.h"
@@ -41,9 +42,16 @@ typedef struct TBMIterateResult
{
BlockNumber blockno; /* page number containing tuples */
int ntuples; /* -1 indicates lossy result */
- bool recheck; /* should the tuples be rechecked? */
/* Note: recheck is always true if ntuples < 0 */
- OffsetNumber offsets[FLEXIBLE_ARRAY_MEMBER];
+ bool recheck; /* should the tuples be rechecked? */
+
+ /*
+ * The maximum number of tuples per page is not large (typically 256 with
+ * 8K pages, or 1024 with 32K pages). So there's not much point in making
+ * the per-page bitmaps variable size. We just legislate that the size is
+ * this:
+ */
+ OffsetNumber offsets[MaxHeapTuplesPerPage];
} TBMIterateResult;
/* function prototypes in nodes/tidbitmap.c */
--
2.44.0
[text/x-patch] v14-0016-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch (20.7K, ../../[email protected]/17-v14-0016-Separate-TBM-Shared-Iterator-and-TBMIterateResul.patch)
download | inline diff:
From 99f4982bc9f861f080b2aa71caf92169b5c21334 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 15 Feb 2024 21:23:41 -0500
Subject: [PATCH v14 16/19] Separate TBM[Shared]Iterator and TBMIterateResult
Remove the TBMIterateResult from the TBMIterator and TBMSharedIterator
and have tbm_[shared_]iterate() take a TBMIterateResult as a parameter.
This will allow multiple TBMIterateResults to exist concurrently
allowing asynchronous use of the TIDBitmap for prefetching, for example.
tbm_[shared]_iterate() now sets blockno to InvalidBlockNumber when the
bitmap is exhausted instead of returning NULL.
BitmapHeapScan callers of tbm_iterate make a TBMIterateResult locally
and pass it in.
Because GIN only needs a single TBMIterateResult, inline the matchResult
in the GinScanEntry to avoid having to separately manage memory for the
TBMIterateResult.
---
src/backend/access/gin/ginget.c | 48 +++++++++------
src/backend/access/gin/ginscan.c | 2 +-
src/backend/access/heap/heapam_handler.c | 30 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 ++++++++-------
src/backend/nodes/tidbitmap.c | 73 ++++++++++++-----------
src/include/access/gin_private.h | 2 +-
src/include/executor/nodeBitmapHeapscan.h | 2 +-
src/include/nodes/tidbitmap.h | 4 +-
8 files changed, 113 insertions(+), 95 deletions(-)
diff --git a/src/backend/access/gin/ginget.c b/src/backend/access/gin/ginget.c
index 0b4f2ebadb6..3aa457a29e1 100644
--- a/src/backend/access/gin/ginget.c
+++ b/src/backend/access/gin/ginget.c
@@ -332,10 +332,22 @@ restartScanEntry:
entry->list = NULL;
entry->nlist = 0;
entry->matchBitmap = NULL;
- entry->matchResult = NULL;
entry->reduceResult = false;
entry->predictNumberResult = 0;
+ /*
+ * MTODO: is it enough to set blockno to InvalidBlockNumber? In all the
+ * places were we previously set matchResult to NULL, I just set blockno
+ * to InvalidBlockNumber. It seems like this should be okay because that
+ * is usually what we check before using the matchResult members. But it
+ * might be safer to zero out the offsets array. But that is expensive.
+ */
+ entry->matchResult.blockno = InvalidBlockNumber;
+ entry->matchResult.ntuples = 0;
+ entry->matchResult.recheck = true;
+ memset(entry->matchResult.offsets, 0,
+ sizeof(OffsetNumber) * MaxHeapTuplesPerPage);
+
/*
* we should find entry, and begin scan of posting tree or just store
* posting list in memory
@@ -374,6 +386,7 @@ restartScanEntry:
{
if (entry->matchIterator)
tbm_end_iterate(entry->matchIterator);
+ entry->matchResult.blockno = InvalidBlockNumber;
entry->matchIterator = NULL;
tbm_free(entry->matchBitmap);
entry->matchBitmap = NULL;
@@ -823,18 +836,19 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
{
/*
* If we've exhausted all items on this block, move to next block
- * in the bitmap.
+ * in the bitmap. tbm_iterate() sets matchResult->blockno to
+ * InvalidBlockNumber when the bitmap is exhausted.
*/
- while (entry->matchResult == NULL ||
- (entry->matchResult->ntuples >= 0 &&
- entry->offset >= entry->matchResult->ntuples) ||
- entry->matchResult->blockno < advancePastBlk ||
+ while ((!BlockNumberIsValid(entry->matchResult.blockno)) ||
+ (entry->matchResult.ntuples >= 0 &&
+ entry->offset >= entry->matchResult.ntuples) ||
+ entry->matchResult.blockno < advancePastBlk ||
(ItemPointerIsLossyPage(&advancePast) &&
- entry->matchResult->blockno == advancePastBlk))
+ entry->matchResult.blockno == advancePastBlk))
{
- entry->matchResult = tbm_iterate(entry->matchIterator);
+ tbm_iterate(entry->matchIterator, &entry->matchResult);
- if (entry->matchResult == NULL)
+ if (!BlockNumberIsValid(entry->matchResult.blockno))
{
ItemPointerSetInvalid(&entry->curItem);
tbm_end_iterate(entry->matchIterator);
@@ -858,10 +872,10 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* We're now on the first page after advancePast which has any
* items on it. If it's a lossy result, return that.
*/
- if (entry->matchResult->ntuples < 0)
+ if (entry->matchResult.ntuples < 0)
{
ItemPointerSetLossyPage(&entry->curItem,
- entry->matchResult->blockno);
+ entry->matchResult.blockno);
/*
* We might as well fall out of the loop; we could not
@@ -875,27 +889,27 @@ entryGetItem(GinState *ginstate, GinScanEntry entry,
* Not a lossy page. Skip over any offsets <= advancePast, and
* return that.
*/
- if (entry->matchResult->blockno == advancePastBlk)
+ if (entry->matchResult.blockno == advancePastBlk)
{
/*
* First, do a quick check against the last offset on the
* page. If that's > advancePast, so are all the other
* offsets, so just go back to the top to get the next page.
*/
- if (entry->matchResult->offsets[entry->matchResult->ntuples - 1] <= advancePastOff)
+ if (entry->matchResult.offsets[entry->matchResult.ntuples - 1] <= advancePastOff)
{
- entry->offset = entry->matchResult->ntuples;
+ entry->offset = entry->matchResult.ntuples;
continue;
}
/* Otherwise scan to find the first item > advancePast */
- while (entry->matchResult->offsets[entry->offset] <= advancePastOff)
+ while (entry->matchResult.offsets[entry->offset] <= advancePastOff)
entry->offset++;
}
ItemPointerSet(&entry->curItem,
- entry->matchResult->blockno,
- entry->matchResult->offsets[entry->offset]);
+ entry->matchResult.blockno,
+ entry->matchResult.offsets[entry->offset]);
entry->offset++;
/* Done unless we need to reduce the result */
diff --git a/src/backend/access/gin/ginscan.c b/src/backend/access/gin/ginscan.c
index af24d38544e..033d5253394 100644
--- a/src/backend/access/gin/ginscan.c
+++ b/src/backend/access/gin/ginscan.c
@@ -106,7 +106,7 @@ ginFillScanEntry(GinScanOpaque so, OffsetNumber attnum,
ItemPointerSetMin(&scanEntry->curItem);
scanEntry->matchBitmap = NULL;
scanEntry->matchIterator = NULL;
- scanEntry->matchResult = NULL;
+ scanEntry->matchResult.blockno = InvalidBlockNumber;
scanEntry->list = NULL;
scanEntry->nlist = 0;
scanEntry->offset = InvalidOffsetNumber;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index fe31b0efee4..ec16d8c5e3c 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2196,7 +2196,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult *tbmres;
+ TBMIterateResult tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
@@ -2208,9 +2208,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(scan->rs_bhs_iterator);
+ bhs_iterate(scan->rs_bhs_iterator, &tbmres);
- if (tbmres == NULL)
+ if (!BlockNumberIsValid(tbmres.blockno))
{
/* no more entries in the bitmap */
Assert(hscan->rs_empty_tuples_pending == 0);
@@ -2225,11 +2225,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* isolation though, as we need to examine all invisible tuples
* reachable by the index.
*/
- } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+ } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
- *recheck = tbmres->recheck;
+ *blockno = tbmres.blockno;
+ *recheck = tbmres.recheck;
/*
* We can skip fetching the heap page if we don't need any fields from the
@@ -2237,19 +2237,19 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* page are visible to our transaction.
*/
if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ !tbmres.recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
{
/* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
+ Assert(tbmres.ntuples >= 0);
Assert(hscan->rs_empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ hscan->rs_empty_tuples_pending += tbmres.ntuples;
return true;
}
- block = tbmres->blockno;
+ block = tbmres.blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2278,7 +2278,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres->ntuples >= 0)
+ if (tbmres.ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2287,9 +2287,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres->ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres.ntuples; curslot++)
{
- OffsetNumber offnum = tbmres->offsets[curslot];
+ OffsetNumber offnum = tbmres.offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2339,7 +2339,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres->ntuples < 0)
+ if (tbmres.ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 261e00849a7..1086e20234b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -77,15 +77,16 @@ bhs_begin_iterate(TIDBitmap *tbm, dsa_pointer shared_area, dsa_area *personal_ar
return result;
}
-TBMIterateResult *
-bhs_iterate(BitmapHeapIterator *iterator)
+void
+bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result)
{
Assert(iterator);
+ Assert(result);
if (iterator->serial)
- return tbm_iterate(iterator->serial);
+ tbm_iterate(iterator->serial, result);
else
- return tbm_shared_iterate(iterator->parallel);
+ tbm_shared_iterate(iterator->parallel, result);
}
void
@@ -349,7 +350,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
if (pstate == NULL)
{
@@ -361,8 +362,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
return;
}
@@ -395,8 +396,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (prefetch_iterator)
{
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ node->pfblockno = tbmpre.blockno;
}
}
}
@@ -463,10 +464,12 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ TBMIterateResult tbmpre;
bool skip_fetch;
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -474,7 +477,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -483,13 +486,13 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* prefetch_pages?)
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
@@ -502,7 +505,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
while (1)
{
- TBMIterateResult *tbmpre;
+ TBMIterateResult tbmpre;
bool do_prefetch = false;
bool skip_fetch;
@@ -521,8 +524,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
{
/* No more pages to prefetch */
bhs_end_iterate(prefetch_iterator);
@@ -530,17 +533,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ node->pfblockno = tbmpre.blockno;
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
+ !tbmpre.recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
+ tbmpre.blockno,
&node->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
}
}
}
diff --git a/src/backend/nodes/tidbitmap.c b/src/backend/nodes/tidbitmap.c
index 1dc4c99bf99..309a44bdb84 100644
--- a/src/backend/nodes/tidbitmap.c
+++ b/src/backend/nodes/tidbitmap.c
@@ -172,7 +172,6 @@ struct TBMIterator
int spageptr; /* next spages index */
int schunkptr; /* next schunks index */
int schunkbit; /* next bit to check in current schunk */
- TBMIterateResult output;
};
/*
@@ -213,7 +212,6 @@ struct TBMSharedIterator
PTEntryArray *ptbase; /* pagetable element array */
PTIterationArray *ptpages; /* sorted exact page index list */
PTIterationArray *ptchunks; /* sorted lossy page index list */
- TBMIterateResult output;
};
/* Local function prototypes */
@@ -944,20 +942,21 @@ tbm_advance_schunkbit(PagetableEntry *chunk, int *schunkbitp)
/*
* tbm_iterate - scan through next page of a TIDBitmap
*
- * Returns a TBMIterateResult representing one page, or NULL if there are
- * no more pages to scan. Pages are guaranteed to be delivered in numerical
- * order. If result->ntuples < 0, then the bitmap is "lossy" and failed to
- * remember the exact tuples to look at on this page --- the caller must
- * examine all tuples on the page and check if they meet the intended
- * condition. If result->recheck is true, only the indicated tuples need
- * be examined, but the condition must be rechecked anyway. (For ease of
- * testing, recheck is always set true when ntuples < 0.)
+ * Caller must pass in a TBMIterateResult to be filled.
+ *
+ * Pages are guaranteed to be delivered in numerical order. tbmres->blockno is
+ * set to InvalidBlockNumber when there are no more pages to scan. If
+ * tbmres->ntuples < 0, then the bitmap is "lossy" and failed to remember the
+ * exact tuples to look at on this page --- the caller must examine all tuples
+ * on the page and check if they meet the intended condition. If
+ * tbmres->recheck is true, only the indicated tuples need be examined, but the
+ * condition must be rechecked anyway. (For ease of testing, recheck is always
+ * set true when ntuples < 0.)
*/
-TBMIterateResult *
-tbm_iterate(TBMIterator *iterator)
+void
+tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres)
{
TIDBitmap *tbm = iterator->tbm;
- TBMIterateResult *output = &(iterator->output);
Assert(tbm->iterating == TBM_ITERATING_PRIVATE);
@@ -985,6 +984,7 @@ tbm_iterate(TBMIterator *iterator)
* If both chunk and per-page data remain, must output the numerically
* earlier page.
*/
+ Assert(tbmres);
if (iterator->schunkptr < tbm->nchunks)
{
PagetableEntry *chunk = tbm->schunks[iterator->schunkptr];
@@ -995,11 +995,11 @@ tbm_iterate(TBMIterator *iterator)
chunk_blockno < tbm->spages[iterator->spageptr]->blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
iterator->schunkbit++;
- return output;
+ return;
}
}
@@ -1015,16 +1015,17 @@ tbm_iterate(TBMIterator *iterator)
page = tbm->spages[iterator->spageptr];
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
iterator->spageptr++;
- return output;
+ return;
}
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
@@ -1034,10 +1035,9 @@ tbm_iterate(TBMIterator *iterator)
* across multiple processes. We need to acquire the iterator LWLock,
* before accessing the shared members.
*/
-TBMIterateResult *
-tbm_shared_iterate(TBMSharedIterator *iterator)
+void
+tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres)
{
- TBMIterateResult *output = &iterator->output;
TBMSharedIteratorState *istate = iterator->state;
PagetableEntry *ptbase = NULL;
int *idxpages = NULL;
@@ -1088,13 +1088,13 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
chunk_blockno < ptbase[idxpages[istate->spageptr]].blockno)
{
/* Return a lossy page indicator from the chunk */
- output->blockno = chunk_blockno;
- output->ntuples = -1;
- output->recheck = true;
+ tbmres->blockno = chunk_blockno;
+ tbmres->ntuples = -1;
+ tbmres->recheck = true;
istate->schunkbit++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
}
@@ -1104,21 +1104,22 @@ tbm_shared_iterate(TBMSharedIterator *iterator)
int ntuples;
/* scan bitmap to extract individual offset numbers */
- ntuples = tbm_extract_page_tuple(page, output);
- output->blockno = page->blockno;
- output->ntuples = ntuples;
- output->recheck = page->recheck;
+ ntuples = tbm_extract_page_tuple(page, tbmres);
+ tbmres->blockno = page->blockno;
+ tbmres->ntuples = ntuples;
+ tbmres->recheck = page->recheck;
istate->spageptr++;
LWLockRelease(&istate->lock);
- return output;
+ return;
}
LWLockRelease(&istate->lock);
/* Nothing more in the bitmap */
- return NULL;
+ tbmres->blockno = InvalidBlockNumber;
+ return;
}
/*
diff --git a/src/include/access/gin_private.h b/src/include/access/gin_private.h
index 3013a44bae1..3b432263bb0 100644
--- a/src/include/access/gin_private.h
+++ b/src/include/access/gin_private.h
@@ -353,7 +353,7 @@ typedef struct GinScanEntryData
/* for a partial-match or full-scan query, we accumulate all TIDs here */
TIDBitmap *matchBitmap;
TBMIterator *matchIterator;
- TBMIterateResult *matchResult;
+ TBMIterateResult matchResult;
/* used for Posting list and one page in Posting tree */
ItemPointerData *list;
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index cb56d20dc6f..3c330f86e62 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -34,7 +34,7 @@ typedef struct BitmapHeapIterator
struct TBMSharedIterator *parallel;
} BitmapHeapIterator;
-extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_iterate(BitmapHeapIterator *iterator, TBMIterateResult *result);
extern void bhs_end_iterate(BitmapHeapIterator *iterator);
diff --git a/src/include/nodes/tidbitmap.h b/src/include/nodes/tidbitmap.h
index 432fae52962..f000c1af28f 100644
--- a/src/include/nodes/tidbitmap.h
+++ b/src/include/nodes/tidbitmap.h
@@ -72,8 +72,8 @@ extern bool tbm_is_empty(const TIDBitmap *tbm);
extern TBMIterator *tbm_begin_iterate(TIDBitmap *tbm);
extern dsa_pointer tbm_prepare_shared_iterate(TIDBitmap *tbm);
-extern TBMIterateResult *tbm_iterate(TBMIterator *iterator);
-extern TBMIterateResult *tbm_shared_iterate(TBMSharedIterator *iterator);
+extern void tbm_iterate(TBMIterator *iterator, TBMIterateResult *tbmres);
+extern void tbm_shared_iterate(TBMSharedIterator *iterator, TBMIterateResult *tbmres);
extern void tbm_end_iterate(TBMIterator *iterator);
extern void tbm_end_shared_iterate(TBMSharedIterator *iterator);
extern TBMSharedIterator *tbm_attach_shared_iterate(dsa_area *dsa,
--
2.44.0
[text/x-patch] v14-0017-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (31.5K, ../../[email protected]/18-v14-0017-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From 8f271b04d9108b2c36b30141d807ef4976d59526 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v14 17/19] Push BitmapHeapScan prefetch code into heapam.c
In preparation for transitioning to using the streaming read API for
prefetching [1], move all of the BitmapHeapScanState members related to
prefetching and the functions for accessing them into the
HeapScanDescData and TableScanDescData. Members that still need to be
accessed in BitmapHeapNext() could not be moved into heap AM-specific
code. Specifically, parallel iterator setup requires several components
which seem odd to pass to the table AM API.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJkOiOCa%2Bmag4BF%2BzHo7qo%3Do9CFheB8%3Dg6uT5TUm2gkvA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 26 ++
src/backend/access/heap/heapam_handler.c | 262 +++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 341 ++--------------------
src/include/access/heapam.h | 17 ++
src/include/access/relscan.h | 8 +
src/include/access/tableam.h | 26 +-
src/include/nodes/execnodes.h | 14 -
7 files changed, 355 insertions(+), 339 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 10f2faaa60b..bda6abb8d0c 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -967,8 +967,16 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+
+ scan->rs_base.blockno = InvalidBlockNumber;
+
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
+ scan->pvmbuffer = InvalidBuffer;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1051,6 +1059,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
+ scan->rs_base.blockno = InvalidBlockNumber;
+
+ scan->pfblockno = InvalidBlockNumber;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+
/*
* unpin scan buffers
*/
@@ -1063,6 +1077,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1088,6 +1108,12 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
+ if (BufferIsValid(scan->pvmbuffer))
+ {
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ec16d8c5e3c..0138a23e9d4 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -60,6 +60,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
+static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2186,6 +2189,73 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ TBMIterateResult tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (prefetch_iterator)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->rs_base.prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (prefetch_iterator)
+ {
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ scan->pfblockno = tbmpre.blockno;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
bool *recheck, BlockNumber *blockno,
@@ -2204,6 +2274,8 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(hscan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2344,6 +2416,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->bm_parallel == NULL &&
+ scan->rs_pf_bhs_iterator &&
+ hscan->pfblockno < hscan->rs_base.blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(hscan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2354,6 +2438,154 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ int prefetch_maximum = scan->rs_base.prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(HeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
+ BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
+
+ if (pstate == NULL)
+ {
+ if (prefetch_iterator)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult tbmpre;
+ bool skip_fetch;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre.blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (prefetch_iterator)
+ {
+ while (1)
+ {
+ TBMIterateResult tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ bhs_iterate(prefetch_iterator, &tbmpre);
+ if (!BlockNumberIsValid(tbmpre.blockno))
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ scan->rs_base.rs_pf_bhs_iterator = NULL;
+ break;
+ }
+
+ scan->pfblockno = tbmpre.blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre.recheck &&
+ VM_ALL_VISIBLE(scan->rs_base.rs_rd,
+ tbmpre.blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
@@ -2379,6 +2611,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!scan->bm_parallel)
+ {
+ if (hscan->prefetch_target < scan->prefetch_maximum)
+ hscan->prefetch_target++;
+ }
+ else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&scan->bm_parallel->mutex);
+ if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
+ scan->bm_parallel->prefetch_target++;
+ SpinLockRelease(&scan->bm_parallel->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(hscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 1086e20234b..ea309cf93e3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
static BitmapHeapIterator *bhs_begin_iterate(TIDBitmap *tbm,
dsa_pointer shared_area,
@@ -122,7 +118,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TableScanDesc scan;
TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
@@ -142,7 +137,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
@@ -154,7 +149,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+ int pf_maximum = 0;
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -169,23 +173,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
+ node->pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
@@ -217,19 +214,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
+ scan->prefetch_maximum = pf_maximum;
+ scan->bm_parallel = node->pstate;
+
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->tbmiterator : InvalidDsaPointer,
+ scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
- node->pf_iterator = bhs_begin_iterate(tbm,
- pstate ? pstate->prefetch_iterator : InvalidDsaPointer,
- dsa);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
+ scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
+ scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
+ dsa);
}
#endif /* USE_PREFETCH */
@@ -244,36 +241,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -297,23 +264,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &node->blockno,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- node->pf_iterator &&
- node->pfblockno < node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -337,219 +290,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- node->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- node->pf_iterator = NULL;
- break;
- }
-
- node->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre.blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -595,22 +335,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator)
- {
- bhs_end_iterate(node->pf_iterator);
- node->pf_iterator = NULL;
- }
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -649,14 +379,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (node->pf_iterator)
- bhs_end_iterate(node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -689,17 +415,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->pf_iterator = NULL;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -739,13 +459,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -829,7 +542,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 750ea30852e..2e262bf4354 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -86,6 +86,23 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
+ /*
+ * These fields only used for prefetching in bitmap table scans
+ */
+
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+
+ /*
+ * These fields only used in serial BHS
+ */
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index fb22f305bf6..7938b741d66 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -26,6 +26,7 @@
struct ParallelTableScanDescData;
struct BitmapHeapIterator;
+struct ParallelBitmapHeapState;
/*
* Generic descriptor for table scans. This is the base-class for table scans,
@@ -45,6 +46,13 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
+ struct BitmapHeapIterator *rs_pf_bhs_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+ struct ParallelBitmapHeapState *bm_parallel;
+ /* used to validate BHS prefetch and current block stay in sync */
+ BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 70e538b76ba..a2b229fb877 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -785,17 +785,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the block's representation in the bitmap
* is lossy, otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -945,6 +934,9 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
+ result->rs_pf_bhs_iterator = NULL;
+ result->prefetch_maximum = 0;
+ result->bm_parallel = NULL;
return result;
}
@@ -996,6 +988,12 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -1012,6 +1010,12 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
+
+ if (scan->rs_pf_bhs_iterator)
+ {
+ bhs_end_iterate(scan->rs_pf_bhs_iterator);
+ scan->rs_pf_bhs_iterator = NULL;
+ }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cf8b4995f0d..592215d5ee7 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,18 +1794,11 @@ struct BitmapHeapIterator;
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1813,18 +1806,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- struct BitmapHeapIterator *pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.44.0
[text/x-patch] v14-0018-Remove-table_scan_bitmap_next_block.patch (11.8K, ../../[email protected]/19-v14-0018-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 8076d43ea02fa6d5bc137ce3721846a3dc492b72 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v14 18/19] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block(). Heap AM's implementation of
table_scan_bitmap_next_tuple() now calls the original
heapam_scan_bitmap_next_block() function, but it is no longer an
implementation of a table AM callback but instead a helper for
heapam_scan_bitmap_next_tuple()
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 45 ++++++--------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 75 +++++------------------
6 files changed, 63 insertions(+), 111 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index bda6abb8d0c..6c575ffc168 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -324,6 +324,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 0138a23e9d4..3e92725a821 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2183,12 +2183,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2222,8 +2216,8 @@ BitmapAdjustPrefetchIterator(HeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->rs_base.prefetch_maximum > 0)
{
@@ -2586,30 +2580,43 @@ BitmapPrefetch(HeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
- return true;
- }
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -3006,7 +3013,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index 55b8caeadf2..0b87530a9c4 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -90,8 +90,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index ea309cf93e3..7f651004113 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -231,44 +231,35 @@ BitmapHeapNext(BitmapHeapScanState *node)
#endif /* USE_PREFETCH */
node->initialized = true;
-
- goto new_page;
}
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck, &scan->blockno,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346f..cf56cc572fe 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index a2b229fb877..79689ec3773 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -773,36 +773,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the block's representation in the bitmap
- * is lossy, otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- BlockNumber *blockno,
- long *lossy_pages,
- long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -1981,44 +1965,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented if bitmap is lossy for the selected block and exact_pages is
- * incremented otherwise.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
- long *lossy_pages, long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan, recheck,
- blockno, lossy_pages,
- exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2029,7 +1982,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.44.0
[text/x-patch] v14-0019-BitmapHeapScan-uses-read-stream-API.patch (26.2K, ../../[email protected]/20-v14-0019-BitmapHeapScan-uses-read-stream-API.patch)
download | inline diff:
From 9a98aa4c74d301500d5d3ce9b0ddf46360167a41 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 16:51:40 -0400
Subject: [PATCH v14 19/19] BitmapHeapScan uses read stream API
Remove all of the code to do prefetching from BitmapHeapScan code and
rely on the read stream API prefetching. Heap table AM implements a read
stream callback which uses the iterator to get the next valid block that
needs to be fetched for the read stream API.
---
src/backend/access/heap/heapam.c | 95 ++++--
src/backend/access/heap/heapam_handler.c | 347 +++-------------------
src/backend/executor/nodeBitmapHeapscan.c | 43 +--
src/include/access/heapam.h | 21 +-
src/include/access/relscan.h | 6 -
src/include/access/tableam.h | 14 -
src/include/nodes/execnodes.h | 9 +-
7 files changed, 115 insertions(+), 420 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 6c575ffc168..b0af15eb9c0 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -113,6 +113,8 @@ static XLogRecPtr log_heap_new_cid(Relation relation, HeapTuple tup);
static HeapTuple ExtractReplicaIdentity(Relation relation, HeapTuple tp, bool key_required,
bool *copy);
+static BlockNumber bitmapheap_stream_read_next(ReadStream *pgsr, void *pgsr_private,
+ void *per_buffer_data);
/*
* Each tuple lock mode has a corresponding heavyweight lock, and one or two
@@ -969,16 +971,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
-
- scan->rs_base.blockno = InvalidBlockNumber;
-
scan->rs_vmbuffer = InvalidBuffer;
scan->rs_empty_tuples_pending = 0;
- scan->pvmbuffer = InvalidBuffer;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1033,6 +1027,23 @@ heap_beginscan(Relation relation, Snapshot snapshot,
initscan(scan, key, false);
+ /*
+ * Make the read stream object for BitmapHeapScan. BitmapHeapScan does not
+ * use a BufferAccessStrategy, so we could do this before initscan(), but
+ * wait until after so the scan descriptor is fully set up. On rescan, the
+ * stream will be reset.
+ */
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ scan->rs_read_stream = read_stream_begin_relation(READ_STREAM_DEFAULT,
+ scan->rs_strategy,
+ scan->rs_base.rs_rd,
+ MAIN_FORKNUM,
+ bitmapheap_stream_read_next,
+ scan,
+ sizeof(TBMIterateResult));
+ }
+
return (TableScanDesc) scan;
}
@@ -1061,12 +1072,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_base.rs_flags &= ~SO_ALLOW_PAGEMODE;
}
- scan->rs_base.blockno = InvalidBlockNumber;
-
- scan->pfblockno = InvalidBlockNumber;
- scan->prefetch_target = -1;
- scan->prefetch_pages = 0;
-
/*
* unpin scan buffers
*/
@@ -1079,11 +1084,8 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_read_stream)
+ read_stream_reset(scan->rs_read_stream);
/*
* reinitialize scan descriptor
@@ -1110,11 +1112,8 @@ heap_endscan(TableScanDesc sscan)
scan->rs_vmbuffer = InvalidBuffer;
}
- if (BufferIsValid(scan->pvmbuffer))
- {
- ReleaseBuffer(scan->pvmbuffer);
- scan->pvmbuffer = InvalidBuffer;
- }
+ if (scan->rs_base.rs_flags & SO_TYPE_BITMAPSCAN && scan->rs_read_stream)
+ read_stream_end(scan->rs_read_stream);
/*
* decrement relation reference count and free scan descriptor storage
@@ -10130,3 +10129,51 @@ HeapCheckForSerializableConflictOut(bool visible, Relation relation,
CheckForSerializableConflictOut(relation, xid, snapshot);
}
+
+static BlockNumber
+bitmapheap_stream_read_next(ReadStream *pgsr, void *private_data,
+ void *per_buffer_data)
+{
+ TBMIterateResult *tbmres = per_buffer_data;
+ HeapScanDesc hdesc = (HeapScanDesc) private_data;
+
+ for (;;)
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ bhs_iterate(hdesc->rs_base.rs_bhs_iterator, tbmres);
+
+ /* no more entries in the bitmap */
+ if (!BlockNumberIsValid(tbmres->blockno))
+ return InvalidBlockNumber;
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ if (!IsolationIsSerializable() && tbmres->blockno >= hdesc->rs_nblocks)
+ continue;
+
+ /*
+ * We can skip fetching the heap page if we don't need any fields from
+ * the heap, the bitmap entries don't need rechecking, and all tuples
+ * on the page are visible to our transaction.
+ */
+ if (!(hdesc->rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(hdesc->rs_base.rs_rd, tbmres->blockno, &hdesc->rs_vmbuffer))
+ {
+ hdesc->rs_empty_tuples_pending += tbmres->ntuples;
+ continue;
+ }
+
+ return tbmres->blockno;
+ }
+
+ /* not reachable */
+ Assert(false);
+}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3e92725a821..cc6289b6343 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -60,9 +60,6 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
-static inline void BitmapAdjustPrefetchIterator(HeapScanDesc scan);
-static inline void BitmapAdjustPrefetchTarget(HeapScanDesc scan);
-static inline void BitmapPrefetch(HeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2183,147 +2180,68 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- TBMIterateResult tbmpre;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- scan->prefetch_pages--;
- }
- else if (prefetch_iterator)
- {
- /* Do not let the prefetch iterator get behind the main one */
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * heapam_bitmap_next_block() keeps prefetch distance higher across the
- * parallel workers.
- */
- if (scan->rs_base.prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (prefetch_iterator)
- {
- bhs_iterate(prefetch_iterator, &tbmpre);
- scan->pfblockno = tbmpre.blockno;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck, BlockNumber *blockno,
+heapam_scan_bitmap_next_block(TableScanDesc scan, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
+ void *per_buffer_data;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
- TBMIterateResult tbmres;
+ TBMIterateResult *tbmres;
+
+ Assert(hscan->rs_read_stream);
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
- BitmapAdjustPrefetchIterator(hscan);
-
- do
+ /* Release buffer containing previous block. */
+ if (BufferIsValid(hscan->rs_cbuf))
{
- CHECK_FOR_INTERRUPTS();
+ ReleaseBuffer(hscan->rs_cbuf);
+ hscan->rs_cbuf = InvalidBuffer;
+ }
- bhs_iterate(scan->rs_bhs_iterator, &tbmres);
+ hscan->rs_cbuf = read_stream_next_buffer(hscan->rs_read_stream, &per_buffer_data);
- if (!BlockNumberIsValid(tbmres.blockno))
+ if (BufferIsInvalid(hscan->rs_cbuf))
+ {
+ if (BufferIsValid(hscan->rs_vmbuffer))
{
- /* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
- return false;
+ ReleaseBuffer(hscan->rs_vmbuffer);
+ hscan->rs_vmbuffer = InvalidBuffer;
}
/*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE
- * isolation though, as we need to examine all invisible tuples
- * reachable by the index.
+ * Bitmap is exhausted. Time to emit empty tuples if relevant. We emit
+ * all empty tuples at the end instead of emitting them per block we
+ * skip fetching. This is necessary because the streaming read API
+ * will only return TBMIterateResults for blocks actually fetched.
+ * When we skip fetching a block, we keep track of how many empty
+ * tuples to emit at the end of the BitmapHeapScan. We do not recheck
+ * all NULL tuples.
*/
- } while (!IsolationIsSerializable() && tbmres.blockno >= hscan->rs_nblocks);
+ *recheck = false;
+ return hscan->rs_empty_tuples_pending > 0;
+ }
- /* Got a valid block */
- *blockno = tbmres.blockno;
- *recheck = tbmres.recheck;
+ Assert(per_buffer_data);
- /*
- * We can skip fetching the heap page if we don't need any fields from the
- * heap, the bitmap entries don't need rechecking, and all tuples on the
- * page are visible to our transaction.
- */
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmres.recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres.blockno, &hscan->rs_vmbuffer))
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres.ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ tbmres = per_buffer_data;
- hscan->rs_empty_tuples_pending += tbmres.ntuples;
+ Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
- return true;
- }
+ *recheck = tbmres->recheck;
- block = tbmres.blockno;
+ hscan->rs_cblock = tbmres->blockno;
+ hscan->rs_ntuples = tbmres->ntuples;
- /*
- * Acquire pin on the target heap page, trading in any pin we held before.
- */
- hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
- block);
- hscan->rs_cblock = block;
+ block = tbmres->blockno;
buffer = hscan->rs_cbuf;
snapshot = scan->rs_snapshot;
@@ -2344,7 +2262,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
/*
* We need two separate strategies for lossy and non-lossy cases.
*/
- if (tbmres.ntuples >= 0)
+ if (tbmres->ntuples >= 0)
{
/*
* Bitmap is non-lossy, so we just look through the offsets listed in
@@ -2353,9 +2271,9 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
*/
int curslot;
- for (curslot = 0; curslot < tbmres.ntuples; curslot++)
+ for (curslot = 0; curslot < tbmres->ntuples; curslot++)
{
- OffsetNumber offnum = tbmres.offsets[curslot];
+ OffsetNumber offnum = tbmres->offsets[curslot];
ItemPointerData tid;
HeapTupleData heapTuple;
@@ -2405,23 +2323,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
- if (tbmres.ntuples < 0)
+ if (tbmres->ntuples < 0)
(*lossy_pages)++;
else
(*exact_pages)++;
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (scan->bm_parallel == NULL &&
- scan->rs_pf_bhs_iterator &&
- hscan->pfblockno < hscan->rs_base.blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(hscan);
-
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2432,153 +2338,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- int prefetch_maximum = scan->rs_base.prefetch_maximum;
-
- if (pstate == NULL)
- {
- if (scan->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (scan->prefetch_target >= prefetch_maximum / 2)
- scan->prefetch_target = prefetch_maximum;
- else if (scan->prefetch_target > 0)
- scan->prefetch_target *= 2;
- else
- scan->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= prefetch_maximum / 2)
- pstate->prefetch_target = prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(HeapScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = scan->rs_base.bm_parallel;
- BitmapHeapIterator *prefetch_iterator = scan->rs_base.rs_pf_bhs_iterator;
-
- if (pstate == NULL)
- {
- if (prefetch_iterator)
- {
- while (scan->prefetch_pages < scan->prefetch_target)
- {
- TBMIterateResult tbmpre;
- bool skip_fetch;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
-
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
- scan->prefetch_pages++;
- scan->pfblockno = tbmpre.blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (prefetch_iterator)
- {
- while (1)
- {
- TBMIterateResult tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- bhs_iterate(prefetch_iterator, &tbmpre);
- if (!BlockNumberIsValid(tbmpre.blockno))
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- scan->rs_base.rs_pf_bhs_iterator = NULL;
- break;
- }
-
- scan->pfblockno = tbmpre.blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_base.rs_flags & SO_NEED_TUPLE) &&
- !tbmpre.recheck &&
- VM_ALL_VISIBLE(scan->rs_base.rs_rd,
- tbmpre.blockno,
- &scan->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_base.rs_rd, MAIN_FORKNUM, tbmpre.blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
@@ -2613,41 +2372,11 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
return true;
}
- if (!heapam_scan_bitmap_next_block(scan, recheck, &scan->blockno,
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
lossy_pages, exact_pages))
return false;
}
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the second
- * page if we don't stop reading after the first tuple.
- */
- if (!scan->bm_parallel)
- {
- if (hscan->prefetch_target < scan->prefetch_maximum)
- hscan->prefetch_target++;
- }
- else if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&scan->bm_parallel->mutex);
- if (scan->bm_parallel->prefetch_target < scan->prefetch_maximum)
- scan->bm_parallel->prefetch_target++;
- SpinLockRelease(&scan->bm_parallel->mutex);
- }
-
- /*
- * We issue prefetch requests *after* fetching the current page to try to
- * avoid having prefetching interfere with the main I/O. Also, this should
- * happen only when we have determined there is still something to do on
- * the current page, else we may uselessly prefetch the same page we are
- * just about to request for real.
- */
- BitmapPrefetch(hscan);
-#endif /* USE_PREFETCH */
-
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 7f651004113..38a6262e3c4 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -131,14 +131,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
@@ -149,15 +141,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- int pf_maximum = 0;
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(node->ss.ss_currentRelation->rd_rel->reltablespace);
-#endif
-
if (!node->pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,13 +157,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
* multiple processes to iterate jointly.
*/
node->pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
+
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(node->pstate);
}
@@ -214,22 +191,12 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
- scan->prefetch_maximum = pf_maximum;
scan->bm_parallel = node->pstate;
scan->rs_bhs_iterator = bhs_begin_iterate(tbm,
scan->bm_parallel ? scan->bm_parallel->tbmiterator : InvalidDsaPointer,
dsa);
-#ifdef USE_PREFETCH
- if (scan->prefetch_maximum > 0)
- {
- scan->rs_pf_bhs_iterator = bhs_begin_iterate(tbm,
- scan->bm_parallel ? scan->bm_parallel->prefetch_iterator : InvalidDsaPointer,
- dsa);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
}
@@ -526,14 +493,10 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
return;
pstate = shm_toc_allocate(pcxt->toc, sizeof(ParallelBitmapHeapState));
-
pstate->tbmiterator = 0;
- pstate->prefetch_iterator = 0;
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
@@ -564,11 +527,7 @@ ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
if (DsaPointerIsValid(pstate->tbmiterator))
tbm_free_shared_area(dsa, pstate->tbmiterator);
- if (DsaPointerIsValid(pstate->prefetch_iterator))
- tbm_free_shared_area(dsa, pstate->prefetch_iterator);
-
pstate->tbmiterator = InvalidDsaPointer;
- pstate->prefetch_iterator = InvalidDsaPointer;
}
/* ----------------------------------------------------------------
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2e262bf4354..8702a20741b 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/dsm.h"
#include "storage/lockdefs.h"
#include "storage/shm_toc.h"
+#include "storage/read_stream.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -76,6 +77,9 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* Streaming read control object for scans supporting it */
+ ReadStream *rs_read_stream;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
@@ -86,23 +90,6 @@ typedef struct HeapScanDescData
Buffer rs_vmbuffer;
int rs_empty_tuples_pending;
- /*
- * These fields only used for prefetching in bitmap table scans
- */
-
- /* buffer for visibility-map lookups of prefetched pages */
- Buffer pvmbuffer;
-
- /*
- * These fields only used in serial BHS
- */
- /* Current target for prefetch distance */
- int prefetch_target;
- /* # pages prefetch iterator is ahead of current */
- int prefetch_pages;
- /* used to validate prefetch block stays ahead of current block */
- BlockNumber pfblockno;
-
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 7938b741d66..02893bf99bc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -46,13 +46,7 @@ typedef struct TableScanDescData
/* Only used for Bitmap table scans */
struct BitmapHeapIterator *rs_bhs_iterator;
- struct BitmapHeapIterator *rs_pf_bhs_iterator;
-
- /* maximum value for prefetch_target */
- int prefetch_maximum;
struct ParallelBitmapHeapState *bm_parallel;
- /* used to validate BHS prefetch and current block stay in sync */
- BlockNumber blockno;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 79689ec3773..aead772e6fd 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -918,8 +918,6 @@ table_beginscan_bm(Relation rel, Snapshot snapshot,
result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
result->rs_bhs_iterator = NULL;
- result->rs_pf_bhs_iterator = NULL;
- result->prefetch_maximum = 0;
result->bm_parallel = NULL;
return result;
}
@@ -972,12 +970,6 @@ table_endscan(TableScanDesc scan)
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_end(scan);
@@ -994,12 +986,6 @@ table_rescan(TableScanDesc scan,
{
bhs_end_iterate(scan->rs_bhs_iterator);
scan->rs_bhs_iterator = NULL;
-
- if (scan->rs_pf_bhs_iterator)
- {
- bhs_end_iterate(scan->rs_pf_bhs_iterator);
- scan->rs_pf_bhs_iterator = NULL;
- }
}
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 592215d5ee7..8d86b900a0b 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1767,11 +1767,7 @@ typedef enum
/* ----------------
* ParallelBitmapHeapState information
* tbmiterator iterator for scanning current pages
- * prefetch_iterator iterator for prefetching ahead of current page
- * mutex mutual exclusion for the prefetching variable
- * and state
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
+ * mutex mutual exclusion for state
* state current state of the TIDBitmap
* cv conditional wait variable
* ----------------
@@ -1779,10 +1775,7 @@ typedef enum
typedef struct ParallelBitmapHeapState
{
dsa_pointer tbmiterator;
- dsa_pointer prefetch_iterator;
slock_t mutex;
- int prefetch_pages;
- int prefetch_target;
SharedBitmapState state;
ConditionVariable cv;
} ParallelBitmapHeapState;
--
2.44.0
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-31 15:45 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-03 22:57 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-04 14:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
@ 2024-04-05 08:06 ` Melanie Plageman <[email protected]>
2024-04-05 23:53 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Melanie Plageman @ 2024-04-05 08:06 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Thomas Munro <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Thu, Apr 04, 2024 at 04:35:45PM +0200, Tomas Vondra wrote:
>
>
> On 4/4/24 00:57, Melanie Plageman wrote:
> > On Sun, Mar 31, 2024 at 11:45:51AM -0400, Melanie Plageman wrote:
> >> On Fri, Mar 29, 2024 at 12:05:15PM +0100, Tomas Vondra wrote:
> >>>
> >>>
> >>> On 3/29/24 02:12, Thomas Munro wrote:
> >>>> On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
> >>>> <[email protected]> wrote:
> >>>>> I think there's some sort of bug, triggering this assert in heapam
> >>>>>
> >>>>> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
> >>>>
> >>>> Thanks for the repro. I can't seem to reproduce it (still trying) but
> >>>> I assume this is with Melanie's v11 patch set which had
> >>>> v11-0016-v10-Read-Stream-API.patch.
> >>>>
> >>>> Would you mind removing that commit and instead applying the v13
> >>>> stream_read.c patches[1]? v10 stream_read.c was a little confused
> >>>> about random I/O combining, which I fixed with a small adjustment to
> >>>> the conditions for the "if" statement right at the end of
> >>>> read_stream_look_ahead(). Sorry about that. The fixed version, with
> >>>> eic=4, with your test query using WHERE a < a, ends its scan with:
> >>>>
> >>>
> >>> I'll give that a try. Unfortunately unfortunately the v11 still has the
> >>> problem I reported about a week ago:
> >>>
> >>> ERROR: prefetch and main iterators are out of sync
> >>>
> >>> So I can't run the full benchmarks :-( but master vs. streaming read API
> >>> should work, I think.
> >>
> >> Odd, I didn't notice you reporting this ERROR popping up. Now that I
> >> take a look, v11 (at least, maybe also v10) had this very sill mistake:
> >>
> >> if (scan->bm_parallel == NULL &&
> >> scan->rs_pf_bhs_iterator &&
> >> hscan->pfblockno > hscan->rs_base.blockno)
> >> elog(ERROR, "prefetch and main iterators are out of sync");
> >>
> >> It errors out if the prefetch block is ahead of the current block --
> >> which is the opposite of what we want. I've fixed this in attached v12.
> >>
> >> This version also has v13 of the streaming read API. I noticed one
> >> mistake in my bitmapheap scan streaming read user -- it freed the
> >> streaming read object at the wrong time. I don't know if this was
> >> causing any other issues, but it at least is fixed in this version.
> >
> > Attached v13 is rebased over master (which includes the streaming read
> > API now). I also reset the streaming read object on rescan instead of
> > creating a new one each time.
> >
> > I don't know how much chance any of this has of going in to 17 now, but
> > I thought I would start looking into the regression repro Tomas provided
> > in [1].
> >
>
> My personal opinion is that we should try to get in as many of the the
> refactoring patches as possible, but I think it's probably too late for
> the actual switch to the streaming API.
Cool. In the attached v15, I have dropped all commits that are related
to the streaming read API and included *only* commits that are
beneficial to master. A few of the commits are merged or reordered as
well.
While going through the commits with this new goal in mind (forget about
the streaming read API for now), I realized that it doesn't make much
sense to just eliminate the layering violation for the current block and
leave it there for the prefetch block. I had de-prioritized solving this
when I thought we would just delete the prefetch code and replace it
with the streaming read.
Now that we aren't doing that, I've spent the day trying to resolve the
issues with pushing the prefetch code into heapam.c that I cited in [1].
0010 - 0013 are the result of this. They are not very polished yet and
need more cleanup and review (especially 0011, which is probably too
large), but I am happy with the solution I came up with.
Basically, there are too many members needed for bitmap heap scan to put
them all in the HeapScanDescData (don't want to bloat it). So, I've made
a new BitmapHeapScanDescData and associated begin/rescan/end() functions
In the end, with all patches applied, BitmapHeapNext() loops invoking
table_scan_bitmap_next_tuple() and table AMs can implement that however
they choose.
> > I'm also not sure if I should try and group the commits into fewer
> > commits now or wait until I have some idea of whether or not the
> > approach in 0013 and 0014 is worth pursuing.
> >
>
> You mean whether to pursue the approach in general, or for v17? I think
> it looks like the right approach, but for v17 see above :-(
>
> As for merging, I wouldn't do that. I looked at the commits and while
> some of them seem somewhat "trivial", I really like how you organized
> the commits, and kept those that just "move" code around, and those that
> actually change stuff. It's much easier to understand, IMO.
>
> I went through the first ~10 commits, and added some review - either as
> a separate commit, when possible, in the code as XXX comment, and also
> in the commit message. The code tweaks are utterly trivial (whitespace
> or indentation to make the line shorter). It shouldn't take much time to
> deal with those, I think.
Attached v15 incorporates your v14-0002-review.
For your v14-0008-review, I actually ended up removing that commit
because once I removed everything that was for streaming read API, it
became redundant with another commit.
For your v14-0010-review, we actually can't easily get rid of those
local variables because we make the iterators before we make the scan
descriptors and the commit following that commit moves the iterators
from the BitmapHeapScanState to the scan descriptor.
> I think the main focus should be updating the commit messages. If it was
> only a single patch, I'd probably try to write the messages myself, but
> with this many patches it'd be great if you could update those and I'll
> review that before commit.
I did my best to update the commit messages to be less specific and more
focused on "why should I care". I found myself wanting to explain why I
implemented something the way I did and then getting back into the
implementation details again. I'm not sure if I suceeded in having less
details and more substance.
> I always struggle with writing commit messages myself, and it takes me
> ages to write a good one (well, I think the message is good, but who
> knows ...). But I think a good message should be concise enough to
> explain what and why it's done. It may reference a thread for all the
> gory details, but the basic reasoning should be in the commit message.
> For example the message for "BitmapPrefetch use prefetch block recheck
> for skip fetch" now says that it "makes more sense to do X" but does not
> really say why that's the case. The linked message does, but it'd be
> good to have that in the message (because how would I know how much of
> the thread to read?).
I fixed that particular one. I tried to take that feedback and apply it
to other commit messages. I don't know how successful I was...
> Also, it'd be very helpful if you could update the author & reviewed-by
> fields. I'll review those before commit, ofc, but I admit I lost track
> of who reviewed which part.
I have updated reviewers. I didn't add reviewers on the ones that
haven't really been reviewed yet.
> I'd focus on the first ~8-9 commits or so for now, we can commit more if
> things go reasonably well.
Sounds good. I will spend cleanup time on 0010-0013 tomorrow but would
love to know if you agree with the direction before I spend more time.
- Melanie
[1] https://www.postgresql.org/message-id/20240323002211.on5vb5ulk6lsdb2u%40liskov
Attachments:
[text/x-diff] v15-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.4K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/2-v15-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From adae81352723f22ba2528dc4bf5f2216c7641428 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v15 01/13] BitmapHeapScan begin scan after bitmap creation
It makes more sense for BitmapHeapScan to scan the index, build the
bitmap, and then begin the scan on the underlying table.
This is primarily a cosmetic change for now, but later commits will pass
parameters to table_beginscan_bm() that are unavailable in
ExecInitBitmapHeapScan().
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 27 +++++++++++++++++------
1 file changed, 20 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..c8c466e3c5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = table_beginscan_bm(node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+
+ node->ss.ss_currentScanDesc = scan;
+ }
+
node->initialized = true;
}
@@ -601,7 +616,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +694,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +801,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v15-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.3K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/3-v15-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 6957ee25cf85784906a1315d148180858aedaad2 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v15 02/13] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() instead of
in ExecInitBitmapHeapScan(). This is a preliminary step to pushing the
skip fetch optimization into heap AM code.
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c8c466e3c5..2148a21531 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -743,16 +753,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v15-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (14.7K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/4-v15-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From 7566751f0ce9d68154e1f5616f7585334f497716 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 15:34:25 +0200
Subject: [PATCH v15 03/13] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM-specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block. The layering violation is
still present in BitmapHeapScans's prefetching code, which uses the
visibility map to decide whether or not to prefetch a block. However,
this will be fixed in an upcoming commit.
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index dada2ecd1e..10f2faaa60 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -967,6 +967,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1055,6 +1057,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1074,6 +1082,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3e7a6b5548..929b2cf8e7 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2198,6 +2199,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2310,6 +2329,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2148a21531..a8bc5dec53 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,10 +185,24 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
node->ss.ss_currentScanDesc = scan;
}
@@ -208,7 +212,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -229,37 +233,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -302,52 +283,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -519,7 +481,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -570,7 +532,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -640,8 +602,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -651,7 +611,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -696,8 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -741,8 +698,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -753,7 +708,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2765efc4e5..750ea30852 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -76,6 +76,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e7eeb75409..55f1139757 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -63,6 +63,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -937,9 +944,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e57ebd7288..fa2f70b7a4 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,10 +1794,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1817,9 +1814,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v15-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.4K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/5-v15-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From 9d18888d6e80bc7e2bb676b2803a463f5ee4b04c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v15 04/13] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. This doesn't seem right. We should use the
prefetch block's recheck flag to determine whether or not to prefetch
it. The current block's recheck flag has no relationship to the prefetch
block's recheck flag. See this [1] thread on hackers reporting the
issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a8bc5dec53..2e2cec8b3b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -475,14 +475,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -533,7 +528,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v15-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.5K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/6-v15-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 200f5b4e4e79d9cd9e0d8f02df574d616a3569eb Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v15 05/13] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2e2cec8b3b..795a893035 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -231,7 +231,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -342,7 +342,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -361,7 +361,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v15-0006-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (3.1K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/7-v15-0006-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From 001b361b38b294b8cbf7d7c4487804d56f0929e1 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v15 06/13] Reduce scope of BitmapHeapScan tbmiterator local
variables
A future patch will change where tbmiterators are initialized and saved.
To simplify that diff, move them into a tighter scope.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 795a893035..8403be84f3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -207,6 +203,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -222,9 +220,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v15-0007-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.1K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/8-v15-0007-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From b865213848634d95f22ce50aa701eb1ac4722c5c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v15 07/13] table_scan_bitmap_next_block counts lossy and exact
pages
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So the table AM must keep
track of whether or not individual TBMIterateResult's blocks were
represented lossily in the bitmap for the purposes of EXPLAIN counters.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 8 +++++++-
src/backend/executor/nodeBitmapHeapscan.c | 13 +++----------
src/include/access/tableam.h | 22 ++++++++++++++++------
3 files changed, 26 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 929b2cf8e7..b5ab104ec2 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,7 +2188,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2316,6 +2317,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 8403be84f3..68f9ded168 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -210,8 +210,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -231,19 +229,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
-
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
- node->lossy_pages++;
-
- if (!valid)
+ if (!table_scan_bitmap_next_block(scan, tbmres,
+ &node->lossy_pages, &node->exact_pages))
{
/* AM doesn't think this block is valid, skip */
continue;
}
+
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 55f1139757..e2ad8d0728 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -789,6 +789,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy_pages is incremented if the bitmap is lossy for the selected
+ * block; otherwise, exact_pages is incremented.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -804,7 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ long *lossy_pages, long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1968,17 +1972,21 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
+ * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of a
+ * bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy_pages is incremented is the block's
+ * representation in the bitmap is lossy; otherwise, exact_pages is
+ * incremented.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ long *lossy_pages,
+ long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1989,7 +1997,9 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v15-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/9-v15-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From c4098d4d1ec206b837711b03f5dfc322346be497 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v15 08/13] Remove table_scan_bitmap_next_tuple parameter
tbmres
Future commits will push all of the logic for choosing the next block
into the table AM specific code. The table AM will be responsible for
iterating and reading in the right blocks.
Thus, it no longer makes sense to use the TBMIterateResult (which
contains a block number) as a means of communication between
table_scan_bitmap_next_tuple() and table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index b5ab104ec2..efd1a66a09 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2327,7 +2327,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 68f9ded168..951a98c101 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -280,7 +280,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e2ad8d0728..2efa97f602 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -780,10 +780,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -814,15 +811,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2012,7 +2004,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2024,7 +2015,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v15-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.4K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/10-v15-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From 520acaf3df90cec9db518f6fda660ae9bb81fba0 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v15 09/13] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to the
table AM.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 58 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 186 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 190 insertions(+), 141 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index efd1a66a09..9d3e7c7fda 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,18 +2188,52 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
+ BlockNumber *blockno, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2218,16 +2252,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2322,7 +2347,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 951a98c101..e1b13ddaa6 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -71,7 +70,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +81,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +108,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +160,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -203,48 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- if (!table_scan_bitmap_next_block(scan, tbmres,
- &node->lossy_pages, &node->exact_pages))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -266,45 +232,56 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->blockno, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
+ break;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno < node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -330,13 +307,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -350,14 +331,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -382,7 +366,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -460,6 +447,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -517,6 +505,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -578,12 +568,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -591,13 +577,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -628,28 +614,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -682,8 +664,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -691,9 +671,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2efa97f602..42c67a128e 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -22,6 +22,7 @@
#include "access/xact.h"
#include "commands/vacuum.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -773,19 +774,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy_pages is incremented if the bitmap is lossy for the selected
* block; otherwise, exact_pages is incremented.
*
@@ -804,7 +800,7 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
+ BlockNumber *blockno, bool *recheck,
long *lossy_pages, long *exact_pages);
/*
@@ -942,9 +938,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -991,6 +991,21 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1001,6 +1016,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1964,19 +1994,18 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of a
- * bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy_pages is incremented is the block's
- * representation in the bitmap is lossy; otherwise, exact_pages is
- * incremented.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented is the block's representation in the bitmap is lossy; otherwise,
+ * exact_pages is incremented.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
+ BlockNumber *blockno, bool *recheck,
long *lossy_pages,
long *exact_pages)
{
@@ -1989,9 +2018,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres,
- lossy_pages,
- exact_pages);
+ blockno, recheck,
+ lossy_pages, exact_pages);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index fa2f70b7a4..d96703b04d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1792,8 +1792,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1802,9 +1800,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1812,8 +1812,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1822,9 +1820,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v15-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.7K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/11-v15-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From dcdfa8174ef7927f64bfb07f29bf0e9c2a45fde7 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v15 10/13] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext().
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 172 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 40 +----
src/include/executor/nodeBitmapHeapscan.h | 7 +
src/include/nodes/execnodes.h | 13 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 124 insertions(+), 121 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 9d3e7c7fda..5e57d19d91 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2208,10 +2208,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(&scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index e1b13ddaa6..70b560b8ee 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -57,6 +57,61 @@ static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+/*
+ * Start iteration on a shared or non-shared bitmap iterator. Note that tbm
+ * will only be provided by serial BitmapHeapScan callers. dsa and dsp will
+ * only be provided by parallel BitmapHeapScan callers.
+ */
+void
+bhs_begin_iterate(BitmapHeapIterator *iterator, TIDBitmap *tbm,
+ dsa_area *dsa, dsa_pointer dsp)
+{
+ Assert(iterator);
+
+ iterator->serial = NULL;
+ iterator->parallel = NULL;
+ iterator->exhausted = false;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(dsp))
+ iterator->parallel = tbm_attach_shared_iterate(dsa, dsp);
+ else
+ iterator->serial = tbm_begin_iterate(tbm);
+}
+
+/*
+ * Get the next TBMIterateResult from the bitmap iterator.
+ */
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+ Assert(!iterator->exhausted);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+/*
+ * Clean up the bitmap iterator.
+ */
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ tbm_end_iterate(iterator->serial);
+ else if (iterator->parallel)
+ tbm_end_shared_iterate(iterator->parallel);
+
+ iterator->serial = NULL;
+ iterator->parallel = NULL;
+ iterator->exhausted = true;
+}
+
/* ----------------------------------------------------------------
* BitmapHeapNext
@@ -96,43 +151,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (init_shared_state)
{
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
- {
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -153,21 +188,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +221,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ bhs_begin_iterate(&scan->rs_bhs_iterator, tbm, dsa,
+ pstate ?
+ pstate->tbmiterator :
+ InvalidDsaPointer);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ bhs_begin_iterate(&node->pf_iterator, tbm, dsa,
+ pstate ?
+ pstate->prefetch_iterator :
+ InvalidDsaPointer);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -276,7 +314,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ !node->pf_iterator.exhausted &&
node->pfblockno < node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -317,21 +355,20 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
node->prefetch_pages--;
}
- else if (prefetch_iterator)
+ else if (!prefetch_iterator->exhausted)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -344,8 +381,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -365,9 +400,9 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
* we don't validate the blockno here as we do in non-parallel
* case.
*/
- if (prefetch_iterator)
+ if (!prefetch_iterator->exhausted)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -427,23 +462,21 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
+ if (!prefetch_iterator->exhausted)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
break;
}
node->prefetch_pages++;
@@ -471,9 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
+ if (!prefetch_iterator->exhausted)
{
while (1)
{
@@ -496,12 +527,11 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
break;
}
@@ -568,18 +598,14 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator.exhausted)
+ bhs_end_iterate(&node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -624,12 +650,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (!node->pf_iterator.exhausted)
+ bhs_end_iterate(&node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -667,11 +691,9 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..e520186b41 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,13 +20,11 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
-
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -44,8 +42,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ BitmapHeapIterator rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 42c67a128e..618ab2b449 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -938,13 +938,9 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
- return result;
+ return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
/*
@@ -991,20 +987,9 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
- {
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
- }
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
+ !scan->rs_bhs_iterator.exhausted)
+ bhs_end_iterate(&scan->rs_bhs_iterator);
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1016,20 +1001,9 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
- {
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
- }
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
+ !scan->rs_bhs_iterator.exhausted)
+ bhs_end_iterate(&scan->rs_bhs_iterator);
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caa..7064f54686 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -29,4 +29,11 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_begin_iterate(BitmapHeapIterator *iterator, TIDBitmap *tbm,
+ dsa_area *dsa, dsa_pointer dsp);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index d96703b04d..714eb3e534 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1787,6 +1787,13 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+ bool exhausted;
+} BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1795,12 +1802,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1815,12 +1821,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ BitmapHeapIterator pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index f3b8641d76..2348a8793e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v15-0011-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch (47.7K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/12-v15-0011-Push-BitmapHeapScan-prefetch-code-into-heapam.c.patch)
download | inline diff:
From d141ad447470458de47c6c9b54c97491bb9f168c Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 09:42:23 -0400
Subject: [PATCH v15 11/13] Push BitmapHeapScan prefetch code into heapam.c
7566751f0ce9 eliminated a table AM layering violation for the current
block in a BitmapHeapScan but left the violation in BitmapHeapScan
prefetching code.
To resolve this, move the BitmapHeapScan prefetching logic entirely into
heap AM code. There is a fair amount of setup that is specific to
BitmapHeapScan and once the state associated with that lives in the scan
descriptor, it made more sense to make dedicated
begin_scan()/rescan()/endscan() routines for BitmapHeapScan.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 129 ++++++-
src/backend/access/heap/heapam_handler.c | 306 ++++++++++++++--
src/backend/executor/nodeBitmapHeapscan.c | 410 +++-------------------
src/include/access/heapam.h | 62 +++-
src/include/access/relscan.h | 3 -
src/include/access/tableam.h | 74 ++--
src/include/executor/nodeBitmapHeapscan.h | 6 +
src/include/nodes/execnodes.h | 21 --
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 553 insertions(+), 459 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 10f2faaa60..a21ec92d71 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -938,6 +938,48 @@ continue_page:
* ----------------------------------------------------------------
*/
+TableScanDesc
+heap_beginscan_bm(Relation relation, Snapshot snapshot, uint32 flags)
+{
+ BitmapHeapScanDesc scan;
+
+ /*
+ * increment relation ref count while scanning relation
+ *
+ * This is just to make really sure the relcache entry won't go away while
+ * the scan has a pointer to it. Caller should be holding the rel open
+ * anyway, so this is redundant in all normal scenarios...
+ */
+ RelationIncrementReferenceCount(relation);
+ scan = (BitmapHeapScanDesc) palloc(sizeof(BitmapHeapScanDescData));
+
+ scan->heap_common.rs_base.rs_rd = relation;
+ scan->heap_common.rs_base.rs_snapshot = snapshot;
+ scan->heap_common.rs_base.rs_nkeys = 0;
+ scan->heap_common.rs_base.rs_flags = flags;
+ scan->heap_common.rs_base.rs_parallel = NULL;
+ scan->heap_common.rs_strategy = NULL;
+
+ Assert(snapshot && IsMVCCSnapshot(snapshot));
+
+ /* we only need to set this up once */
+ scan->heap_common.rs_ctup.t_tableOid = RelationGetRelid(relation);
+
+ scan->heap_common.rs_parallelworkerdata = NULL;
+ scan->heap_common.rs_base.rs_key = NULL;
+
+ initscan(&scan->heap_common, NULL, false);
+
+ scan->iterator.serial = NULL;
+ scan->iterator.parallel = NULL;
+ scan->vmbuffer = InvalidBuffer;
+ scan->pf_iterator.serial = NULL;
+ scan->pf_iterator.parallel = NULL;
+ scan->pvmbuffer = InvalidBuffer;
+
+ return (TableScanDesc) scan;
+}
+
TableScanDesc
heap_beginscan(Relation relation, Snapshot snapshot,
@@ -967,8 +1009,6 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
- scan->rs_vmbuffer = InvalidBuffer;
- scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1026,6 +1066,35 @@ heap_beginscan(Relation relation, Snapshot snapshot,
return (TableScanDesc) scan;
}
+/*
+ * Cleanup BitmapHeapScan table state
+ */
+void
+heap_endscan_bm(TableScanDesc sscan)
+{
+ BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
+
+ if (BufferIsValid(scan->heap_common.rs_cbuf))
+ ReleaseBuffer(scan->heap_common.rs_cbuf);
+
+ if (BufferIsValid(scan->vmbuffer))
+ ReleaseBuffer(scan->vmbuffer);
+
+ if (BufferIsValid(scan->pvmbuffer))
+ ReleaseBuffer(scan->pvmbuffer);
+
+ bhs_end_iterate(&scan->pf_iterator);
+ bhs_end_iterate(&scan->iterator);
+
+ /*
+ * decrement relation reference count and free scan descriptor storage
+ */
+ RelationDecrementReferenceCount(scan->heap_common.rs_base.rs_rd);
+
+ pfree(scan);
+}
+
+
void
heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
bool allow_strat, bool allow_sync, bool allow_pagemode)
@@ -1057,12 +1126,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
- if (BufferIsValid(scan->rs_vmbuffer))
- {
- ReleaseBuffer(scan->rs_vmbuffer);
- scan->rs_vmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1082,12 +1145,6 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
- if (BufferIsValid(scan->rs_vmbuffer))
- {
- ReleaseBuffer(scan->rs_vmbuffer);
- scan->rs_vmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1334,6 +1391,50 @@ heap_getnextslot_tidrange(TableScanDesc sscan, ScanDirection direction,
return true;
}
+void
+heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa, int pf_maximum)
+{
+ BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
+
+ if (BufferIsValid(scan->vmbuffer))
+ ReleaseBuffer(scan->vmbuffer);
+ scan->vmbuffer = InvalidBuffer;
+
+ if (BufferIsValid(scan->pvmbuffer))
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+
+ bhs_end_iterate(&scan->pf_iterator);
+ bhs_end_iterate(&scan->iterator);
+
+ scan->prefetch_maximum = 0;
+ scan->empty_tuples_pending = 0;
+ scan->pstate = NULL;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+ scan->pfblockno = InvalidBlockNumber;
+
+ bhs_begin_iterate(&scan->iterator,
+ tbm, dsa,
+ pstate ?
+ pstate->tbmiterator :
+ InvalidDsaPointer);
+#ifdef USE_PREFETCH
+ if (pf_maximum > 0)
+ {
+ bhs_begin_iterate(&scan->pf_iterator, tbm, dsa,
+ pstate ?
+ pstate->prefetch_iterator :
+ InvalidDsaPointer);
+ }
+#endif /* USE_PREFETCH */
+
+ scan->pstate = pstate;
+ scan->prefetch_maximum = pf_maximum;
+}
+
+
/*
* heap_fetch - retrieve tuple with given tid
*
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 5e57d19d91..c513cbd2da 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -60,6 +60,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanDesc scan);
+static inline void BitmapPrefetch(BitmapHeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2186,12 +2189,80 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* ------------------------------------------------------------------------
*/
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+static inline void
+BitmapAdjustPrefetchIterator(BitmapHeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ BitmapHeapIterator *prefetch_iterator = &scan->pf_iterator;
+ ParallelBitmapHeapState *pstate = scan->pstate;
+ TBMIterateResult *tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ scan->prefetch_pages--;
+ }
+ else if (!prefetch_iterator->exhausted)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ tbmpre = bhs_iterate(prefetch_iterator);
+ scan->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (scan->prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (!prefetch_iterator->exhausted)
+ {
+ tbmpre = bhs_iterate(prefetch_iterator);
+ scan->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
- BlockNumber *blockno, bool *recheck,
+heapam_scan_bitmap_next_block(TableScanDesc sscan,
+ bool *recheck,
long *lossy_pages, long *exact_pages)
{
- HeapScanDesc hscan = (HeapScanDesc) scan;
+ BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
+ HeapScanDesc hscan = &scan->heap_common;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
@@ -2201,19 +2272,21 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(scan);
+
do
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(&scan->rs_bhs_iterator);
+ Assert(!scan->iterator.exhausted);
+ tbmres = bhs_iterate(&scan->iterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
+ Assert(scan->empty_tuples_pending == 0);
return false;
}
@@ -2228,7 +2301,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
} while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
*recheck = tbmres->recheck;
/*
@@ -2236,15 +2308,15 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* heap, the bitmap entries don't need rechecking, and all tuples on the
* page are visible to our transaction.
*/
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ if (!(hscan->rs_base.rs_flags & SO_NEED_TUPLE) &&
!tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ VM_ALL_VISIBLE(hscan->rs_base.rs_rd, tbmres->blockno, &scan->vmbuffer))
{
/* can't be lossy in the skip_fetch case */
Assert(tbmres->ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ Assert(scan->empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ scan->empty_tuples_pending += tbmres->ntuples;
return true;
}
@@ -2255,18 +2327,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* Acquire pin on the target heap page, trading in any pin we held before.
*/
hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
+ hscan->rs_base.rs_rd,
block);
hscan->rs_cblock = block;
buffer = hscan->rs_cbuf;
- snapshot = scan->rs_snapshot;
+ snapshot = hscan->rs_base.rs_snapshot;
ntup = 0;
/*
* Prune and repair fragmentation for the whole page, if possible.
*/
- heap_page_prune_opt(scan->rs_rd, buffer);
+ heap_page_prune_opt(hscan->rs_base.rs_rd, buffer);
/*
* We must hold share lock on the buffer content while examining tuple
@@ -2294,7 +2366,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
HeapTupleData heapTuple;
ItemPointerSet(&tid, block, offnum);
- if (heap_hot_search_buffer(&tid, scan->rs_rd, buffer, snapshot,
+ if (heap_hot_search_buffer(&tid, hscan->rs_base.rs_rd, buffer, snapshot,
&heapTuple, NULL, true))
hscan->rs_vistuples[ntup++] = ItemPointerGetOffsetNumber(&tid);
}
@@ -2320,16 +2392,16 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
continue;
loctup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
loctup.t_len = ItemIdGetLength(lp);
- loctup.t_tableOid = scan->rs_rd->rd_id;
+ loctup.t_tableOid = hscan->rs_base.rs_rd->rd_id;
ItemPointerSet(&loctup.t_self, block, offnum);
valid = HeapTupleSatisfiesVisibility(&loctup, snapshot, buffer);
if (valid)
{
hscan->rs_vistuples[ntup++] = offnum;
- PredicateLockTID(scan->rs_rd, &loctup.t_self, snapshot,
+ PredicateLockTID(hscan->rs_base.rs_rd, &loctup.t_self, snapshot,
HeapTupleHeaderGetXmin(loctup.t_data));
}
- HeapCheckForSerializableConflictOut(valid, scan->rs_rd, &loctup,
+ HeapCheckForSerializableConflictOut(valid, hscan->rs_base.rs_rd, &loctup,
buffer, snapshot);
}
}
@@ -2344,6 +2416,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->pstate == NULL &&
+ !scan->pf_iterator.exhausted &&
+ scan->pfblockno < block)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(scan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2354,22 +2438,168 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+static inline void
+BitmapAdjustPrefetchTarget(BitmapHeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->pstate;
+ int prefetch_maximum = scan->prefetch_maximum;
+
+ if (pstate == NULL)
+ {
+ if (scan->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
+ else
+ scan->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+static inline void
+BitmapPrefetch(BitmapHeapScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = scan->pstate;
+ BitmapHeapIterator *prefetch_iterator = &scan->pf_iterator;
+ Relation rel = scan->heap_common.rs_base.rs_rd;
+
+ if (pstate == NULL)
+ {
+ if (!prefetch_iterator->exhausted)
+ {
+ while (scan->prefetch_pages < scan->prefetch_target)
+ {
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ bool skip_fetch;
+
+ if (tbmpre == NULL)
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ break;
+ }
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre->blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->heap_common.rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre->recheck &&
+ VM_ALL_VISIBLE(rel,
+ tbmpre->blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(rel, MAIN_FORKNUM, tbmpre->blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (!prefetch_iterator->exhausted)
+ {
+ while (1)
+ {
+ TBMIterateResult *tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ tbmpre = bhs_iterate(prefetch_iterator);
+ if (tbmpre == NULL)
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ break;
+ }
+
+ scan->pfblockno = tbmpre->blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->heap_common.rs_base.rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre->recheck &&
+ VM_ALL_VISIBLE(rel,
+ tbmpre->blockno,
+ &scan->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(rel, MAIN_FORKNUM, tbmpre->blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
{
- HeapScanDesc hscan = (HeapScanDesc) scan;
+ BitmapHeapScanDesc bscan = (BitmapHeapScanDesc) scan;
+ HeapScanDesc hscan = &bscan->heap_common;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ if (bscan->empty_tuples_pending > 0)
{
/*
* If we don't have to fetch the tuple, just return nulls.
*/
ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
+ bscan->empty_tuples_pending--;
return true;
}
@@ -2379,6 +2609,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!bscan->pstate)
+ {
+ if (bscan->prefetch_target < bscan->prefetch_maximum)
+ bscan->prefetch_target++;
+ }
+ else if (bscan->pstate->prefetch_target < bscan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&bscan->pstate->mutex);
+ if (bscan->pstate->prefetch_target < bscan->prefetch_maximum)
+ bscan->pstate->prefetch_target++;
+ SpinLockRelease(&bscan->pstate->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(bscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
@@ -2704,6 +2964,10 @@ static const TableAmRoutine heapam_methods = {
.scan_set_tidrange = heap_set_tidrange,
.scan_getnextslot_tidrange = heap_getnextslot_tidrange,
+ .scan_rescan_bm = heap_rescan_bm,
+ .scan_begin_bm = heap_beginscan_bm,
+ .scan_end_bm = heap_endscan_bm,
+
.parallelscan_estimate = table_block_parallelscan_estimate,
.parallelscan_initialize = table_block_parallelscan_initialize,
.parallelscan_reinitialize = table_block_parallelscan_reinitialize,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 70b560b8ee..d22a433f06 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,10 +51,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
/*
@@ -122,20 +118,9 @@ bhs_end_iterate(BitmapHeapIterator *iterator)
static TupleTableSlot *
BitmapHeapNext(BitmapHeapScanState *node)
{
- ExprContext *econtext;
- TableScanDesc scan;
- TIDBitmap *tbm;
- TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
- dsa_area *dsa = node->ss.ps.state->es_query_dsa;
-
- /*
- * extract necessary information from index scan node
- */
- econtext = node->ss.ps.ps_ExprContext;
- slot = node->ss.ss_ScanTupleSlot;
- scan = node->ss.ss_currentScanDesc;
- tbm = node->tbm;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ TableScanDesc scan = node->ss.ss_currentScanDesc;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -146,25 +131,37 @@ BitmapHeapNext(BitmapHeapScanState *node)
* prefetching. node->prefetch_pages tracks exactly how many pages ahead
* the prefetch iterator is. Also, node->prefetch_target tracks the
* desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
* a scan that stops after a few tuples because of a LIMIT.
*/
if (!node->initialized)
{
+ Relation rel = node->ss.ss_currentRelation;
+ bool pf_maximum = 0;
+ bool init_shared_state = false;
+ uint32 extra_flags = 0;
+
/*
* The leader will immediately come out of the function, but others
* will be blocked until leader populates the TBM and wakes them up.
*/
- bool init_shared_state = node->pstate ?
+ init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate || init_shared_state)
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(rel->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
{
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
+ node->tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
+ if (!node->tbm || !IsA(node->tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
- node->tbm = tbm;
if (init_shared_state)
{
@@ -173,113 +170,52 @@ BitmapHeapNext(BitmapHeapScanState *node)
* dsa_pointer of the iterator state which will be used by
* multiple processes to iterate jointly.
*/
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(node->tbm);
+
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
+ node->pstate->prefetch_iterator =
+ tbm_prepare_shared_iterate(node->tbm);
}
#endif
/* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
+ BitmapDoneInitializingSharedState(node->pstate);
}
}
/*
- * If this is the first scan of the underlying table, create the table
- * scan descriptor and begin the scan.
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
*/
- if (!scan)
- {
- uint32 extra_flags = 0;
-
- /*
- * We can potentially skip fetching heap pages if we do not need
- * any columns of the table, either for checking non-indexable
- * quals or for returning data. This test is a bit simplistic, as
- * it checks the stronger condition that there's no qual or return
- * tlist at all. But in most cases it's probably not worth working
- * harder than that.
- */
- if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
- extra_flags |= SO_NEED_TUPLE;
-
- scan = table_beginscan_bm(node->ss.ss_currentRelation,
- node->ss.ps.state->es_snapshot,
- 0,
- NULL,
- extra_flags);
-
- node->ss.ss_currentScanDesc = scan;
- }
-
- bhs_begin_iterate(&scan->rs_bhs_iterator, tbm, dsa,
- pstate ?
- pstate->tbmiterator :
- InvalidDsaPointer);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- bhs_begin_iterate(&node->pf_iterator, tbm, dsa,
- pstate ?
- pstate->prefetch_iterator :
- InvalidDsaPointer);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
-
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
+ scan = table_beginscan_bm(node->ss.ss_currentScanDesc,
+ rel,
+ node->ss.ps.state->es_snapshot,
+ extra_flags,
+ pf_maximum,
+ node->tbm,
+ node->pstate,
+ node->ss.ps.state->es_query_dsa);
+
+ node->ss.ss_currentScanDesc = scan;
node->initialized = true;
goto new_page;
}
+
for (;;)
{
while (table_scan_bitmap_next_tuple(scan, slot))
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
/*
* If we are using lossy info, we have to recheck the qual
@@ -303,23 +239,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
- if (!table_scan_bitmap_next_block(scan, &node->blockno, &node->recheck,
+ if (!table_scan_bitmap_next_block(scan, &node->recheck,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- !node->pf_iterator.exhausted &&
- node->pfblockno < node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -343,215 +265,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
- TBMIterateResult *tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (!prefetch_iterator->exhausted)
- {
- /* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (!prefetch_iterator->exhausted)
- {
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (!prefetch_iterator->exhausted)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
- bool skip_fetch;
-
- if (tbmpre == NULL)
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (!prefetch_iterator->exhausted)
- {
- while (1)
- {
- TBMIterateResult *tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- break;
- }
-
- node->pfblockno = tbmpre->blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
@@ -597,19 +310,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->ss.ss_currentScanDesc)
table_rescan(node->ss.ss_currentScanDesc, NULL);
- /* release bitmaps and buffers if any */
- if (node->pf_iterator.exhausted)
- bhs_end_iterate(&node->pf_iterator);
+ /* release bitmaps if any */
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -648,14 +354,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (!node->pf_iterator.exhausted)
- bhs_end_iterate(&node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -688,16 +390,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -737,13 +434,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
@@ -827,7 +517,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 750ea30852..73690d15c5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -76,22 +76,60 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* these fields only used in page-at-a-time mode and for bitmap scans */
+ int rs_cindex; /* current tuple's index in vistuples */
+ int rs_ntuples; /* number of visible tuples on page */
+ OffsetNumber rs_vistuples[MaxHeapTuplesPerPage]; /* their offsets */
+
+} HeapScanDescData;
+typedef struct HeapScanDescData *HeapScanDesc;
+
+typedef struct BitmapHeapScanDescData
+{
+ /* All the non-BitmapHeapScan specific members */
+ HeapScanDescData heap_common;
+
+ /*
+ * Members common to Parallel and Serial BitmapHeapScan
+ */
+ struct BitmapHeapIterator iterator;
+ struct BitmapHeapIterator pf_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
* fetching an all visible block, instead using the number of tuples per
* block reported by the bitmap to determine how many NULL-filled tuples
- * to return.
+ * to return. They are common to parallel and serial BitmapHeapScans
*/
- Buffer rs_vmbuffer;
- int rs_empty_tuples_pending;
+ Buffer vmbuffer;
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
+ int empty_tuples_pending;
+
+ /*
+ * Parallel-only members
+ */
+
+ struct ParallelBitmapHeapState *pstate;
+
+ /*
+ * Serial-only members
+ */
+
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
+} BitmapHeapScanDescData;
+
+typedef struct BitmapHeapScanDescData *BitmapHeapScanDesc;
- /* these fields only used in page-at-a-time mode and for bitmap scans */
- int rs_cindex; /* current tuple's index in vistuples */
- int rs_ntuples; /* number of visible tuples on page */
- OffsetNumber rs_vistuples[MaxHeapTuplesPerPage]; /* their offsets */
-} HeapScanDescData;
-typedef struct HeapScanDescData *HeapScanDesc;
/*
* Descriptor for fetches from heap via an index.
@@ -289,6 +327,12 @@ extern void heap_set_tidrange(TableScanDesc sscan, ItemPointer mintid,
extern bool heap_getnextslot_tidrange(TableScanDesc sscan,
ScanDirection direction,
TupleTableSlot *slot);
+extern TableScanDesc heap_beginscan_bm(Relation relation, Snapshot snapshot, uint32 flags);
+
+extern void heap_endscan_bm(TableScanDesc scan);
+extern void heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa, int pf_maximum);
+
extern bool heap_fetch(Relation relation, Snapshot snapshot,
HeapTuple tuple, Buffer *userbuf, bool keep_buf);
extern bool heap_hot_search_buffer(ItemPointer tid, Relation relation,
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index e520186b41..855b9558bf 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -41,9 +41,6 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
- /* Only used for Bitmap table scans */
- BitmapHeapIterator rs_bhs_iterator;
-
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 618ab2b449..138bf5f6ed 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -391,6 +391,23 @@ typedef struct TableAmRoutine
ScanDirection direction,
TupleTableSlot *slot);
+ /*
+ * TODO: add comment
+ */
+ TableScanDesc (*scan_begin_bm) (Relation rel,
+ Snapshot snapshot,
+ uint32 flags);
+
+ void (*scan_rescan_bm) (TableScanDesc scan, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa,
+ int pf_maximum);
+
+ /*
+ * Release resources and deallocate scan. If TableScanDesc.temp_snap,
+ * TableScanDesc.rs_snapshot needs to be unregistered.
+ */
+ void (*scan_end_bm) (TableScanDesc scan);
+
/* ------------------------------------------------------------------------
* Parallel table scan related functions.
* ------------------------------------------------------------------------
@@ -774,9 +791,9 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table
+ * scan. `scan` was started via table_beginscan_bm(). Return false if the
+ * bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
@@ -785,22 +802,11 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the bitmap is lossy for the selected
* block; otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- BlockNumber *blockno, bool *recheck,
+ bool *recheck,
long *lossy_pages, long *exact_pages);
/*
@@ -929,18 +935,26 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
}
/*
- * table_beginscan_bm is an alternative entry point for setting up a
- * TableScanDesc for a bitmap heap scan. Although that scan technology is
- * really quite unlike a standard seqscan, there is just enough commonality to
- * make it worth using the same data structure.
+ * table_beginscan_bm is the entry point for setting up a TableScanDesc for a
+ * bitmap heap scan.
*/
static inline TableScanDesc
-table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key, uint32 extra_flags)
+table_beginscan_bm(TableScanDesc scan, Relation rel, Snapshot snapshot,
+ uint32 extra_flags, int pf_maximum, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa)
{
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ scan = rel->rd_tableam->scan_begin_bm(rel, snapshot, flags);
+
+ scan->rs_rd->rd_tableam->scan_rescan_bm(scan, tbm, pstate, dsa, pf_maximum);
+
+ return scan;
}
/*
@@ -987,9 +1001,11 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
- !scan->rs_bhs_iterator.exhausted)
- bhs_end_iterate(&scan->rs_bhs_iterator);
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ scan->rs_rd->rd_tableam->scan_end_bm(scan);
+ return;
+ }
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1001,10 +1017,6 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
- !scan->rs_bhs_iterator.exhausted)
- bhs_end_iterate(&scan->rs_bhs_iterator);
-
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1979,7 +1991,7 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- BlockNumber *blockno, bool *recheck,
+ bool *recheck,
long *lossy_pages,
long *exact_pages)
{
@@ -1992,7 +2004,7 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- blockno, recheck,
+ recheck,
lossy_pages, exact_pages);
}
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index 7064f54686..f3c92f7a8a 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -28,6 +28,12 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
ParallelContext *pcxt);
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+ bool exhausted;
+} BitmapHeapIterator;
extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
extern void bhs_begin_iterate(BitmapHeapIterator *iterator, TIDBitmap *tbm,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 714eb3e534..49ce4ff9a7 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1787,30 +1787,16 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
-typedef struct BitmapHeapIterator
-{
- struct TBMIterator *serial;
- struct TBMSharedIterator *parallel;
- bool exhausted;
-} BitmapHeapIterator;
-
/* ----------------
* BitmapHeapScanState information
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1818,18 +1804,11 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- BitmapHeapIterator pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2348a8793e..443b01c053 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -263,6 +263,7 @@ BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
+BitmapHeapScanDesc
BitmapIndexScan
BitmapIndexScanState
BitmapOr
--
2.40.1
[text/x-diff] v15-0012-Move-BitmapHeapScan-initialization-to-helper.patch (6.9K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/13-v15-0012-Move-BitmapHeapScan-initialization-to-helper.patch)
download | inline diff:
From c1c9537f045ba7293416711347c7113fa8ec33f9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 02:50:42 -0400
Subject: [PATCH v15 12/13] Move BitmapHeapScan initialization to helper
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 170 ++++++++++++----------
1 file changed, 92 insertions(+), 78 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index d22a433f06..aaeece6697 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -109,6 +109,94 @@ bhs_end_iterate(BitmapHeapIterator *iterator)
}
+ /*
+ * If we haven't yet performed the underlying index scan, do it, and begin
+ * the iteration over the bitmap.
+ *
+ * For prefetching, we use *two* iterators, one for the pages we are actually
+ * scanning and another that runs ahead of the first for prefetching.
+ * node->prefetch_pages tracks exactly how many pages ahead the prefetch
+ * iterator is. Also, node->prefetch_target tracks the desired prefetch
+ * distance, which starts small and increases up to the
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in a
+ * scan that stops after a few tuples because of a LIMIT.
+ */
+static void
+BitmapHeapInitialize(BitmapHeapScanState *node)
+{
+ Relation rel = node->ss.ss_currentRelation;
+ bool pf_maximum = 0;
+ bool init_shared_state = false;
+ uint32 extra_flags = 0;
+
+ Assert(!node->initialized);
+
+ /*
+ * The leader will immediately come out of the function, but others will
+ * be blocked until leader populates the TBM and wakes them up.
+ */
+ init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
+
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(rel->rd_rel->reltablespace);
+#endif
+
+ if (!node->pstate || init_shared_state)
+ {
+ node->tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
+
+ if (!node->tbm || !IsA(node->tbm, TIDBitmap))
+ elog(ERROR, "unrecognized result from subplan");
+
+ if (init_shared_state)
+ {
+ /*
+ * Prepare to iterate over the TBM. This will return the
+ * dsa_pointer of the iterator state which will be used by
+ * multiple processes to iterate jointly.
+ */
+ node->pstate->tbmiterator = tbm_prepare_shared_iterate(node->tbm);
+
+#ifdef USE_PREFETCH
+ if (pf_maximum > 0)
+ {
+ node->pstate->prefetch_iterator =
+ tbm_prepare_shared_iterate(node->tbm);
+ }
+#endif
+ /* We have initialized the shared state so wake up others. */
+ BitmapDoneInitializingSharedState(node->pstate);
+ }
+ }
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or for
+ * returning data. This test is a bit simplistic, as it checks the
+ * stronger condition that there's no qual or return tlist at all. But in
+ * most cases it's probably not worth working harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
+ node->ss.ss_currentScanDesc = table_beginscan_bm(node->ss.ss_currentScanDesc,
+ rel,
+ node->ss.ps.state->es_snapshot,
+ extra_flags,
+ pf_maximum,
+ node->tbm,
+ node->pstate,
+ node->ss.ps.state->es_query_dsa);
+
+ node->initialized = true;
+}
+
+
/* ----------------------------------------------------------------
* BitmapHeapNext
*
@@ -124,88 +212,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* If we haven't yet performed the underlying index scan, do it, and begin
- * the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
+ * the iteration over the bitmap. This happens on rescan as well.
*/
if (!node->initialized)
{
- Relation rel = node->ss.ss_currentRelation;
- bool pf_maximum = 0;
- bool init_shared_state = false;
- uint32 extra_flags = 0;
-
- /*
- * The leader will immediately come out of the function, but others
- * will be blocked until leader populates the TBM and wakes them up.
- */
- init_shared_state = node->pstate ?
- BitmapShouldInitializeSharedState(node->pstate) : false;
-
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(rel->rd_rel->reltablespace);
-#endif
-
- if (!node->pstate || init_shared_state)
- {
- node->tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
-
- if (!node->tbm || !IsA(node->tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- if (init_shared_state)
- {
- /*
- * Prepare to iterate over the TBM. This will return the
- * dsa_pointer of the iterator state which will be used by
- * multiple processes to iterate jointly.
- */
- node->pstate->tbmiterator = tbm_prepare_shared_iterate(node->tbm);
-
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- node->pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(node->tbm);
- }
-#endif
- /* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(node->pstate);
- }
- }
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
- extra_flags |= SO_NEED_TUPLE;
-
- scan = table_beginscan_bm(node->ss.ss_currentScanDesc,
- rel,
- node->ss.ps.state->es_snapshot,
- extra_flags,
- pf_maximum,
- node->tbm,
- node->pstate,
- node->ss.ps.state->es_query_dsa);
-
- node->ss.ss_currentScanDesc = scan;
- node->initialized = true;
+ BitmapHeapInitialize(node);
+ /* We may have a new scan descriptor */
+ scan = node->ss.ss_currentScanDesc;
goto new_page;
}
--
2.40.1
[text/x-diff] v15-0013-Remove-table_scan_bitmap_next_block.patch (12.4K, ../../20240405080634.d7c23f46xjhhxf5q@liskov/14-v15-0013-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 869023793654571cba6af0ccfd2fac9950ccc635 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 22 Mar 2024 15:43:10 -0400
Subject: [PATCH v15 13/13] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block().
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 48 ++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 61 +++++++++----------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 74 +++++------------------
6 files changed, 73 insertions(+), 116 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index a21ec92d71..9c98109a16 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -324,6 +324,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index c513cbd2da..d7f3bb960f 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2183,12 +2183,6 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
-
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
- */
-
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
*
@@ -2222,8 +2216,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->prefetch_maximum > 0)
{
@@ -2583,9 +2577,15 @@ BitmapPrefetch(BitmapHeapScanDesc scan)
#endif /* USE_PREFETCH */
}
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
BitmapHeapScanDesc bscan = (BitmapHeapScanDesc) scan;
HeapScanDesc hscan = &bscan->heap_common;
@@ -2593,21 +2593,28 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
- if (bscan->empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- bscan->empty_tuples_pending--;
- return true;
- }
+ if (bscan->empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ bscan->empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -3008,7 +3015,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index 55b8caeadf..0b87530a9c 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -90,8 +90,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index aaeece6697..48086aae1d 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -206,58 +206,53 @@ BitmapHeapInitialize(BitmapHeapScanState *node)
static TupleTableSlot *
BitmapHeapNext(BitmapHeapScanState *node)
{
- ExprContext *econtext = node->ss.ps.ps_ExprContext;
- TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
- TableScanDesc scan = node->ss.ss_currentScanDesc;
+ ExprContext *econtext;
+ TupleTableSlot *slot;
+ TableScanDesc scan;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap. This happens on rescan as well.
*/
if (!node->initialized)
- {
BitmapHeapInitialize(node);
- /* We may have a new scan descriptor */
- scan = node->ss.ss_currentScanDesc;
- goto new_page;
- }
+ /*
+ * BitmapHeapInitialize() may make the scan descriptor so don't get it
+ * from the node until after calling it.
+ */
+ econtext = node->ss.ps.ps_ExprContext;
+ slot = node->ss.ss_ScanTupleSlot;
+ scan = node->ss.ss_currentScanDesc;
- for (;;)
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->recheck,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 138bf5f6ed..e4d0c4b0ed 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -790,34 +790,20 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples as part of a bitmap table
- * scan. `scan` was started via table_beginscan_bm(). Return false if the
- * bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the bitmap is lossy for the selected
- * block; otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- bool *recheck,
- long *lossy_pages, long *exact_pages);
-
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -1980,45 +1966,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented is the block's representation in the bitmap is lossy; otherwise,
- * exact_pages is incremented.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- bool *recheck,
- long *lossy_pages,
- long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- recheck,
- lossy_pages, exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2029,7 +1983,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-31 15:45 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-03 22:57 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-04 14:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-04-05 08:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
@ 2024-04-05 23:53 ` Melanie Plageman <[email protected]>
2024-04-06 00:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
0 siblings, 1 reply; 21+ messages in thread
From: Melanie Plageman @ 2024-04-05 23:53 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Thomas Munro <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On Fri, Apr 05, 2024 at 04:06:34AM -0400, Melanie Plageman wrote:
> On Thu, Apr 04, 2024 at 04:35:45PM +0200, Tomas Vondra wrote:
> >
> >
> > On 4/4/24 00:57, Melanie Plageman wrote:
> > > On Sun, Mar 31, 2024 at 11:45:51AM -0400, Melanie Plageman wrote:
> > >> On Fri, Mar 29, 2024 at 12:05:15PM +0100, Tomas Vondra wrote:
> > >>>
> > >>>
> > >>> On 3/29/24 02:12, Thomas Munro wrote:
> > >>>> On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
> > >>>> <[email protected]> wrote:
> > >>>>> I think there's some sort of bug, triggering this assert in heapam
> > >>>>>
> > >>>>> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
> > >>>>
> > >>>> Thanks for the repro. I can't seem to reproduce it (still trying) but
> > >>>> I assume this is with Melanie's v11 patch set which had
> > >>>> v11-0016-v10-Read-Stream-API.patch.
> > >>>>
> > >>>> Would you mind removing that commit and instead applying the v13
> > >>>> stream_read.c patches[1]? v10 stream_read.c was a little confused
> > >>>> about random I/O combining, which I fixed with a small adjustment to
> > >>>> the conditions for the "if" statement right at the end of
> > >>>> read_stream_look_ahead(). Sorry about that. The fixed version, with
> > >>>> eic=4, with your test query using WHERE a < a, ends its scan with:
> > >>>>
> > >>>
> > >>> I'll give that a try. Unfortunately unfortunately the v11 still has the
> > >>> problem I reported about a week ago:
> > >>>
> > >>> ERROR: prefetch and main iterators are out of sync
> > >>>
> > >>> So I can't run the full benchmarks :-( but master vs. streaming read API
> > >>> should work, I think.
> > >>
> > >> Odd, I didn't notice you reporting this ERROR popping up. Now that I
> > >> take a look, v11 (at least, maybe also v10) had this very sill mistake:
> > >>
> > >> if (scan->bm_parallel == NULL &&
> > >> scan->rs_pf_bhs_iterator &&
> > >> hscan->pfblockno > hscan->rs_base.blockno)
> > >> elog(ERROR, "prefetch and main iterators are out of sync");
> > >>
> > >> It errors out if the prefetch block is ahead of the current block --
> > >> which is the opposite of what we want. I've fixed this in attached v12.
> > >>
> > >> This version also has v13 of the streaming read API. I noticed one
> > >> mistake in my bitmapheap scan streaming read user -- it freed the
> > >> streaming read object at the wrong time. I don't know if this was
> > >> causing any other issues, but it at least is fixed in this version.
> > >
> > > Attached v13 is rebased over master (which includes the streaming read
> > > API now). I also reset the streaming read object on rescan instead of
> > > creating a new one each time.
> > >
> > > I don't know how much chance any of this has of going in to 17 now, but
> > > I thought I would start looking into the regression repro Tomas provided
> > > in [1].
> > >
> >
> > My personal opinion is that we should try to get in as many of the the
> > refactoring patches as possible, but I think it's probably too late for
> > the actual switch to the streaming API.
>
> Cool. In the attached v15, I have dropped all commits that are related
> to the streaming read API and included *only* commits that are
> beneficial to master. A few of the commits are merged or reordered as
> well.
>
> While going through the commits with this new goal in mind (forget about
> the streaming read API for now), I realized that it doesn't make much
> sense to just eliminate the layering violation for the current block and
> leave it there for the prefetch block. I had de-prioritized solving this
> when I thought we would just delete the prefetch code and replace it
> with the streaming read.
>
> Now that we aren't doing that, I've spent the day trying to resolve the
> issues with pushing the prefetch code into heapam.c that I cited in [1].
> 0010 - 0013 are the result of this. They are not very polished yet and
> need more cleanup and review (especially 0011, which is probably too
> large), but I am happy with the solution I came up with.
>
> Basically, there are too many members needed for bitmap heap scan to put
> them all in the HeapScanDescData (don't want to bloat it). So, I've made
> a new BitmapHeapScanDescData and associated begin/rescan/end() functions
>
> In the end, with all patches applied, BitmapHeapNext() loops invoking
> table_scan_bitmap_next_tuple() and table AMs can implement that however
> they choose.
>
> > > I'm also not sure if I should try and group the commits into fewer
> > > commits now or wait until I have some idea of whether or not the
> > > approach in 0013 and 0014 is worth pursuing.
> > >
> >
> > You mean whether to pursue the approach in general, or for v17? I think
> > it looks like the right approach, but for v17 see above :-(
> >
> > As for merging, I wouldn't do that. I looked at the commits and while
> > some of them seem somewhat "trivial", I really like how you organized
> > the commits, and kept those that just "move" code around, and those that
> > actually change stuff. It's much easier to understand, IMO.
> >
> > I went through the first ~10 commits, and added some review - either as
> > a separate commit, when possible, in the code as XXX comment, and also
> > in the commit message. The code tweaks are utterly trivial (whitespace
> > or indentation to make the line shorter). It shouldn't take much time to
> > deal with those, I think.
>
> Attached v15 incorporates your v14-0002-review.
>
> For your v14-0008-review, I actually ended up removing that commit
> because once I removed everything that was for streaming read API, it
> became redundant with another commit.
>
> For your v14-0010-review, we actually can't easily get rid of those
> local variables because we make the iterators before we make the scan
> descriptors and the commit following that commit moves the iterators
> from the BitmapHeapScanState to the scan descriptor.
>
> > I think the main focus should be updating the commit messages. If it was
> > only a single patch, I'd probably try to write the messages myself, but
> > with this many patches it'd be great if you could update those and I'll
> > review that before commit.
>
> I did my best to update the commit messages to be less specific and more
> focused on "why should I care". I found myself wanting to explain why I
> implemented something the way I did and then getting back into the
> implementation details again. I'm not sure if I suceeded in having less
> details and more substance.
>
> > I always struggle with writing commit messages myself, and it takes me
> > ages to write a good one (well, I think the message is good, but who
> > knows ...). But I think a good message should be concise enough to
> > explain what and why it's done. It may reference a thread for all the
> > gory details, but the basic reasoning should be in the commit message.
> > For example the message for "BitmapPrefetch use prefetch block recheck
> > for skip fetch" now says that it "makes more sense to do X" but does not
> > really say why that's the case. The linked message does, but it'd be
> > good to have that in the message (because how would I know how much of
> > the thread to read?).
>
> I fixed that particular one. I tried to take that feedback and apply it
> to other commit messages. I don't know how successful I was...
>
> > Also, it'd be very helpful if you could update the author & reviewed-by
> > fields. I'll review those before commit, ofc, but I admit I lost track
> > of who reviewed which part.
>
> I have updated reviewers. I didn't add reviewers on the ones that
> haven't really been reviewed yet.
>
> > I'd focus on the first ~8-9 commits or so for now, we can commit more if
> > things go reasonably well.
>
> Sounds good. I will spend cleanup time on 0010-0013 tomorrow but would
> love to know if you agree with the direction before I spend more time.
In attached v16, I've split out 0010-0013 into 0011-0017. I think it is
much easier to understand.
While I was doing that, I realized that I should remove the call to
table_rescan() from ExecReScanBitmapHeapScan() and just rely on the new
table_rescan_bm() invoked from BitmapHeapNext(). That is done in the
attached.
0010-0018 still need comments updated but I focused on getting the split
out, reviewable version of them ready. I'll add comments (especially to
0011 table AM functions) tomorrow. I also have to double-check if I
should add any asserts for table AMs about having implemented all of the
new begin/re/endscan() functions.
- Melanie
Attachments:
[text/x-diff] v16-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch (2.4K, ../../20240405235330.v4didvweb6isodtk@liskov/2-v16-0001-BitmapHeapScan-begin-scan-after-bitmap-creation.patch)
download | inline diff:
From c6ebe66dcf5ff961ef3d2c66a26b21d2246a499b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:50:29 -0500
Subject: [PATCH v16 01/18] BitmapHeapScan begin scan after bitmap creation
It makes more sense for BitmapHeapScan to scan the index, build the
bitmap, and then begin the scan on the underlying table.
This is primarily a cosmetic change for now, but later commits will pass
parameters to table_beginscan_bm() that are unavailable in
ExecInitBitmapHeapScan().
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 27 +++++++++++++++++------
1 file changed, 20 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index cee7f45aab..c8c466e3c5 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -178,6 +178,21 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
#endif /* USE_PREFETCH */
}
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ {
+ scan = table_beginscan_bm(node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ 0,
+ NULL);
+
+ node->ss.ss_currentScanDesc = scan;
+ }
+
node->initialized = true;
}
@@ -601,7 +616,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* rescan to release any page pin */
- table_rescan(node->ss.ss_currentScanDesc, NULL);
+ if (node->ss.ss_currentScanDesc)
+ table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
if (node->tbmiterator)
@@ -678,7 +694,9 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* close heap scan
*/
- table_endscan(scanDesc);
+ if (scanDesc)
+ table_endscan(scanDesc);
+
}
/* ----------------------------------------------------------------
@@ -783,11 +801,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ss_currentRelation = currentRelation;
- scanstate->ss.ss_currentScanDesc = table_beginscan_bm(currentRelation,
- estate->es_snapshot,
- 0,
- NULL);
-
/*
* all done.
*/
--
2.40.1
[text/x-diff] v16-0002-BitmapHeapScan-set-can_skip_fetch-later.patch (2.3K, ../../20240405235330.v4didvweb6isodtk@liskov/3-v16-0002-BitmapHeapScan-set-can_skip_fetch-later.patch)
download | inline diff:
From 770bf2e20dfd3c29c5fc4937d78ec2e26d3f78d4 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 14:38:41 -0500
Subject: [PATCH v16 02/18] BitmapHeapScan set can_skip_fetch later
Set BitmapHeapScanState->can_skip_fetch in BitmapHeapNext() instead of
in ExecInitBitmapHeapScan(). This is a preliminary step to pushing the
skip fetch optimization into heap AM code.
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index c8c466e3c5..2148a21531 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,6 +105,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
+ node->ss.ps.plan->targetlist == NIL);
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -743,16 +753,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or for
- * returning data. This test is a bit simplistic, as it checks the
- * stronger condition that there's no qual or return tlist at all. But in
- * most cases it's probably not worth working harder than that.
- */
- scanstate->can_skip_fetch = (node->scan.plan.qual == NIL &&
- node->scan.plan.targetlist == NIL);
+ scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
--
2.40.1
[text/x-diff] v16-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch (14.7K, ../../20240405235330.v4didvweb6isodtk@liskov/4-v16-0003-Push-BitmapHeapScan-skip-fetch-optimization-into.patch)
download | inline diff:
From c5d36fc7f201d8e01f68a00b235872165fd0ec94 Mon Sep 17 00:00:00 2001
From: Tomas Vondra <[email protected]>
Date: Thu, 4 Apr 2024 15:34:25 +0200
Subject: [PATCH v16 03/18] Push BitmapHeapScan skip fetch optimization into
table AM
7c70996ebf0949b142 introduced an optimization to allow bitmap table
scans to skip fetching a block from the heap if none of the underlying
data was needed and the block is marked all visible in the visibility
map. With the addition of table AMs, a FIXME was added to this code
indicating that it should be pushed into table AM-specific code, as not
all table AMs may use a visibility map in the same way.
Resolve this FIXME for the current block. The layering violation is
still present in BitmapHeapScans's prefetching code, which uses the
visibility map to decide whether or not to prefetch a block. However,
this will be fixed in an upcoming commit.
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 14 +++
src/backend/access/heap/heapam_handler.c | 29 +++++
src/backend/executor/nodeBitmapHeapscan.c | 124 +++++++---------------
src/include/access/heapam.h | 10 ++
src/include/access/tableam.h | 11 +-
src/include/nodes/execnodes.h | 8 +-
6 files changed, 102 insertions(+), 94 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index dada2ecd1e..10f2faaa60 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -967,6 +967,8 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
+ scan->rs_vmbuffer = InvalidBuffer;
+ scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1055,6 +1057,12 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* reinitialize scan descriptor
*/
@@ -1074,6 +1082,12 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
+ if (BufferIsValid(scan->rs_vmbuffer))
+ {
+ ReleaseBuffer(scan->rs_vmbuffer);
+ scan->rs_vmbuffer = InvalidBuffer;
+ }
+
/*
* decrement relation reference count and free scan descriptor storage
*/
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 3e7a6b5548..929b2cf8e7 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -27,6 +27,7 @@
#include "access/syncscan.h"
#include "access/tableam.h"
#include "access/tsmapi.h"
+#include "access/visibilitymap.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/index.h"
@@ -2198,6 +2199,24 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ /*
+ * We can skip fetching the heap page if we don't need any fields from the
+ * heap, the bitmap entries don't need rechecking, and all tuples on the
+ * page are visible to our transaction.
+ */
+ if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmres->recheck &&
+ VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ {
+ /* can't be lossy in the skip_fetch case */
+ Assert(tbmres->ntuples >= 0);
+ Assert(hscan->rs_empty_tuples_pending >= 0);
+
+ hscan->rs_empty_tuples_pending += tbmres->ntuples;
+
+ return true;
+ }
+
/*
* Ignore any claimed entries past what we think is the end of the
* relation. It may have been extended after the start of our scan (we
@@ -2310,6 +2329,16 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
+ if (hscan->rs_empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ hscan->rs_empty_tuples_pending--;
+ return true;
+ }
+
/*
* Out of range? If so, nothing more to look at on this page
*/
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2148a21531..a8bc5dec53 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -105,16 +105,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- node->can_skip_fetch = (node->ss.ps.plan->qual == NIL &&
- node->ss.ps.plan->targetlist == NIL);
-
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -195,10 +185,24 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!scan)
{
+ uint32 extra_flags = 0;
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need
+ * any columns of the table, either for checking non-indexable
+ * quals or for returning data. This test is a bit simplistic, as
+ * it checks the stronger condition that there's no qual or return
+ * tlist at all. But in most cases it's probably not worth working
+ * harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
scan = table_beginscan_bm(node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
0,
- NULL);
+ NULL,
+ extra_flags);
node->ss.ss_currentScanDesc = scan;
}
@@ -208,7 +212,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool skip_fetch;
+ bool valid;
CHECK_FOR_INTERRUPTS();
@@ -229,37 +233,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres);
+ valid = table_scan_bitmap_next_block(scan, tbmres);
+
if (tbmres->ntuples >= 0)
node->exact_pages++;
else
node->lossy_pages++;
- /*
- * We can skip fetching the heap page if we don't need any fields
- * from the heap, and the bitmap entries don't need rechecking,
- * and all tuples on the page are visible to our transaction.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
- */
- skip_fetch = (node->can_skip_fetch &&
- !tbmres->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmres->blockno,
- &node->vmbuffer));
-
- if (skip_fetch)
- {
- /* can't be lossy in the skip_fetch case */
- Assert(tbmres->ntuples >= 0);
-
- /*
- * The number of tuples on this page is put into
- * node->return_empty_tuples.
- */
- node->return_empty_tuples = tbmres->ntuples;
- }
- else if (!table_scan_bitmap_next_block(scan, tbmres))
+ if (!valid)
{
/* AM doesn't think this block is valid, skip */
continue;
@@ -302,52 +283,33 @@ BitmapHeapNext(BitmapHeapScanState *node)
* should happen only when we have determined there is still something
* to do on the current page, else we may uselessly prefetch the same
* page we are just about to request for real.
- *
- * XXX: It's a layering violation that we do these checks above
- * tableam, they should probably moved below it at some point.
*/
BitmapPrefetch(node, scan);
- if (node->return_empty_tuples > 0)
+ /*
+ * Attempt to fetch tuple from AM.
+ */
+ if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
{
- /*
- * If we don't have to fetch the tuple, just return nulls.
- */
- ExecStoreAllNullTuple(slot);
-
- if (--node->return_empty_tuples == 0)
- {
- /* no more tuples to return in the next round */
- node->tbmres = tbmres = NULL;
- }
+ /* nothing more to look at on this page */
+ node->tbmres = tbmres = NULL;
+ continue;
}
- else
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (tbmres->recheck)
{
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
continue;
}
-
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
}
/* OK to return this tuple */
@@ -519,7 +481,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* it did for the current heap page; which is not a certainty
* but is true in many cases.
*/
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -570,7 +532,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
}
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (node->can_skip_fetch &&
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
(node->tbmres ? !node->tbmres->recheck : false) &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
@@ -640,8 +602,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
@@ -651,7 +611,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->initialized = false;
node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
- node->vmbuffer = InvalidBuffer;
node->pvmbuffer = InvalidBuffer;
ExecScanReScan(&node->ss);
@@ -696,8 +655,6 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
- if (node->vmbuffer != InvalidBuffer)
- ReleaseBuffer(node->vmbuffer);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
@@ -741,8 +698,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->tbm = NULL;
scanstate->tbmiterator = NULL;
scanstate->tbmres = NULL;
- scanstate->return_empty_tuples = 0;
- scanstate->vmbuffer = InvalidBuffer;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -753,7 +708,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
- scanstate->can_skip_fetch = false;
/*
* Miscellaneous initialization
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 2765efc4e5..750ea30852 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -76,6 +76,16 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /*
+ * These fields are only used for bitmap scans for the "skip fetch"
+ * optimization. Bitmap scans needing no fields from the heap may skip
+ * fetching an all visible block, instead using the number of tuples per
+ * block reported by the bitmap to determine how many NULL-filled tuples
+ * to return.
+ */
+ Buffer rs_vmbuffer;
+ int rs_empty_tuples_pending;
+
/* these fields only used in page-at-a-time mode and for bitmap scans */
int rs_cindex; /* current tuple's index in vistuples */
int rs_ntuples; /* number of visible tuples on page */
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e7eeb75409..55f1139757 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -63,6 +63,13 @@ typedef enum ScanOptions
/* unregister snapshot at scan end? */
SO_TEMP_SNAPSHOT = 1 << 9,
+
+ /*
+ * At the discretion of the table AM, bitmap table scans may be able to
+ * skip fetching a block from the table if none of the table data is
+ * needed. If table data may be needed, set SO_NEED_TUPLE.
+ */
+ SO_NEED_TUPLE = 1 << 10,
} ScanOptions;
/*
@@ -937,9 +944,9 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key)
+ int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE;
+ uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e57ebd7288..fa2f70b7a4 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1794,10 +1794,7 @@ typedef struct ParallelBitmapHeapState
* tbm bitmap obtained from child index scan(s)
* tbmiterator iterator for scanning current pages
* tbmres current-page data
- * can_skip_fetch can we potentially skip tuple fetches in this scan?
- * return_empty_tuples number of empty tuples to return
- * vmbuffer buffer for visibility-map lookups
- * pvmbuffer ditto, for prefetched pages
+ * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
* prefetch_iterator iterator for prefetching ahead of current page
@@ -1817,9 +1814,6 @@ typedef struct BitmapHeapScanState
TIDBitmap *tbm;
TBMIterator *tbmiterator;
TBMIterateResult *tbmres;
- bool can_skip_fetch;
- int return_empty_tuples;
- Buffer vmbuffer;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
--
2.40.1
[text/x-diff] v16-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch (2.4K, ../../20240405235330.v4didvweb6isodtk@liskov/5-v16-0004-BitmapPrefetch-use-prefetch-block-recheck-for-sk.patch)
download | inline diff:
From 796e139454a8863c12235d248d76f313da6ef002 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:03:24 -0500
Subject: [PATCH v16 04/18] BitmapPrefetch use prefetch block recheck for skip
fetch
As of 7c70996ebf0949b142a9, BitmapPrefetch() used the recheck flag for
the current block to determine whether or not it could skip prefetching
the proposed prefetch block. This doesn't seem right. We should use the
prefetch block's recheck flag to determine whether or not to prefetch
it. The current block's recheck flag has no relationship to the prefetch
block's recheck flag. See this [1] thread on hackers reporting the
issue.
[1] https://www.postgresql.org/message-id/CAAKRu_bxrXeZ2rCnY8LyeC2Ls88KpjWrQ%2BopUrXDRXdcfwFZGA%40mail.gmail.com
Author: Melanie Plageman
Reviewed-by: Andres Freund, Heikki Linnakangas, Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index a8bc5dec53..2e2cec8b3b 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -475,14 +475,9 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* skip this prefetch call, but continue to run the prefetch
* logic normally. (Would it be better not to increment
* prefetch_pages?)
- *
- * This depends on the assumption that the index AM will
- * report the same recheck flag for this future heap page as
- * it did for the current heap page; which is not a certainty
- * but is true in many cases.
*/
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
@@ -533,7 +528,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- (node->tbmres ? !node->tbmres->recheck : false) &&
+ !tbmpre->recheck &&
VM_ALL_VISIBLE(node->ss.ss_currentRelation,
tbmpre->blockno,
&node->pvmbuffer));
--
2.40.1
[text/x-diff] v16-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch (2.5K, ../../20240405235330.v4didvweb6isodtk@liskov/6-v16-0005-Update-BitmapAdjustPrefetchIterator-parameter-ty.patch)
download | inline diff:
From 2f9abb4c09528925060b7613fe7d3e52d04d8bea Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 19:04:48 -0500
Subject: [PATCH v16 05/18] Update BitmapAdjustPrefetchIterator parameter type
to BlockNumber
BitmapAdjustPrefetchIterator() only used the blockno member of the
passed in TBMIterateResult to ensure that the prefetch iterator and
regular iterator stay in sync. Pass it the BlockNumber only. This will
allow us to move away from using the TBMIterateResult outside of table
AM specific code.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 2e2cec8b3b..795a893035 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -52,7 +52,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres);
+ BlockNumber blockno);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -231,7 +231,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
break;
}
- BitmapAdjustPrefetchIterator(node, tbmres);
+ BitmapAdjustPrefetchIterator(node, tbmres->blockno);
valid = table_scan_bitmap_next_block(scan, tbmres);
@@ -342,7 +342,7 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
*/
static inline void
BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- TBMIterateResult *tbmres)
+ BlockNumber blockno)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
@@ -361,7 +361,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
/* Do not let the prefetch iterator get behind the main one */
TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
- if (tbmpre == NULL || tbmpre->blockno != tbmres->blockno)
+ if (tbmpre == NULL || tbmpre->blockno != blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
}
return;
--
2.40.1
[text/x-diff] v16-0006-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch (3.1K, ../../20240405235330.v4didvweb6isodtk@liskov/7-v16-0006-Reduce-scope-of-BitmapHeapScan-tbmiterator-local.patch)
download | inline diff:
From d5b31a8d8e73fc332748856e3bcb52654d3b134e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Tue, 13 Feb 2024 10:17:47 -0500
Subject: [PATCH v16 06/18] Reduce scope of BitmapHeapScan tbmiterator local
variables
A future patch will change where tbmiterators are initialized and saved.
To simplify that diff, move them into a tighter scope.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 795a893035..8403be84f3 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -71,8 +71,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
@@ -85,10 +83,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- if (pstate == NULL)
- tbmiterator = node->tbmiterator;
- else
- shared_tbmiterator = node->shared_tbmiterator;
tbmres = node->tbmres;
/*
@@ -105,6 +99,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ TBMIterator *tbmiterator = NULL;
+ TBMSharedIterator *shared_tbmiterator = NULL;
+
if (!pstate)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -113,7 +110,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
elog(ERROR, "unrecognized result from subplan");
node->tbm = tbm;
- node->tbmiterator = tbmiterator = tbm_begin_iterate(tbm);
+ tbmiterator = tbm_begin_iterate(tbm);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -166,8 +163,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
/* Allocate a private iterator and attach the shared state to it */
- node->shared_tbmiterator = shared_tbmiterator =
- tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
+ shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
@@ -207,6 +203,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
+ node->tbmiterator = tbmiterator;
+ node->shared_tbmiterator = shared_tbmiterator;
node->initialized = true;
}
@@ -222,9 +220,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
if (tbmres == NULL)
{
if (!pstate)
- node->tbmres = tbmres = tbm_iterate(tbmiterator);
+ node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
else
- node->tbmres = tbmres = tbm_shared_iterate(shared_tbmiterator);
+ node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
--
2.40.1
[text/x-diff] v16-0007-table_scan_bitmap_next_block-counts-lossy-and-ex.patch (5.1K, ../../20240405235330.v4didvweb6isodtk@liskov/8-v16-0007-table_scan_bitmap_next_block-counts-lossy-and-ex.patch)
download | inline diff:
From b0043ddcc0a4d18746f80aa90c3b6dc32304b92f Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 26 Feb 2024 20:34:07 -0500
Subject: [PATCH v16 07/18] table_scan_bitmap_next_block counts lossy and exact
pages
Future commits will remove the TBMIterateResult from BitmapHeapNext() --
pushing it into the table AM-specific code. So the table AM must keep
track of whether or not individual TBMIterateResult's blocks were
represented lossily in the bitmap for the purposes of EXPLAIN counters.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 8 +++++++-
src/backend/executor/nodeBitmapHeapscan.c | 13 +++----------
src/include/access/tableam.h | 22 ++++++++++++++++------
3 files changed, 26 insertions(+), 17 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 929b2cf8e7..b5ab104ec2 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,7 +2188,8 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres)
+ TBMIterateResult *tbmres,
+ long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
BlockNumber block = tbmres->blockno;
@@ -2316,6 +2317,11 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
Assert(ntup <= MaxHeapTuplesPerPage);
hscan->rs_ntuples = ntup;
+ if (tbmres->ntuples < 0)
+ (*lossy_pages)++;
+ else
+ (*exact_pages)++;
+
return ntup > 0;
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 8403be84f3..68f9ded168 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -210,8 +210,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
for (;;)
{
- bool valid;
-
CHECK_FOR_INTERRUPTS();
/*
@@ -231,19 +229,14 @@ BitmapHeapNext(BitmapHeapScanState *node)
BitmapAdjustPrefetchIterator(node, tbmres->blockno);
- valid = table_scan_bitmap_next_block(scan, tbmres);
-
- if (tbmres->ntuples >= 0)
- node->exact_pages++;
- else
- node->lossy_pages++;
-
- if (!valid)
+ if (!table_scan_bitmap_next_block(scan, tbmres,
+ &node->lossy_pages, &node->exact_pages))
{
/* AM doesn't think this block is valid, skip */
continue;
}
+
/* Adjust the prefetch target */
BitmapAdjustPrefetchTarget(node);
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 55f1139757..e2ad8d0728 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -789,6 +789,9 @@ typedef struct TableAmRoutine
* on the page have to be returned, otherwise the tuples at offsets in
* `tbmres->offsets` need to be returned.
*
+ * lossy_pages is incremented if the bitmap is lossy for the selected
+ * block; otherwise, exact_pages is incremented.
+ *
* XXX: Currently this may only be implemented if the AM uses md.c as its
* storage manager, and uses ItemPointer->ip_blkid in a manner that maps
* blockids directly to the underlying storage. nodeBitmapHeapscan.c
@@ -804,7 +807,8 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres);
+ struct TBMIterateResult *tbmres,
+ long *lossy_pages, long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
@@ -1968,17 +1972,21 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of
- * a bitmap table scan. `scan` needs to have been started via
+ * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of a
+ * bitmap table scan. `scan` needs to have been started via
* table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise.
+ * the page, true otherwise. lossy_pages is incremented is the block's
+ * representation in the bitmap is lossy; otherwise, exact_pages is
+ * incremented.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres)
+ struct TBMIterateResult *tbmres,
+ long *lossy_pages,
+ long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_block with valid
@@ -1989,7 +1997,9 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres);
+ tbmres,
+ lossy_pages,
+ exact_pages);
}
/*
--
2.40.1
[text/x-diff] v16-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch (4.1K, ../../20240405235330.v4didvweb6isodtk@liskov/9-v16-0008-Remove-table_scan_bitmap_next_tuple-parameter-tb.patch)
download | inline diff:
From faa5a08e645c7635a38e3e31a58c0aed10418c6b Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 12 Feb 2024 18:13:41 -0500
Subject: [PATCH v16 08/18] Remove table_scan_bitmap_next_tuple parameter
tbmres
Future commits will push all of the logic for choosing the next block
into the table AM specific code. The table AM will be responsible for
iterating and reading in the right blocks.
Thus, it no longer makes sense to use the TBMIterateResult (which
contains a block number) as a means of communication between
table_scan_bitmap_next_tuple() and table_scan_bitmap_next_block().
Note that this parameter was unused by heap AM's implementation of
table_scan_bitmap_next_tuple().
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas, Mark Dilger
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 1 -
src/backend/executor/nodeBitmapHeapscan.c | 2 +-
src/include/access/tableam.h | 12 +-----------
3 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index b5ab104ec2..efd1a66a09 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2327,7 +2327,6 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 68f9ded168..951a98c101 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -280,7 +280,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
/*
* Attempt to fetch tuple from AM.
*/
- if (!table_scan_bitmap_next_tuple(scan, tbmres, slot))
+ if (!table_scan_bitmap_next_tuple(scan, slot))
{
/* nothing more to look at on this page */
node->tbmres = tbmres = NULL;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index e2ad8d0728..2efa97f602 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -780,10 +780,7 @@ typedef struct TableAmRoutine
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time). For some
- * AMs it will make more sense to do all the work referencing `tbmres`
- * contents here, for others it might be better to defer more work to
- * scan_bitmap_next_tuple.
+ * make sense to perform tuple visibility checks at this time).
*
* If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
* on the page have to be returned, otherwise the tuples at offsets in
@@ -814,15 +811,10 @@ typedef struct TableAmRoutine
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * For some AMs it will make more sense to do all the work referencing
- * `tbmres` contents in scan_bitmap_next_block, for others it might be
- * better to defer more work to this callback.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot);
/*
@@ -2012,7 +2004,6 @@ table_scan_bitmap_next_block(TableScanDesc scan,
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
TupleTableSlot *slot)
{
/*
@@ -2024,7 +2015,6 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- tbmres,
slot);
}
--
2.40.1
[text/x-diff] v16-0009-Make-table_scan_bitmap_next_block-async-friendly.patch (22.4K, ../../20240405235330.v4didvweb6isodtk@liskov/10-v16-0009-Make-table_scan_bitmap_next_block-async-friendly.patch)
download | inline diff:
From f22e1320d1746afad905f169dcf4002a9f164f7e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Thu, 14 Mar 2024 12:39:28 -0400
Subject: [PATCH v16 09/18] Make table_scan_bitmap_next_block() async friendly
table_scan_bitmap_next_block() previously returned false if we did not
wish to call table_scan_bitmap_next_tuple() on the tuples on the page.
This could happen when there were no visible tuples on the page or, due
to concurrent activity on the table, the block returned by the iterator
is past the end of the table recorded when the scan started.
This forced the caller to be responsible for determining if additional
blocks should be fetched and then for invoking
table_scan_bitmap_next_block() for these blocks.
It makes more sense for table_scan_bitmap_next_block() to be responsible
for finding a block that is not past the end of the table (as of the
time that the scan began) and for table_scan_bitmap_next_tuple() to
return false if there are no visible tuples on the page.
This also allows us to move responsibility for the iterator to table AM
specific code. This means handling invalid blocks is entirely up to the
table AM.
Author: Melanie Plageman
Reviewed-by: Tomas Vondra, Andres Freund, Heikki Linnakangas
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 58 +++++--
src/backend/executor/nodeBitmapHeapscan.c | 186 ++++++++++------------
src/include/access/relscan.h | 7 +
src/include/access/tableam.h | 68 +++++---
src/include/nodes/execnodes.h | 12 +-
5 files changed, 190 insertions(+), 141 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index efd1a66a09..9d3e7c7fda 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2188,18 +2188,52 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
static bool
heapam_scan_bitmap_next_block(TableScanDesc scan,
- TBMIterateResult *tbmres,
+ BlockNumber *blockno, bool *recheck,
long *lossy_pages, long *exact_pages)
{
HeapScanDesc hscan = (HeapScanDesc) scan;
- BlockNumber block = tbmres->blockno;
+ BlockNumber block;
Buffer buffer;
Snapshot snapshot;
int ntup;
+ TBMIterateResult *tbmres;
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
+ *blockno = InvalidBlockNumber;
+ *recheck = true;
+
+ do
+ {
+ CHECK_FOR_INTERRUPTS();
+
+ if (scan->shared_tbmiterator)
+ tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
+ else
+ tbmres = tbm_iterate(scan->tbmiterator);
+
+ if (tbmres == NULL)
+ {
+ /* no more entries in the bitmap */
+ Assert(hscan->rs_empty_tuples_pending == 0);
+ return false;
+ }
+
+ /*
+ * Ignore any claimed entries past what we think is the end of the
+ * relation. It may have been extended after the start of our scan (we
+ * only hold an AccessShareLock, and it could be inserts from this
+ * backend). We don't take this optimization in SERIALIZABLE
+ * isolation though, as we need to examine all invisible tuples
+ * reachable by the index.
+ */
+ } while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
+
+ /* Got a valid block */
+ *blockno = tbmres->blockno;
+ *recheck = tbmres->recheck;
+
/*
* We can skip fetching the heap page if we don't need any fields from the
* heap, the bitmap entries don't need rechecking, and all tuples on the
@@ -2218,16 +2252,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
return true;
}
- /*
- * Ignore any claimed entries past what we think is the end of the
- * relation. It may have been extended after the start of our scan (we
- * only hold an AccessShareLock, and it could be inserts from this
- * backend). We don't take this optimization in SERIALIZABLE isolation
- * though, as we need to examine all invisible tuples reachable by the
- * index.
- */
- if (!IsolationIsSerializable() && block >= hscan->rs_nblocks)
- return false;
+ block = tbmres->blockno;
/*
* Acquire pin on the target heap page, trading in any pin we held before.
@@ -2322,7 +2347,14 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
else
(*exact_pages)++;
- return ntup > 0;
+ /*
+ * Return true to indicate that a valid block was found and the bitmap is
+ * not exhausted. If there are no visible tuples on this page,
+ * hscan->rs_ntuples will be 0 and heapam_scan_bitmap_next_tuple() will
+ * return false returning control to this function to advance to the next
+ * block in the bitmap.
+ */
+ return true;
}
static bool
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 951a98c101..e1b13ddaa6 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -51,8 +51,7 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
@@ -71,7 +70,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
ExprContext *econtext;
TableScanDesc scan;
TIDBitmap *tbm;
- TBMIterateResult *tbmres;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
dsa_area *dsa = node->ss.ps.state->es_query_dsa;
@@ -83,7 +81,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
tbm = node->tbm;
- tbmres = node->tbmres;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
@@ -111,7 +108,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->tbm = tbm;
tbmiterator = tbm_begin_iterate(tbm);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -164,7 +160,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
/* Allocate a private iterator and attach the shared state to it */
shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
- node->tbmres = tbmres = NULL;
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
@@ -203,48 +198,19 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
- node->tbmiterator = tbmiterator;
- node->shared_tbmiterator = shared_tbmiterator;
+ scan->tbmiterator = tbmiterator;
+ scan->shared_tbmiterator = shared_tbmiterator;
+
node->initialized = true;
+
+ goto new_page;
}
for (;;)
{
- CHECK_FOR_INTERRUPTS();
-
- /*
- * Get next page of results if needed
- */
- if (tbmres == NULL)
- {
- if (!pstate)
- node->tbmres = tbmres = tbm_iterate(node->tbmiterator);
- else
- node->tbmres = tbmres = tbm_shared_iterate(node->shared_tbmiterator);
- if (tbmres == NULL)
- {
- /* no more entries in the bitmap */
- break;
- }
-
- BitmapAdjustPrefetchIterator(node, tbmres->blockno);
-
- if (!table_scan_bitmap_next_block(scan, tbmres,
- &node->lossy_pages, &node->exact_pages))
- {
- /* AM doesn't think this block is valid, skip */
- continue;
- }
-
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
- }
- else
+ while (table_scan_bitmap_next_tuple(scan, slot))
{
- /*
- * Continuing in previously obtained page.
- */
+ CHECK_FOR_INTERRUPTS();
#ifdef USE_PREFETCH
@@ -266,45 +232,56 @@ BitmapHeapNext(BitmapHeapScanState *node)
SpinLockRelease(&pstate->mutex);
}
#endif /* USE_PREFETCH */
- }
- /*
- * We issue prefetch requests *after* fetching the current page to try
- * to avoid having prefetching interfere with the main I/O. Also, this
- * should happen only when we have determined there is still something
- * to do on the current page, else we may uselessly prefetch the same
- * page we are just about to request for real.
- */
- BitmapPrefetch(node, scan);
+ /*
+ * We issue prefetch requests *after* fetching the current page to
+ * try to avoid having prefetching interfere with the main I/O.
+ * Also, this should happen only when we have determined there is
+ * still something to do on the current page, else we may
+ * uselessly prefetch the same page we are just about to request
+ * for real.
+ */
+ BitmapPrefetch(node, scan);
- /*
- * Attempt to fetch tuple from AM.
- */
- if (!table_scan_bitmap_next_tuple(scan, slot))
- {
- /* nothing more to look at on this page */
- node->tbmres = tbmres = NULL;
- continue;
+ /*
+ * If we are using lossy info, we have to recheck the qual
+ * conditions at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
+ {
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
+ }
+ }
+
+ /* OK to return this tuple */
+ return slot;
}
+new_page:
+
+ BitmapAdjustPrefetchIterator(node);
+
+ if (!table_scan_bitmap_next_block(scan, &node->blockno, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
+ break;
+
/*
- * If we are using lossy info, we have to recheck the qual conditions
- * at every tuple.
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
*/
- if (tbmres->recheck)
- {
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
- }
+ if (node->pstate == NULL &&
+ node->prefetch_iterator &&
+ node->pfblockno < node->blockno)
+ elog(ERROR, "prefetch and main iterators are out of sync");
- /* OK to return this tuple */
- return slot;
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(node);
}
/*
@@ -330,13 +307,17 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
/*
* BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
*/
static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
- BlockNumber blockno)
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ TBMIterateResult *tbmpre;
if (pstate == NULL)
{
@@ -350,14 +331,17 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
else if (prefetch_iterator)
{
/* Do not let the prefetch iterator get behind the main one */
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
-
- if (tbmpre == NULL || tbmpre->blockno != blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
+ tbmpre = tbm_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
if (node->prefetch_maximum > 0)
{
TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
@@ -382,7 +366,10 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node,
* case.
*/
if (prefetch_iterator)
- tbm_shared_iterate(prefetch_iterator);
+ {
+ tbmpre = tbm_shared_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
}
}
#endif /* USE_PREFETCH */
@@ -460,6 +447,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -517,6 +505,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
+ node->pfblockno = tbmpre->blockno;
+
/* As above, skip prefetch if we expect not to need page */
skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
@@ -578,12 +568,8 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->tbm)
@@ -591,13 +577,13 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->tbmiterator = NULL;
- node->tbmres = NULL;
node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_tbmiterator = NULL;
node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
+ node->recheck = true;
+ node->blockno = InvalidBlockNumber;
+ node->pfblockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -628,28 +614,24 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
*/
ExecEndNode(outerPlanState(node));
+
+ /*
+ * close heap scan
+ */
+ if (scanDesc)
+ table_endscan(scanDesc);
+
/*
* release bitmaps and buffers if any
*/
- if (node->tbmiterator)
- tbm_end_iterate(node->tbmiterator);
if (node->prefetch_iterator)
tbm_end_iterate(node->prefetch_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_tbmiterator)
- tbm_end_shared_iterate(node->shared_tbmiterator);
if (node->shared_prefetch_iterator)
tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
-
- /*
- * close heap scan
- */
- if (scanDesc)
- table_endscan(scanDesc);
-
}
/* ----------------------------------------------------------------
@@ -682,8 +664,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->tbmiterator = NULL;
- scanstate->tbmres = NULL;
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
@@ -691,9 +671,11 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_tbmiterator = NULL;
scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
+ scanstate->recheck = true;
+ scanstate->blockno = InvalidBlockNumber;
+ scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 521043304a..92b829cebc 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -24,6 +24,9 @@
struct ParallelTableScanDescData;
+struct TBMIterator;
+struct TBMSharedIterator;
+
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -40,6 +43,10 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
+ /* Only used for Bitmap table scans */
+ struct TBMIterator *tbmiterator;
+ struct TBMSharedIterator *shared_tbmiterator;
+
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 2efa97f602..42c67a128e 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -22,6 +22,7 @@
#include "access/xact.h"
#include "commands/vacuum.h"
#include "executor/tuptable.h"
+#include "nodes/tidbitmap.h"
#include "utils/rel.h"
#include "utils/snapshot.h"
@@ -773,19 +774,14 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part
- * of a bitmap table scan. `scan` was started via table_beginscan_bm().
- * Return false if there are no tuples to be found on the page, true
- * otherwise.
+ * Prepare to fetch / check / return tuples from `blockno` as part of a
+ * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
+ * false if the bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
* make sense to perform tuple visibility checks at this time).
*
- * If `tbmres->blockno` is -1, this is a lossy scan and all visible tuples
- * on the page have to be returned, otherwise the tuples at offsets in
- * `tbmres->offsets` need to be returned.
- *
* lossy_pages is incremented if the bitmap is lossy for the selected
* block; otherwise, exact_pages is incremented.
*
@@ -804,7 +800,7 @@ typedef struct TableAmRoutine
* scan_bitmap_next_tuple need to exist, or neither.
*/
bool (*scan_bitmap_next_block) (TableScanDesc scan,
- struct TBMIterateResult *tbmres,
+ BlockNumber *blockno, bool *recheck,
long *lossy_pages, long *exact_pages);
/*
@@ -942,9 +938,13 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
+ TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ result->shared_tbmiterator = NULL;
+ result->tbmiterator = NULL;
+ return result;
}
/*
@@ -991,6 +991,21 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1001,6 +1016,21 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
+ {
+ if (scan->shared_tbmiterator)
+ {
+ tbm_end_shared_iterate(scan->shared_tbmiterator);
+ scan->shared_tbmiterator = NULL;
+ }
+
+ if (scan->tbmiterator)
+ {
+ tbm_end_iterate(scan->tbmiterator);
+ scan->tbmiterator = NULL;
+ }
+ }
+
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
@@ -1964,19 +1994,18 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples from `tbmres->blockno` as part of a
- * bitmap table scan. `scan` needs to have been started via
- * table_beginscan_bm(). Returns false if there are no tuples to be found on
- * the page, true otherwise. lossy_pages is incremented is the block's
- * representation in the bitmap is lossy; otherwise, exact_pages is
- * incremented.
+ * Prepare to fetch / check / return tuples as part of a bitmap table scan.
+ * `scan` needs to have been started via table_beginscan_bm(). Returns false if
+ * there are no more blocks in the bitmap, true otherwise. lossy_pages is
+ * incremented is the block's representation in the bitmap is lossy; otherwise,
+ * exact_pages is incremented.
*
* Note, this is an optionally implemented function, therefore should only be
* used after verifying the presence (at plan time or such).
*/
static inline bool
table_scan_bitmap_next_block(TableScanDesc scan,
- struct TBMIterateResult *tbmres,
+ BlockNumber *blockno, bool *recheck,
long *lossy_pages,
long *exact_pages)
{
@@ -1989,9 +2018,8 @@ table_scan_bitmap_next_block(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- tbmres,
- lossy_pages,
- exact_pages);
+ blockno, recheck,
+ lossy_pages, exact_pages);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index fa2f70b7a4..d96703b04d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1792,8 +1792,6 @@ typedef struct ParallelBitmapHeapState
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * tbmiterator iterator for scanning current pages
- * tbmres current-page data
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
@@ -1802,9 +1800,11 @@ typedef struct ParallelBitmapHeapState
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_tbmiterator shared iterator
* shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
+ * recheck do current page's tuples need recheck
+ * blockno used to validate pf and current block in sync
+ * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1812,8 +1812,6 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- TBMIterator *tbmiterator;
- TBMIterateResult *tbmres;
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
@@ -1822,9 +1820,11 @@ typedef struct BitmapHeapScanState
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_tbmiterator;
TBMSharedIterator *shared_prefetch_iterator;
ParallelBitmapHeapState *pstate;
+ bool recheck;
+ BlockNumber blockno;
+ BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v16-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch (16.7K, ../../20240405235330.v4didvweb6isodtk@liskov/11-v16-0010-Unify-parallel-and-serial-BitmapHeapScan-iterato.patch)
download | inline diff:
From afb28afbf45efbdd9fa4992a3dc1f1fe7f07afd3 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Mon, 25 Mar 2024 11:05:51 -0400
Subject: [PATCH v16 10/18] Unify parallel and serial BitmapHeapScan iterator
interfaces
Introduce a new type, BitmapHeapIterator, which allows unified access to both
TBMIterator and TBMSharedIterators. This encapsulates the parallel and serial
iterators and their access and makes the bitmap heap scan code a bit cleaner.
This naturally lends itself to a bit of reorganization of the
!node->initialized path in BitmapHeapNext().
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 5 +-
src/backend/executor/nodeBitmapHeapscan.c | 172 ++++++++++++----------
src/include/access/relscan.h | 7 +-
src/include/access/tableam.h | 40 +----
src/include/executor/nodeBitmapHeapscan.h | 7 +
src/include/nodes/execnodes.h | 13 +-
src/tools/pgindent/typedefs.list | 1 +
7 files changed, 124 insertions(+), 121 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 9d3e7c7fda..5e57d19d91 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2208,10 +2208,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- if (scan->shared_tbmiterator)
- tbmres = tbm_shared_iterate(scan->shared_tbmiterator);
- else
- tbmres = tbm_iterate(scan->tbmiterator);
+ tbmres = bhs_iterate(&scan->rs_bhs_iterator);
if (tbmres == NULL)
{
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index e1b13ddaa6..70b560b8ee 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -57,6 +57,61 @@ static inline void BitmapPrefetch(BitmapHeapScanState *node,
TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
+/*
+ * Start iteration on a shared or non-shared bitmap iterator. Note that tbm
+ * will only be provided by serial BitmapHeapScan callers. dsa and dsp will
+ * only be provided by parallel BitmapHeapScan callers.
+ */
+void
+bhs_begin_iterate(BitmapHeapIterator *iterator, TIDBitmap *tbm,
+ dsa_area *dsa, dsa_pointer dsp)
+{
+ Assert(iterator);
+
+ iterator->serial = NULL;
+ iterator->parallel = NULL;
+ iterator->exhausted = false;
+
+ /* Allocate a private iterator and attach the shared state to it */
+ if (DsaPointerIsValid(dsp))
+ iterator->parallel = tbm_attach_shared_iterate(dsa, dsp);
+ else
+ iterator->serial = tbm_begin_iterate(tbm);
+}
+
+/*
+ * Get the next TBMIterateResult from the bitmap iterator.
+ */
+TBMIterateResult *
+bhs_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+ Assert(!iterator->exhausted);
+
+ if (iterator->serial)
+ return tbm_iterate(iterator->serial);
+ else
+ return tbm_shared_iterate(iterator->parallel);
+}
+
+/*
+ * Clean up the bitmap iterator.
+ */
+void
+bhs_end_iterate(BitmapHeapIterator *iterator)
+{
+ Assert(iterator);
+
+ if (iterator->serial)
+ tbm_end_iterate(iterator->serial);
+ else if (iterator->parallel)
+ tbm_end_shared_iterate(iterator->parallel);
+
+ iterator->serial = NULL;
+ iterator->parallel = NULL;
+ iterator->exhausted = true;
+}
+
/* ----------------------------------------------------------------
* BitmapHeapNext
@@ -96,43 +151,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
- TBMIterator *tbmiterator = NULL;
- TBMSharedIterator *shared_tbmiterator = NULL;
+ /*
+ * The leader will immediately come out of the function, but others
+ * will be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
- if (!pstate)
+ if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
if (!tbm || !IsA(tbm, TIDBitmap))
elog(ERROR, "unrecognized result from subplan");
-
node->tbm = tbm;
- tbmiterator = tbm_begin_iterate(tbm);
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (init_shared_state)
{
- node->prefetch_iterator = tbm_begin_iterate(tbm);
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
- }
-#endif /* USE_PREFETCH */
- }
- else
- {
- /*
- * The leader will immediately come out of the function, but
- * others will be blocked until leader populates the TBM and wakes
- * them up.
- */
- if (BitmapShouldInitializeSharedState(pstate))
- {
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
-
- node->tbm = tbm;
-
/*
* Prepare to iterate over the TBM. This will return the
* dsa_pointer of the iterator state which will be used by
@@ -153,21 +188,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate->prefetch_target = -1;
}
#endif
-
/* We have initialized the shared state so wake up others. */
BitmapDoneInitializingSharedState(pstate);
}
-
- /* Allocate a private iterator and attach the shared state to it */
- shared_tbmiterator = tbm_attach_shared_iterate(dsa, pstate->tbmiterator);
-
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- node->shared_prefetch_iterator =
- tbm_attach_shared_iterate(dsa, pstate->prefetch_iterator);
- }
-#endif /* USE_PREFETCH */
}
/*
@@ -198,8 +221,23 @@ BitmapHeapNext(BitmapHeapScanState *node)
node->ss.ss_currentScanDesc = scan;
}
- scan->tbmiterator = tbmiterator;
- scan->shared_tbmiterator = shared_tbmiterator;
+ bhs_begin_iterate(&scan->rs_bhs_iterator, tbm, dsa,
+ pstate ?
+ pstate->tbmiterator :
+ InvalidDsaPointer);
+
+#ifdef USE_PREFETCH
+ if (node->prefetch_maximum > 0)
+ {
+ bhs_begin_iterate(&node->pf_iterator, tbm, dsa,
+ pstate ?
+ pstate->prefetch_iterator :
+ InvalidDsaPointer);
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
+ }
+#endif /* USE_PREFETCH */
node->initialized = true;
@@ -276,7 +314,7 @@ new_page:
* ahead of the current block.
*/
if (node->pstate == NULL &&
- node->prefetch_iterator &&
+ !node->pf_iterator.exhausted &&
node->pfblockno < node->blockno)
elog(ERROR, "prefetch and main iterators are out of sync");
@@ -317,21 +355,20 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
if (node->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
node->prefetch_pages--;
}
- else if (prefetch_iterator)
+ else if (!prefetch_iterator->exhausted)
{
/* Do not let the prefetch iterator get behind the main one */
- tbmpre = tbm_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
@@ -344,8 +381,6 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
*/
if (node->prefetch_maximum > 0)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
{
@@ -365,9 +400,9 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
* we don't validate the blockno here as we do in non-parallel
* case.
*/
- if (prefetch_iterator)
+ if (!prefetch_iterator->exhausted)
{
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
@@ -427,23 +462,21 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
{
#ifdef USE_PREFETCH
ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
if (pstate == NULL)
{
- TBMIterator *prefetch_iterator = node->prefetch_iterator;
-
- if (prefetch_iterator)
+ if (!prefetch_iterator->exhausted)
{
while (node->prefetch_pages < node->prefetch_target)
{
- TBMIterateResult *tbmpre = tbm_iterate(prefetch_iterator);
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_iterate(prefetch_iterator);
- node->prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
break;
}
node->prefetch_pages++;
@@ -471,9 +504,7 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (pstate->prefetch_pages < pstate->prefetch_target)
{
- TBMSharedIterator *prefetch_iterator = node->shared_prefetch_iterator;
-
- if (prefetch_iterator)
+ if (!prefetch_iterator->exhausted)
{
while (1)
{
@@ -496,12 +527,11 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
if (!do_prefetch)
return;
- tbmpre = tbm_shared_iterate(prefetch_iterator);
+ tbmpre = bhs_iterate(prefetch_iterator);
if (tbmpre == NULL)
{
/* No more pages to prefetch */
- tbm_end_shared_iterate(prefetch_iterator);
- node->shared_prefetch_iterator = NULL;
+ bhs_end_iterate(prefetch_iterator);
break;
}
@@ -568,18 +598,14 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
table_rescan(node->ss.ss_currentScanDesc, NULL);
/* release bitmaps and buffers if any */
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
+ if (node->pf_iterator.exhausted)
+ bhs_end_iterate(&node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
- node->prefetch_iterator = NULL;
node->initialized = false;
- node->shared_prefetch_iterator = NULL;
node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
@@ -624,12 +650,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
* release bitmaps and buffers if any
*/
- if (node->prefetch_iterator)
- tbm_end_iterate(node->prefetch_iterator);
+ if (!node->pf_iterator.exhausted)
+ bhs_end_iterate(&node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->shared_prefetch_iterator)
- tbm_end_shared_iterate(node->shared_prefetch_iterator);
if (node->pvmbuffer != InvalidBuffer)
ReleaseBuffer(node->pvmbuffer);
}
@@ -667,11 +691,9 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_iterator = NULL;
scanstate->prefetch_pages = 0;
scanstate->prefetch_target = 0;
scanstate->initialized = false;
- scanstate->shared_prefetch_iterator = NULL;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index 92b829cebc..e520186b41 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -20,13 +20,11 @@
#include "storage/buf.h"
#include "storage/spin.h"
#include "utils/relcache.h"
+#include "executor/nodeBitmapHeapscan.h"
struct ParallelTableScanDescData;
-struct TBMIterator;
-struct TBMSharedIterator;
-
/*
* Generic descriptor for table scans. This is the base-class for table scans,
* which needs to be embedded in the scans of individual AMs.
@@ -44,8 +42,7 @@ typedef struct TableScanDescData
ItemPointerData rs_maxtid;
/* Only used for Bitmap table scans */
- struct TBMIterator *tbmiterator;
- struct TBMSharedIterator *shared_tbmiterator;
+ BitmapHeapIterator rs_bhs_iterator;
/*
* Information about type and behaviour of the scan, a bitmask of members
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 42c67a128e..618ab2b449 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -938,13 +938,9 @@ static inline TableScanDesc
table_beginscan_bm(Relation rel, Snapshot snapshot,
int nkeys, struct ScanKeyData *key, uint32 extra_flags)
{
- TableScanDesc result;
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- result = rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
- result->shared_tbmiterator = NULL;
- result->tbmiterator = NULL;
- return result;
+ return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
}
/*
@@ -991,20 +987,9 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
- {
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
- }
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
+ !scan->rs_bhs_iterator.exhausted)
+ bhs_end_iterate(&scan->rs_bhs_iterator);
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1016,20 +1001,9 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN)
- {
- if (scan->shared_tbmiterator)
- {
- tbm_end_shared_iterate(scan->shared_tbmiterator);
- scan->shared_tbmiterator = NULL;
- }
-
- if (scan->tbmiterator)
- {
- tbm_end_iterate(scan->tbmiterator);
- scan->tbmiterator = NULL;
- }
- }
+ if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
+ !scan->rs_bhs_iterator.exhausted)
+ bhs_end_iterate(&scan->rs_bhs_iterator);
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index ea003a9caa..7064f54686 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -29,4 +29,11 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
+extern void bhs_begin_iterate(BitmapHeapIterator *iterator, TIDBitmap *tbm,
+ dsa_area *dsa, dsa_pointer dsp);
+
+extern void bhs_end_iterate(BitmapHeapIterator *iterator);
+
+
#endif /* NODEBITMAPHEAPSCAN_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index d96703b04d..714eb3e534 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1787,6 +1787,13 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+ bool exhausted;
+} BitmapHeapIterator;
+
/* ----------------
* BitmapHeapScanState information
*
@@ -1795,12 +1802,11 @@ typedef struct ParallelBitmapHeapState
* pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * prefetch_iterator iterator for prefetching ahead of current page
+ * pf_iterator for prefetching ahead of current page
* prefetch_pages # pages prefetch iterator is ahead of current
* prefetch_target current target prefetch distance
* prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
- * shared_prefetch_iterator shared iterator for prefetching
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
@@ -1815,12 +1821,11 @@ typedef struct BitmapHeapScanState
Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- TBMIterator *prefetch_iterator;
int prefetch_pages;
int prefetch_target;
int prefetch_maximum;
bool initialized;
- TBMSharedIterator *shared_prefetch_iterator;
+ BitmapHeapIterator pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 01845ee71d..848ce9b30c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -259,6 +259,7 @@ BitString
BitmapAnd
BitmapAndPath
BitmapAndState
+BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
--
2.40.1
[text/x-diff] v16-0011-Add-BitmapHeapScanDesc-and-scan-functions.patch (18.0K, ../../20240405235330.v4didvweb6isodtk@liskov/12-v16-0011-Add-BitmapHeapScanDesc-and-scan-functions.patch)
download | inline diff:
From eac4e11efc7c19091b8076f6193b90aa3989b19e Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 19:30:43 -0400
Subject: [PATCH v16 11/18] Add BitmapHeapScanDesc and scan functions
Future commits will push all prefetching related code into the heap AM
to eliminate a layering violation. Move all of the members used to
manage prefetching for bitmap heap scan from the BitmapHeapScanState
into a new scan descriptor BitmapHeapScanDesc.
With this new scan descriptor, add all of the relevant functions for
beginning, rescanning, and ending a bitmap heap scan. There is a lot of
bitmap-specific logic to do there now, so it makes sense for bitmap heap
scan to have its own.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 98 +++++++++++++++++++----
src/backend/access/heap/heapam_handler.c | 42 +++++-----
src/backend/executor/nodeBitmapHeapscan.c | 47 +++++------
src/include/access/heapam.h | 36 ++++++---
src/include/access/tableam.h | 45 +++++++++--
src/tools/pgindent/typedefs.list | 1 +
6 files changed, 193 insertions(+), 76 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 10f2faaa60..82faf447e9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -13,8 +13,11 @@
*
* INTERFACE ROUTINES
* heap_beginscan - begin relation scan
+ * heap_beginscan_bm - begin bitmap heap scan
* heap_rescan - restart a relation scan
+ * heap_rescan_bm - restart a bitmap heap scan
* heap_endscan - end relation scan
+ * heap_endscan_bm - end a bitmap heap scan
* heap_getnext - retrieve next tuple in scan
* heap_fetch - retrieve tuple with given tid
* heap_insert - insert tuple into a relation
@@ -938,6 +941,43 @@ continue_page:
* ----------------------------------------------------------------
*/
+TableScanDesc
+heap_beginscan_bm(Relation relation, Snapshot snapshot, uint32 flags)
+{
+ BitmapHeapScanDesc scan;
+
+ /*
+ * increment relation ref count while scanning relation
+ *
+ * This is just to make really sure the relcache entry won't go away while
+ * the scan has a pointer to it. Caller should be holding the rel open
+ * anyway, so this is redundant in all normal scenarios...
+ */
+ RelationIncrementReferenceCount(relation);
+ scan = (BitmapHeapScanDesc) palloc(sizeof(BitmapHeapScanDescData));
+
+ scan->heap_common.rs_base.rs_rd = relation;
+ scan->heap_common.rs_base.rs_snapshot = snapshot;
+ scan->heap_common.rs_base.rs_nkeys = 0;
+ scan->heap_common.rs_base.rs_flags = flags;
+ scan->heap_common.rs_base.rs_parallel = NULL;
+ scan->heap_common.rs_strategy = NULL;
+
+ Assert(snapshot && IsMVCCSnapshot(snapshot));
+
+ /* we only need to set this up once */
+ scan->heap_common.rs_ctup.t_tableOid = RelationGetRelid(relation);
+
+ scan->heap_common.rs_parallelworkerdata = NULL;
+ scan->heap_common.rs_base.rs_key = NULL;
+
+ initscan(&scan->heap_common, NULL, false);
+
+ scan->vmbuffer = InvalidBuffer;
+
+ return (TableScanDesc) scan;
+}
+
TableScanDesc
heap_beginscan(Relation relation, Snapshot snapshot,
@@ -967,8 +1007,6 @@ heap_beginscan(Relation relation, Snapshot snapshot,
scan->rs_base.rs_flags = flags;
scan->rs_base.rs_parallel = parallel_scan;
scan->rs_strategy = NULL; /* set in initscan */
- scan->rs_vmbuffer = InvalidBuffer;
- scan->rs_empty_tuples_pending = 0;
/*
* Disable page-at-a-time mode if it's not a MVCC-safe snapshot.
@@ -1026,6 +1064,29 @@ heap_beginscan(Relation relation, Snapshot snapshot,
return (TableScanDesc) scan;
}
+/*
+ * Cleanup BitmapHeapScan table state
+ */
+void
+heap_endscan_bm(TableScanDesc sscan)
+{
+ BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
+
+ if (BufferIsValid(scan->heap_common.rs_cbuf))
+ ReleaseBuffer(scan->heap_common.rs_cbuf);
+
+ if (BufferIsValid(scan->vmbuffer))
+ ReleaseBuffer(scan->vmbuffer);
+
+ /*
+ * decrement relation reference count and free scan descriptor storage
+ */
+ RelationDecrementReferenceCount(scan->heap_common.rs_base.rs_rd);
+
+ pfree(scan);
+}
+
+
void
heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
bool allow_strat, bool allow_sync, bool allow_pagemode)
@@ -1057,12 +1118,6 @@ heap_rescan(TableScanDesc sscan, ScanKey key, bool set_params,
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
- if (BufferIsValid(scan->rs_vmbuffer))
- {
- ReleaseBuffer(scan->rs_vmbuffer);
- scan->rs_vmbuffer = InvalidBuffer;
- }
-
/*
* reinitialize scan descriptor
*/
@@ -1082,12 +1137,6 @@ heap_endscan(TableScanDesc sscan)
if (BufferIsValid(scan->rs_cbuf))
ReleaseBuffer(scan->rs_cbuf);
- if (BufferIsValid(scan->rs_vmbuffer))
- {
- ReleaseBuffer(scan->rs_vmbuffer);
- scan->rs_vmbuffer = InvalidBuffer;
- }
-
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1334,6 +1383,27 @@ heap_getnextslot_tidrange(TableScanDesc sscan, ScanDirection direction,
return true;
}
+void
+heap_rescan_bm(TableScanDesc sscan)
+{
+ BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
+
+ if (BufferIsValid(scan->heap_common.rs_cbuf))
+ ReleaseBuffer(scan->heap_common.rs_cbuf);
+
+ if (BufferIsValid(scan->vmbuffer))
+ ReleaseBuffer(scan->vmbuffer);
+ scan->vmbuffer = InvalidBuffer;
+
+ scan->empty_tuples_pending = 0;
+
+ /*
+ * reinitialize heap scan descriptor
+ */
+ initscan(&scan->heap_common, NULL, true);
+}
+
+
/*
* heap_fetch - retrieve tuple with given tid
*
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 5e57d19d91..43721cbd42 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2187,11 +2187,12 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
*/
static bool
-heapam_scan_bitmap_next_block(TableScanDesc scan,
+heapam_scan_bitmap_next_block(TableScanDesc sscan,
BlockNumber *blockno, bool *recheck,
long *lossy_pages, long *exact_pages)
{
- HeapScanDesc hscan = (HeapScanDesc) scan;
+ BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
+ HeapScanDesc hscan = &scan->heap_common;
BlockNumber block;
Buffer buffer;
Snapshot snapshot;
@@ -2208,12 +2209,12 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(&scan->rs_bhs_iterator);
+ tbmres = bhs_iterate(&sscan->rs_bhs_iterator);
if (tbmres == NULL)
{
/* no more entries in the bitmap */
- Assert(hscan->rs_empty_tuples_pending == 0);
+ Assert(scan->empty_tuples_pending == 0);
return false;
}
@@ -2236,15 +2237,15 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* heap, the bitmap entries don't need rechecking, and all tuples on the
* page are visible to our transaction.
*/
- if (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ if (!(hscan->rs_base.rs_flags & SO_NEED_TUPLE) &&
!tbmres->recheck &&
- VM_ALL_VISIBLE(scan->rs_rd, tbmres->blockno, &hscan->rs_vmbuffer))
+ VM_ALL_VISIBLE(hscan->rs_base.rs_rd, tbmres->blockno, &scan->vmbuffer))
{
/* can't be lossy in the skip_fetch case */
Assert(tbmres->ntuples >= 0);
- Assert(hscan->rs_empty_tuples_pending >= 0);
+ Assert(scan->empty_tuples_pending >= 0);
- hscan->rs_empty_tuples_pending += tbmres->ntuples;
+ scan->empty_tuples_pending += tbmres->ntuples;
return true;
}
@@ -2255,18 +2256,18 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
* Acquire pin on the target heap page, trading in any pin we held before.
*/
hscan->rs_cbuf = ReleaseAndReadBuffer(hscan->rs_cbuf,
- scan->rs_rd,
+ hscan->rs_base.rs_rd,
block);
hscan->rs_cblock = block;
buffer = hscan->rs_cbuf;
- snapshot = scan->rs_snapshot;
+ snapshot = hscan->rs_base.rs_snapshot;
ntup = 0;
/*
* Prune and repair fragmentation for the whole page, if possible.
*/
- heap_page_prune_opt(scan->rs_rd, buffer);
+ heap_page_prune_opt(hscan->rs_base.rs_rd, buffer);
/*
* We must hold share lock on the buffer content while examining tuple
@@ -2294,7 +2295,7 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
HeapTupleData heapTuple;
ItemPointerSet(&tid, block, offnum);
- if (heap_hot_search_buffer(&tid, scan->rs_rd, buffer, snapshot,
+ if (heap_hot_search_buffer(&tid, hscan->rs_base.rs_rd, buffer, snapshot,
&heapTuple, NULL, true))
hscan->rs_vistuples[ntup++] = ItemPointerGetOffsetNumber(&tid);
}
@@ -2320,16 +2321,16 @@ heapam_scan_bitmap_next_block(TableScanDesc scan,
continue;
loctup.t_data = (HeapTupleHeader) PageGetItem(page, lp);
loctup.t_len = ItemIdGetLength(lp);
- loctup.t_tableOid = scan->rs_rd->rd_id;
+ loctup.t_tableOid = hscan->rs_base.rs_rd->rd_id;
ItemPointerSet(&loctup.t_self, block, offnum);
valid = HeapTupleSatisfiesVisibility(&loctup, snapshot, buffer);
if (valid)
{
hscan->rs_vistuples[ntup++] = offnum;
- PredicateLockTID(scan->rs_rd, &loctup.t_self, snapshot,
+ PredicateLockTID(hscan->rs_base.rs_rd, &loctup.t_self, snapshot,
HeapTupleHeaderGetXmin(loctup.t_data));
}
- HeapCheckForSerializableConflictOut(valid, scan->rs_rd, &loctup,
+ HeapCheckForSerializableConflictOut(valid, hscan->rs_base.rs_rd, &loctup,
buffer, snapshot);
}
}
@@ -2358,18 +2359,19 @@ static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
TupleTableSlot *slot)
{
- HeapScanDesc hscan = (HeapScanDesc) scan;
+ BitmapHeapScanDesc bscan = (BitmapHeapScanDesc) scan;
+ HeapScanDesc hscan = &bscan->heap_common;
OffsetNumber targoffset;
Page page;
ItemId lp;
- if (hscan->rs_empty_tuples_pending > 0)
+ if (bscan->empty_tuples_pending > 0)
{
/*
* If we don't have to fetch the tuple, just return nulls.
*/
ExecStoreAllNullTuple(slot);
- hscan->rs_empty_tuples_pending--;
+ bscan->empty_tuples_pending--;
return true;
}
@@ -2704,6 +2706,10 @@ static const TableAmRoutine heapam_methods = {
.scan_set_tidrange = heap_set_tidrange,
.scan_getnextslot_tidrange = heap_getnextslot_tidrange,
+ .scan_rescan_bm = heap_rescan_bm,
+ .scan_begin_bm = heap_beginscan_bm,
+ .scan_end_bm = heap_endscan_bm,
+
.parallelscan_estimate = table_block_parallelscan_estimate,
.parallelscan_initialize = table_block_parallelscan_initialize,
.parallelscan_reinitialize = table_block_parallelscan_reinitialize,
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 70b560b8ee..0b28c46d08 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -151,6 +151,8 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ uint32 extra_flags = 0;
+
/*
* The leader will immediately come out of the function, but others
* will be blocked until leader populates the TBM and wakes them up.
@@ -193,33 +195,26 @@ BitmapHeapNext(BitmapHeapScanState *node)
}
}
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or
+ * for returning data. This test is a bit simplistic, as it checks
+ * the stronger condition that there's no qual or return tlist at all.
+ * But in most cases it's probably not worth working harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
/*
* If this is the first scan of the underlying table, create the table
* scan descriptor and begin the scan.
*/
- if (!scan)
- {
- uint32 extra_flags = 0;
+ scan = table_beginscan_bm(node->ss.ss_currentScanDesc,
+ node->ss.ss_currentRelation,
+ node->ss.ps.state->es_snapshot,
+ extra_flags);
- /*
- * We can potentially skip fetching heap pages if we do not need
- * any columns of the table, either for checking non-indexable
- * quals or for returning data. This test is a bit simplistic, as
- * it checks the stronger condition that there's no qual or return
- * tlist at all. But in most cases it's probably not worth working
- * harder than that.
- */
- if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
- extra_flags |= SO_NEED_TUPLE;
-
- scan = table_beginscan_bm(node->ss.ss_currentRelation,
- node->ss.ps.state->es_snapshot,
- 0,
- NULL,
- extra_flags);
-
- node->ss.ss_currentScanDesc = scan;
- }
+ node->ss.ss_currentScanDesc = scan;
bhs_begin_iterate(&scan->rs_bhs_iterator, tbm, dsa,
pstate ?
@@ -593,10 +588,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
{
PlanState *outerPlan = outerPlanState(node);
- /* rescan to release any page pin */
- if (node->ss.ss_currentScanDesc)
- table_rescan(node->ss.ss_currentScanDesc, NULL);
-
/* release bitmaps and buffers if any */
if (node->pf_iterator.exhausted)
bhs_end_iterate(&node->pf_iterator);
@@ -642,10 +633,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
/*
- * close heap scan
+ * close bitmapheap scan
*/
if (scanDesc)
- table_endscan(scanDesc);
+ table_endscan_bm(scanDesc);
/*
* release bitmaps and buffers if any
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 750ea30852..545760aab5 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -76,22 +76,35 @@ typedef struct HeapScanDescData
*/
ParallelBlockTableScanWorkerData *rs_parallelworkerdata;
+ /* these fields only used in page-at-a-time mode and for bitmap scans */
+ int rs_cindex; /* current tuple's index in vistuples */
+ int rs_ntuples; /* number of visible tuples on page */
+ OffsetNumber rs_vistuples[MaxHeapTuplesPerPage]; /* their offsets */
+} HeapScanDescData;
+typedef struct HeapScanDescData *HeapScanDesc;
+
+
+typedef struct BitmapHeapScanDescData
+{
+ /* All the non-BitmapHeapScan specific members */
+ HeapScanDescData heap_common;
+
+ /*
+ * Members common to Parallel and Serial BitmapHeapScan
+ */
+
/*
* These fields are only used for bitmap scans for the "skip fetch"
* optimization. Bitmap scans needing no fields from the heap may skip
* fetching an all visible block, instead using the number of tuples per
* block reported by the bitmap to determine how many NULL-filled tuples
- * to return.
+ * to return. They are common to parallel and serial BitmapHeapScans
*/
- Buffer rs_vmbuffer;
- int rs_empty_tuples_pending;
+ Buffer vmbuffer;
+ int empty_tuples_pending;
+} BitmapHeapScanDescData;
- /* these fields only used in page-at-a-time mode and for bitmap scans */
- int rs_cindex; /* current tuple's index in vistuples */
- int rs_ntuples; /* number of visible tuples on page */
- OffsetNumber rs_vistuples[MaxHeapTuplesPerPage]; /* their offsets */
-} HeapScanDescData;
-typedef struct HeapScanDescData *HeapScanDesc;
+typedef struct BitmapHeapScanDescData *BitmapHeapScanDesc;
/*
* Descriptor for fetches from heap via an index.
@@ -289,6 +302,11 @@ extern void heap_set_tidrange(TableScanDesc sscan, ItemPointer mintid,
extern bool heap_getnextslot_tidrange(TableScanDesc sscan,
ScanDirection direction,
TupleTableSlot *slot);
+extern TableScanDesc heap_beginscan_bm(Relation relation, Snapshot snapshot, uint32 flags);
+
+extern void heap_endscan_bm(TableScanDesc scan);
+extern void heap_rescan_bm(TableScanDesc sscan);
+
extern bool heap_fetch(Relation relation, Snapshot snapshot,
HeapTuple tuple, Buffer *userbuf, bool keep_buf);
extern bool heap_hot_search_buffer(ItemPointer tid, Relation relation,
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 618ab2b449..3b0fa0610b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -391,6 +391,21 @@ typedef struct TableAmRoutine
ScanDirection direction,
TupleTableSlot *slot);
+ /*
+ * TODO: add comment
+ */
+ TableScanDesc (*scan_begin_bm) (Relation rel,
+ Snapshot snapshot,
+ uint32 flags);
+
+ void (*scan_rescan_bm) (TableScanDesc scan);
+
+ /*
+ * Release resources and deallocate scan. If TableScanDesc.temp_snap,
+ * TableScanDesc.rs_snapshot needs to be unregistered.
+ */
+ void (*scan_end_bm) (TableScanDesc scan);
+
/* ------------------------------------------------------------------------
* Parallel table scan related functions.
* ------------------------------------------------------------------------
@@ -929,18 +944,34 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
}
/*
- * table_beginscan_bm is an alternative entry point for setting up a
- * TableScanDesc for a bitmap heap scan. Although that scan technology is
- * really quite unlike a standard seqscan, there is just enough commonality to
- * make it worth using the same data structure.
+ * table_beginscan_bm is the entry point for setting up a TableScanDesc for a
+ * bitmap heap scan.
*/
static inline TableScanDesc
-table_beginscan_bm(Relation rel, Snapshot snapshot,
- int nkeys, struct ScanKeyData *key, uint32 extra_flags)
+table_beginscan_bm(TableScanDesc scan, Relation rel, Snapshot snapshot,
+ uint32 extra_flags)
{
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
- return rel->rd_tableam->scan_begin(rel, snapshot, nkeys, key, NULL, flags);
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ if (!scan)
+ scan = rel->rd_tableam->scan_begin_bm(rel, snapshot, flags);
+
+ scan->rs_rd->rd_tableam->scan_rescan_bm(scan);
+
+ return scan;
+}
+
+/*
+ * End Bitmap Table Scan
+ */
+static inline void
+table_endscan_bm(TableScanDesc scan)
+{
+ scan->rs_rd->rd_tableam->scan_end_bm(scan);
}
/*
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 848ce9b30c..8369c15a82 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -263,6 +263,7 @@ BitmapHeapIterator
BitmapHeapPath
BitmapHeapScan
BitmapHeapScanState
+BitmapHeapScanDesc
BitmapIndexScan
BitmapIndexScanState
BitmapOr
--
2.40.1
[text/x-diff] v16-0012-BitmapHeapScan-initialize-some-prefetch-state-el.patch (2.5K, ../../20240405235330.v4didvweb6isodtk@liskov/13-v16-0012-BitmapHeapScan-initialize-some-prefetch-state-el.patch)
download | inline diff:
From ad2395adba28db8d1bbf0f87c64bf619367d48b9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 17:38:36 -0400
Subject: [PATCH v16 12/18] BitmapHeapScan initialize some prefetch state
elsewhere
These members can be initialized elsewhere. This makes the diff more
straightforward for moving those members from BitmapHeapScanState to
BitmapHeapScanDesc.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 17 +++++------------
1 file changed, 5 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 0b28c46d08..4f7138bbd9 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -181,13 +181,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
-
- /*
- * We don't need the mutex here as we haven't yet woke up
- * others.
- */
- pstate->prefetch_pages = 0;
- pstate->prefetch_target = -1;
}
#endif
/* We have initialized the shared state so wake up others. */
@@ -228,9 +221,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
pstate ?
pstate->prefetch_iterator :
InvalidDsaPointer);
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
}
#endif /* USE_PREFETCH */
@@ -601,6 +591,9 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->recheck = true;
node->blockno = InvalidBlockNumber;
node->pfblockno = InvalidBlockNumber;
+ /* Only used for serial BHS */
+ node->prefetch_pages = 0;
+ node->prefetch_target = -1;
ExecScanReScan(&node->ss);
@@ -683,7 +676,7 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = 0;
+ scanstate->prefetch_target = -1;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
@@ -818,7 +811,7 @@ ExecBitmapHeapInitializeDSM(BitmapHeapScanState *node,
/* Initialize the mutex */
SpinLockInit(&pstate->mutex);
pstate->prefetch_pages = 0;
- pstate->prefetch_target = 0;
+ pstate->prefetch_target = -1;
pstate->state = BM_INITIAL;
ConditionVariableInit(&pstate->cv);
--
2.40.1
[text/x-diff] v16-0013-BitmapHeapScan-begin-iteration-in-rescan.patch (4.3K, ../../20240405235330.v4didvweb6isodtk@liskov/14-v16-0013-BitmapHeapScan-begin-iteration-in-rescan.patch)
download | inline diff:
From d16551c7330822ccd0132565773ceab7569aa6b5 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 17:48:08 -0400
Subject: [PATCH v16 13/18] BitmapHeapScan begin iteration in rescan
Now that BitmapHeapScan has its own dedicated rescan function, it is
easy to begin iteration there instead. Doing this in heap AM code will
allow us to move the current block iterator from the TableScanDesc to
the BitmapHeapScanDesc.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 8 +++++++-
src/backend/executor/nodeBitmapHeapscan.c | 10 ++++------
src/include/access/heapam.h | 3 ++-
src/include/access/tableam.h | 8 +++++---
4 files changed, 18 insertions(+), 11 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 82faf447e9..a876a34b74 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1384,7 +1384,8 @@ heap_getnextslot_tidrange(TableScanDesc sscan, ScanDirection direction,
}
void
-heap_rescan_bm(TableScanDesc sscan)
+heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa)
{
BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
@@ -1401,6 +1402,11 @@ heap_rescan_bm(TableScanDesc sscan)
* reinitialize heap scan descriptor
*/
initscan(&scan->heap_common, NULL, true);
+
+ bhs_begin_iterate(&scan->heap_common.rs_base.rs_bhs_iterator, tbm, dsa,
+ pstate ?
+ pstate->tbmiterator :
+ InvalidDsaPointer);
}
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 4f7138bbd9..f8627991d0 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -205,15 +205,13 @@ BitmapHeapNext(BitmapHeapScanState *node)
scan = table_beginscan_bm(node->ss.ss_currentScanDesc,
node->ss.ss_currentRelation,
node->ss.ps.state->es_snapshot,
- extra_flags);
+ extra_flags,
+ node->tbm,
+ node->pstate,
+ node->ss.ps.state->es_query_dsa);
node->ss.ss_currentScanDesc = scan;
- bhs_begin_iterate(&scan->rs_bhs_iterator, tbm, dsa,
- pstate ?
- pstate->tbmiterator :
- InvalidDsaPointer);
-
#ifdef USE_PREFETCH
if (node->prefetch_maximum > 0)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 545760aab5..7b44c94d29 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -305,7 +305,8 @@ extern bool heap_getnextslot_tidrange(TableScanDesc sscan,
extern TableScanDesc heap_beginscan_bm(Relation relation, Snapshot snapshot, uint32 flags);
extern void heap_endscan_bm(TableScanDesc scan);
-extern void heap_rescan_bm(TableScanDesc sscan);
+extern void heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa);
extern bool heap_fetch(Relation relation, Snapshot snapshot,
HeapTuple tuple, Buffer *userbuf, bool keep_buf);
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 3b0fa0610b..d02f5055b0 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -398,7 +398,8 @@ typedef struct TableAmRoutine
Snapshot snapshot,
uint32 flags);
- void (*scan_rescan_bm) (TableScanDesc scan);
+ void (*scan_rescan_bm) (TableScanDesc scan, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa);
/*
* Release resources and deallocate scan. If TableScanDesc.temp_snap,
@@ -949,7 +950,8 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(TableScanDesc scan, Relation rel, Snapshot snapshot,
- uint32 extra_flags)
+ uint32 extra_flags, TIDBitmap *tbm,
+ ParallelBitmapHeapState *pstate, dsa_area *dsa)
{
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
@@ -960,7 +962,7 @@ table_beginscan_bm(TableScanDesc scan, Relation rel, Snapshot snapshot,
if (!scan)
scan = rel->rd_tableam->scan_begin_bm(rel, snapshot, flags);
- scan->rs_rd->rd_tableam->scan_rescan_bm(scan);
+ scan->rs_rd->rd_tableam->scan_rescan_bm(scan, tbm, pstate, dsa);
return scan;
}
--
2.40.1
[text/x-diff] v16-0014-Move-BitmapHeapScan-current-block-iterator-to-Bi.patch (4.2K, ../../20240405235330.v4didvweb6isodtk@liskov/15-v16-0014-Move-BitmapHeapScan-current-block-iterator-to-Bi.patch)
download | inline diff:
From 91b04fc00567fe4395c9820eb95e7ee3e10af2ae Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 18:05:02 -0400
Subject: [PATCH v16 14/18] Move BitmapHeapScan current block iterator to
BitmapHeapScanDesc
After beginning iteration in heap AM code, we can move the current block
iterator into the BitmapHeapScanDesc and take it out of the generic
TableScanDesc.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 9 ++++++++-
src/backend/access/heap/heapam_handler.c | 2 +-
src/include/access/heapam.h | 1 +
src/include/access/relscan.h | 3 ---
src/include/access/tableam.h | 8 --------
5 files changed, 10 insertions(+), 13 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index a876a34b74..c35dc3c4e7 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -973,6 +973,9 @@ heap_beginscan_bm(Relation relation, Snapshot snapshot, uint32 flags)
initscan(&scan->heap_common, NULL, false);
+ scan->iterator.serial = NULL;
+ scan->iterator.parallel = NULL;
+
scan->vmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
@@ -1078,6 +1081,8 @@ heap_endscan_bm(TableScanDesc sscan)
if (BufferIsValid(scan->vmbuffer))
ReleaseBuffer(scan->vmbuffer);
+ bhs_end_iterate(&scan->iterator);
+
/*
* decrement relation reference count and free scan descriptor storage
*/
@@ -1396,6 +1401,8 @@ heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
ReleaseBuffer(scan->vmbuffer);
scan->vmbuffer = InvalidBuffer;
+ bhs_end_iterate(&scan->iterator);
+
scan->empty_tuples_pending = 0;
/*
@@ -1403,7 +1410,7 @@ heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
*/
initscan(&scan->heap_common, NULL, true);
- bhs_begin_iterate(&scan->heap_common.rs_base.rs_bhs_iterator, tbm, dsa,
+ bhs_begin_iterate(&scan->iterator, tbm, dsa,
pstate ?
pstate->tbmiterator :
InvalidDsaPointer);
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 43721cbd42..cc32d77100 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2209,7 +2209,7 @@ heapam_scan_bitmap_next_block(TableScanDesc sscan,
{
CHECK_FOR_INTERRUPTS();
- tbmres = bhs_iterate(&sscan->rs_bhs_iterator);
+ tbmres = bhs_iterate(&scan->iterator);
if (tbmres == NULL)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 7b44c94d29..e33e4433a7 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -92,6 +92,7 @@ typedef struct BitmapHeapScanDescData
/*
* Members common to Parallel and Serial BitmapHeapScan
*/
+ BitmapHeapIterator iterator;
/*
* These fields are only used for bitmap scans for the "skip fetch"
diff --git a/src/include/access/relscan.h b/src/include/access/relscan.h
index e520186b41..855b9558bf 100644
--- a/src/include/access/relscan.h
+++ b/src/include/access/relscan.h
@@ -41,9 +41,6 @@ typedef struct TableScanDescData
ItemPointerData rs_mintid;
ItemPointerData rs_maxtid;
- /* Only used for Bitmap table scans */
- BitmapHeapIterator rs_bhs_iterator;
-
/*
* Information about type and behaviour of the scan, a bitmask of members
* of the ScanOptions enum (see tableam.h).
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index d02f5055b0..f344d97f1f 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -1020,10 +1020,6 @@ table_beginscan_tid(Relation rel, Snapshot snapshot)
static inline void
table_endscan(TableScanDesc scan)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
- !scan->rs_bhs_iterator.exhausted)
- bhs_end_iterate(&scan->rs_bhs_iterator);
-
scan->rs_rd->rd_tableam->scan_end(scan);
}
@@ -1034,10 +1030,6 @@ static inline void
table_rescan(TableScanDesc scan,
struct ScanKeyData *key)
{
- if (scan->rs_flags & SO_TYPE_BITMAPSCAN &&
- !scan->rs_bhs_iterator.exhausted)
- bhs_end_iterate(&scan->rs_bhs_iterator);
-
scan->rs_rd->rd_tableam->scan_rescan(scan, key, false, false, false, false);
}
--
2.40.1
[text/x-diff] v16-0015-Lift-and-shift-BitmapHeapScan-prefetch-funcs-to-.patch (14.6K, ../../20240405235330.v4didvweb6isodtk@liskov/16-v16-0015-Lift-and-shift-BitmapHeapScan-prefetch-funcs-to-.patch)
download | inline diff:
From 86012b83948098f0c0f512676a275bf5595a1294 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 16:59:31 -0400
Subject: [PATCH v16 15/18] Lift and shift BitmapHeapScan prefetch funcs to
heap AM
We will soon move all the members for managing prefetching from the
BitmapHeapScanState to the BitmapHeapScanDesc. First move the prefetch
functions as-is to heapam_handler.c
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam_handler.c | 211 +++++++++++++++++++++
src/backend/executor/nodeBitmapHeapscan.c | 214 +---------------------
src/include/access/heapam.h | 4 +
3 files changed, 216 insertions(+), 213 deletions(-)
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index cc32d77100..ebaba33406 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2180,6 +2180,217 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
HEAP_USABLE_BYTES_PER_PAGE);
}
+/*
+ * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
+ *
+ * We keep track of how far the prefetch iterator is ahead of the main
+ * iterator in prefetch_pages. For each block the main iterator returns, we
+ * decrement prefetch_pages.
+ */
+void
+BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
+ TBMIterateResult *tbmpre;
+
+ if (pstate == NULL)
+ {
+ if (node->prefetch_pages > 0)
+ {
+ /* The main iterator has closed the distance by one page */
+ node->prefetch_pages--;
+ }
+ else if (!prefetch_iterator->exhausted)
+ {
+ /* Do not let the prefetch iterator get behind the main one */
+ tbmpre = bhs_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
+ return;
+ }
+
+ /*
+ * Adjusting the prefetch iterator before invoking
+ * table_scan_bitmap_next_block() keeps prefetch distance higher across
+ * the parallel workers.
+ */
+ if (node->prefetch_maximum > 0)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages > 0)
+ {
+ pstate->prefetch_pages--;
+ SpinLockRelease(&pstate->mutex);
+ }
+ else
+ {
+ /* Release the mutex before iterating */
+ SpinLockRelease(&pstate->mutex);
+
+ /*
+ * In case of shared mode, we can not ensure that the current
+ * blockno of the main iterator and that of the prefetch iterator
+ * are same. It's possible that whatever blockno we are
+ * prefetching will be processed by another process. Therefore,
+ * we don't validate the blockno here as we do in non-parallel
+ * case.
+ */
+ if (!prefetch_iterator->exhausted)
+ {
+ tbmpre = bhs_iterate(prefetch_iterator);
+ node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
+
+/*
+ * BitmapAdjustPrefetchTarget - Adjust the prefetch target
+ *
+ * Increase prefetch target if it's not yet at the max. Note that
+ * we will increase it to zero after fetching the very first
+ * page/tuple, then to one after the second tuple is fetched, then
+ * it doubles as later pages are fetched.
+ */
+void
+BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = node->pstate;
+
+ if (pstate == NULL)
+ {
+ if (node->prefetch_target >= node->prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (node->prefetch_target >= node->prefetch_maximum / 2)
+ node->prefetch_target = node->prefetch_maximum;
+ else if (node->prefetch_target > 0)
+ node->prefetch_target *= 2;
+ else
+ node->prefetch_target++;
+ return;
+ }
+
+ /* Do an unlocked check first to save spinlock acquisitions. */
+ if (pstate->prefetch_target < node->prefetch_maximum)
+ {
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_target >= node->prefetch_maximum)
+ /* don't increase any further */ ;
+ else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
+ pstate->prefetch_target = node->prefetch_maximum;
+ else if (pstate->prefetch_target > 0)
+ pstate->prefetch_target *= 2;
+ else
+ pstate->prefetch_target++;
+ SpinLockRelease(&pstate->mutex);
+ }
+#endif /* USE_PREFETCH */
+}
+
+/*
+ * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
+ */
+void
+BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
+{
+#ifdef USE_PREFETCH
+ ParallelBitmapHeapState *pstate = node->pstate;
+ BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
+
+ if (pstate == NULL)
+ {
+ if (!prefetch_iterator->exhausted)
+ {
+ while (node->prefetch_pages < node->prefetch_target)
+ {
+ TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
+ bool skip_fetch;
+
+ if (tbmpre == NULL)
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ break;
+ }
+ node->prefetch_pages++;
+ node->pfblockno = tbmpre->blockno;
+
+ /*
+ * If we expect not to have to actually read this heap page,
+ * skip this prefetch call, but continue to run the prefetch
+ * logic normally. (Would it be better not to increment
+ * prefetch_pages?)
+ */
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre->recheck &&
+ VM_ALL_VISIBLE(node->ss.ss_currentRelation,
+ tbmpre->blockno,
+ &node->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ }
+ }
+
+ return;
+ }
+
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ if (!prefetch_iterator->exhausted)
+ {
+ while (1)
+ {
+ TBMIterateResult *tbmpre;
+ bool do_prefetch = false;
+ bool skip_fetch;
+
+ /*
+ * Recheck under the mutex. If some other process has already
+ * done enough prefetching then we need not to do anything.
+ */
+ SpinLockAcquire(&pstate->mutex);
+ if (pstate->prefetch_pages < pstate->prefetch_target)
+ {
+ pstate->prefetch_pages++;
+ do_prefetch = true;
+ }
+ SpinLockRelease(&pstate->mutex);
+
+ if (!do_prefetch)
+ return;
+
+ tbmpre = bhs_iterate(prefetch_iterator);
+ if (tbmpre == NULL)
+ {
+ /* No more pages to prefetch */
+ bhs_end_iterate(prefetch_iterator);
+ break;
+ }
+
+ node->pfblockno = tbmpre->blockno;
+
+ /* As above, skip prefetch if we expect not to need page */
+ skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ !tbmpre->recheck &&
+ VM_ALL_VISIBLE(node->ss.ss_currentRelation,
+ tbmpre->blockno,
+ &node->pvmbuffer));
+
+ if (!skip_fetch)
+ PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ }
+ }
+ }
+#endif /* USE_PREFETCH */
+}
+
+
/* ------------------------------------------------------------------------
* Executor related callbacks for the heap AM
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index f8627991d0..6f1afd427d 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -40,6 +40,7 @@
#include "access/relscan.h"
#include "access/tableam.h"
#include "access/visibilitymap.h"
+#include "access/heapam.h"
#include "executor/executor.h"
#include "executor/nodeBitmapHeapscan.h"
#include "miscadmin.h"
@@ -51,10 +52,6 @@
static TupleTableSlot *BitmapHeapNext(BitmapHeapScanState *node);
static inline void BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate);
-static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-static inline void BitmapPrefetch(BitmapHeapScanState *node,
- TableScanDesc scan);
static bool BitmapShouldInitializeSharedState(ParallelBitmapHeapState *pstate);
/*
@@ -326,215 +323,6 @@ BitmapDoneInitializingSharedState(ParallelBitmapHeapState *pstate)
ConditionVariableBroadcast(&pstate->cv);
}
-/*
- * BitmapAdjustPrefetchIterator - Adjust the prefetch iterator
- *
- * We keep track of how far the prefetch iterator is ahead of the main
- * iterator in prefetch_pages. For each block the main iterator returns, we
- * decrement prefetch_pages.
- */
-static inline void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
- TBMIterateResult *tbmpre;
-
- if (pstate == NULL)
- {
- if (node->prefetch_pages > 0)
- {
- /* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
- }
- else if (!prefetch_iterator->exhausted)
- {
- /* Do not let the prefetch iterator get behind the main one */
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
- }
- return;
- }
-
- /*
- * Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
- */
- if (node->prefetch_maximum > 0)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages > 0)
- {
- pstate->prefetch_pages--;
- SpinLockRelease(&pstate->mutex);
- }
- else
- {
- /* Release the mutex before iterating */
- SpinLockRelease(&pstate->mutex);
-
- /*
- * In case of shared mode, we can not ensure that the current
- * blockno of the main iterator and that of the prefetch iterator
- * are same. It's possible that whatever blockno we are
- * prefetching will be processed by another process. Therefore,
- * we don't validate the blockno here as we do in non-parallel
- * case.
- */
- if (!prefetch_iterator->exhausted)
- {
- tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapAdjustPrefetchTarget - Adjust the prefetch target
- *
- * Increase prefetch target if it's not yet at the max. Note that
- * we will increase it to zero after fetching the very first
- * page/tuple, then to one after the second tuple is fetched, then
- * it doubles as later pages are fetched.
- */
-static inline void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
-
- if (pstate == NULL)
- {
- if (node->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
- else
- node->prefetch_target++;
- return;
- }
-
- /* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
- {
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
- /* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
- else if (pstate->prefetch_target > 0)
- pstate->prefetch_target *= 2;
- else
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-}
-
-/*
- * BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
- */
-static inline void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
-{
-#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
-
- if (pstate == NULL)
- {
- if (!prefetch_iterator->exhausted)
- {
- while (node->prefetch_pages < node->prefetch_target)
- {
- TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
- bool skip_fetch;
-
- if (tbmpre == NULL)
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- break;
- }
- node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
-
- /*
- * If we expect not to have to actually read this heap page,
- * skip this prefetch call, but continue to run the prefetch
- * logic normally. (Would it be better not to increment
- * prefetch_pages?)
- */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
- }
- }
-
- return;
- }
-
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- if (!prefetch_iterator->exhausted)
- {
- while (1)
- {
- TBMIterateResult *tbmpre;
- bool do_prefetch = false;
- bool skip_fetch;
-
- /*
- * Recheck under the mutex. If some other process has already
- * done enough prefetching then we need not to do anything.
- */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_pages < pstate->prefetch_target)
- {
- pstate->prefetch_pages++;
- do_prefetch = true;
- }
- SpinLockRelease(&pstate->mutex);
-
- if (!do_prefetch)
- return;
-
- tbmpre = bhs_iterate(prefetch_iterator);
- if (tbmpre == NULL)
- {
- /* No more pages to prefetch */
- bhs_end_iterate(prefetch_iterator);
- break;
- }
-
- node->pfblockno = tbmpre->blockno;
-
- /* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
- !tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
- tbmpre->blockno,
- &node->pvmbuffer));
-
- if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
- }
- }
- }
-#endif /* USE_PREFETCH */
-}
-
/*
* BitmapHeapRecheck -- access method routine to recheck a tuple in EvalPlanQual
*/
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index e33e4433a7..f81a295685 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -426,6 +426,10 @@ extern bool heapam_scan_analyze_next_tuple(TableScanDesc scan,
double *liverows, double *deadrows,
TupleTableSlot *slot);
+extern void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
+extern void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
+extern void BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan);
+
/*
* To avoid leaking too much knowledge about reorderbuffer implementation
* details this is implemented in reorderbuffer.c not heapam_visibility.c
--
2.40.1
[text/x-diff] v16-0016-Push-BitmapHeapScan-prefetch-code-into-heap-AM.patch (25.3K, ../../20240405235330.v4didvweb6isodtk@liskov/17-v16-0016-Push-BitmapHeapScan-prefetch-code-into-heap-AM.patch)
download | inline diff:
From 7da12cdb96ae5255c815d657048f7a812094fcfa Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 18:48:27 -0400
Subject: [PATCH v16 16/18] Push BitmapHeapScan prefetch code into heap AM
An earlier commit eliminated a table AM layering violation for the
current block in a BitmapHeapScan but left the violation in
BitmapHeapScan prefetching code.
To resolve this, move and manage all state used for prefetching entirely
into the heap AM.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 33 +++++-
src/backend/access/heap/heapam_handler.c | 125 +++++++++++++++-------
src/backend/executor/nodeBitmapHeapscan.c | 98 +++--------------
src/include/access/heapam.h | 29 ++++-
src/include/access/tableam.h | 18 +---
src/include/executor/nodeBitmapHeapscan.h | 7 ++
src/include/nodes/execnodes.h | 19 ----
7 files changed, 168 insertions(+), 161 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index c35dc3c4e7..5728b2ea8a 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -975,8 +975,10 @@ heap_beginscan_bm(Relation relation, Snapshot snapshot, uint32 flags)
scan->iterator.serial = NULL;
scan->iterator.parallel = NULL;
-
scan->vmbuffer = InvalidBuffer;
+ scan->pf_iterator.serial = NULL;
+ scan->pf_iterator.parallel = NULL;
+ scan->pvmbuffer = InvalidBuffer;
return (TableScanDesc) scan;
}
@@ -1081,6 +1083,10 @@ heap_endscan_bm(TableScanDesc sscan)
if (BufferIsValid(scan->vmbuffer))
ReleaseBuffer(scan->vmbuffer);
+ if (BufferIsValid(scan->pvmbuffer))
+ ReleaseBuffer(scan->pvmbuffer);
+
+ bhs_end_iterate(&scan->pf_iterator);
bhs_end_iterate(&scan->iterator);
/*
@@ -1390,7 +1396,7 @@ heap_getnextslot_tidrange(TableScanDesc sscan, ScanDirection direction,
void
heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
- ParallelBitmapHeapState *pstate, dsa_area *dsa)
+ ParallelBitmapHeapState *pstate, dsa_area *dsa, int pf_maximum)
{
BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
@@ -1401,9 +1407,19 @@ heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
ReleaseBuffer(scan->vmbuffer);
scan->vmbuffer = InvalidBuffer;
+ if (BufferIsValid(scan->pvmbuffer))
+ ReleaseBuffer(scan->pvmbuffer);
+ scan->pvmbuffer = InvalidBuffer;
+
+ bhs_end_iterate(&scan->pf_iterator);
bhs_end_iterate(&scan->iterator);
+ scan->prefetch_maximum = 0;
scan->empty_tuples_pending = 0;
+ scan->pstate = NULL;
+ scan->prefetch_target = -1;
+ scan->prefetch_pages = 0;
+ scan->pfblockno = InvalidBlockNumber;
/*
* reinitialize heap scan descriptor
@@ -1414,6 +1430,19 @@ heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
pstate ?
pstate->tbmiterator :
InvalidDsaPointer);
+
+#ifdef USE_PREFETCH
+ if (pf_maximum > 0)
+ {
+ bhs_begin_iterate(&scan->pf_iterator, tbm, dsa,
+ pstate ?
+ pstate->prefetch_iterator :
+ InvalidDsaPointer);
+ }
+#endif /* USE_PREFETCH */
+
+ scan->pstate = pstate;
+ scan->prefetch_maximum = pf_maximum;
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index ebaba33406..fca9c7233e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -60,6 +60,9 @@ static bool SampleHeapTupleVisible(TableScanDesc scan, Buffer buffer,
OffsetNumber tupoffset);
static BlockNumber heapam_scan_get_blocks_done(HeapScanDesc hscan);
+static inline void BitmapAdjustPrefetchIterator(BitmapHeapScanDesc scan);
+static inline void BitmapAdjustPrefetchTarget(BitmapHeapScanDesc scan);
+static inline void BitmapPrefetch(BitmapHeapScanDesc scan);
static const TableAmRoutine heapam_methods;
@@ -2187,26 +2190,26 @@ heapam_estimate_rel_size(Relation rel, int32 *attr_widths,
* iterator in prefetch_pages. For each block the main iterator returns, we
* decrement prefetch_pages.
*/
-void
-BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
+static inline void
+BitmapAdjustPrefetchIterator(BitmapHeapScanDesc scan)
{
#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
+ BitmapHeapIterator *prefetch_iterator = &scan->pf_iterator;
+ ParallelBitmapHeapState *pstate = scan->pstate;
TBMIterateResult *tbmpre;
if (pstate == NULL)
{
- if (node->prefetch_pages > 0)
+ if (scan->prefetch_pages > 0)
{
/* The main iterator has closed the distance by one page */
- node->prefetch_pages--;
+ scan->prefetch_pages--;
}
else if (!prefetch_iterator->exhausted)
{
/* Do not let the prefetch iterator get behind the main one */
tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ scan->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
return;
}
@@ -2216,7 +2219,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
* table_scan_bitmap_next_block() keeps prefetch distance higher across
* the parallel workers.
*/
- if (node->prefetch_maximum > 0)
+ if (scan->prefetch_maximum > 0)
{
SpinLockAcquire(&pstate->mutex);
if (pstate->prefetch_pages > 0)
@@ -2240,7 +2243,7 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
if (!prefetch_iterator->exhausted)
{
tbmpre = bhs_iterate(prefetch_iterator);
- node->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
+ scan->pfblockno = tbmpre ? tbmpre->blockno : InvalidBlockNumber;
}
}
}
@@ -2256,33 +2259,34 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanState *node)
* page/tuple, then to one after the second tuple is fetched, then
* it doubles as later pages are fetched.
*/
-void
-BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
+static inline void
+BitmapAdjustPrefetchTarget(BitmapHeapScanDesc scan)
{
#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
+ ParallelBitmapHeapState *pstate = scan->pstate;
+ int prefetch_maximum = scan->prefetch_maximum;
if (pstate == NULL)
{
- if (node->prefetch_target >= node->prefetch_maximum)
+ if (scan->prefetch_target >= prefetch_maximum)
/* don't increase any further */ ;
- else if (node->prefetch_target >= node->prefetch_maximum / 2)
- node->prefetch_target = node->prefetch_maximum;
- else if (node->prefetch_target > 0)
- node->prefetch_target *= 2;
+ else if (scan->prefetch_target >= prefetch_maximum / 2)
+ scan->prefetch_target = prefetch_maximum;
+ else if (scan->prefetch_target > 0)
+ scan->prefetch_target *= 2;
else
- node->prefetch_target++;
+ scan->prefetch_target++;
return;
}
/* Do an unlocked check first to save spinlock acquisitions. */
- if (pstate->prefetch_target < node->prefetch_maximum)
+ if (pstate->prefetch_target < prefetch_maximum)
{
SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target >= node->prefetch_maximum)
+ if (pstate->prefetch_target >= prefetch_maximum)
/* don't increase any further */ ;
- else if (pstate->prefetch_target >= node->prefetch_maximum / 2)
- pstate->prefetch_target = node->prefetch_maximum;
+ else if (pstate->prefetch_target >= prefetch_maximum / 2)
+ pstate->prefetch_target = prefetch_maximum;
else if (pstate->prefetch_target > 0)
pstate->prefetch_target *= 2;
else
@@ -2295,18 +2299,19 @@ BitmapAdjustPrefetchTarget(BitmapHeapScanState *node)
/*
* BitmapPrefetch - Prefetch, if prefetch_pages are behind prefetch_target
*/
-void
-BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
+static inline void
+BitmapPrefetch(BitmapHeapScanDesc scan)
{
#ifdef USE_PREFETCH
- ParallelBitmapHeapState *pstate = node->pstate;
- BitmapHeapIterator *prefetch_iterator = &node->pf_iterator;
+ ParallelBitmapHeapState *pstate = scan->pstate;
+ BitmapHeapIterator *prefetch_iterator = &scan->pf_iterator;
+ Relation rel = scan->heap_common.rs_base.rs_rd;
if (pstate == NULL)
{
if (!prefetch_iterator->exhausted)
{
- while (node->prefetch_pages < node->prefetch_target)
+ while (scan->prefetch_pages < scan->prefetch_target)
{
TBMIterateResult *tbmpre = bhs_iterate(prefetch_iterator);
bool skip_fetch;
@@ -2317,8 +2322,8 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
bhs_end_iterate(prefetch_iterator);
break;
}
- node->prefetch_pages++;
- node->pfblockno = tbmpre->blockno;
+ scan->prefetch_pages++;
+ scan->pfblockno = tbmpre->blockno;
/*
* If we expect not to have to actually read this heap page,
@@ -2326,14 +2331,14 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
* logic normally. (Would it be better not to increment
* prefetch_pages?)
*/
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ skip_fetch = (!(scan->heap_common.rs_base.rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
+ VM_ALL_VISIBLE(rel,
tbmpre->blockno,
- &node->pvmbuffer));
+ &scan->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(rel, MAIN_FORKNUM, tbmpre->blockno);
}
}
@@ -2373,17 +2378,17 @@ BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan)
break;
}
- node->pfblockno = tbmpre->blockno;
+ scan->pfblockno = tbmpre->blockno;
/* As above, skip prefetch if we expect not to need page */
- skip_fetch = (!(scan->rs_flags & SO_NEED_TUPLE) &&
+ skip_fetch = (!(scan->heap_common.rs_base.rs_flags & SO_NEED_TUPLE) &&
!tbmpre->recheck &&
- VM_ALL_VISIBLE(node->ss.ss_currentRelation,
+ VM_ALL_VISIBLE(rel,
tbmpre->blockno,
- &node->pvmbuffer));
+ &scan->pvmbuffer));
if (!skip_fetch)
- PrefetchBuffer(scan->rs_rd, MAIN_FORKNUM, tbmpre->blockno);
+ PrefetchBuffer(rel, MAIN_FORKNUM, tbmpre->blockno);
}
}
}
@@ -2416,6 +2421,8 @@ heapam_scan_bitmap_next_block(TableScanDesc sscan,
*blockno = InvalidBlockNumber;
*recheck = true;
+ BitmapAdjustPrefetchIterator(scan);
+
do
{
CHECK_FOR_INTERRUPTS();
@@ -2556,6 +2563,18 @@ heapam_scan_bitmap_next_block(TableScanDesc sscan,
else
(*exact_pages)++;
+ /*
+ * If serial, we can error out if the the prefetch block doesn't stay
+ * ahead of the current block.
+ */
+ if (scan->pstate == NULL &&
+ !scan->pf_iterator.exhausted &&
+ scan->pfblockno < block)
+ elog(ERROR, "prefetch and main iterators are out of sync");
+
+ /* Adjust the prefetch target */
+ BitmapAdjustPrefetchTarget(scan);
+
/*
* Return true to indicate that a valid block was found and the bitmap is
* not exhausted. If there are no visible tuples on this page,
@@ -2592,6 +2611,36 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
return false;
+#ifdef USE_PREFETCH
+
+ /*
+ * Try to prefetch at least a few pages even before we get to the second
+ * page if we don't stop reading after the first tuple.
+ */
+ if (!bscan->pstate)
+ {
+ if (bscan->prefetch_target < bscan->prefetch_maximum)
+ bscan->prefetch_target++;
+ }
+ else if (bscan->pstate->prefetch_target < bscan->prefetch_maximum)
+ {
+ /* take spinlock while updating shared state */
+ SpinLockAcquire(&bscan->pstate->mutex);
+ if (bscan->pstate->prefetch_target < bscan->prefetch_maximum)
+ bscan->pstate->prefetch_target++;
+ SpinLockRelease(&bscan->pstate->mutex);
+ }
+
+ /*
+ * We issue prefetch requests *after* fetching the current page to try to
+ * avoid having prefetching interfere with the main I/O. Also, this should
+ * happen only when we have determined there is still something to do on
+ * the current page, else we may uselessly prefetch the same page we are
+ * just about to request for real.
+ */
+ BitmapPrefetch(bscan);
+#endif /* USE_PREFETCH */
+
targoffset = hscan->rs_vistuples[hscan->rs_cindex];
page = BufferGetPage(hscan->rs_cbuf);
lp = PageGetItemId(page, targoffset);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index 6f1afd427d..ae36dfd773 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -124,7 +124,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
TIDBitmap *tbm;
TupleTableSlot *slot;
ParallelBitmapHeapState *pstate = node->pstate;
- dsa_area *dsa = node->ss.ps.state->es_query_dsa;
/*
* extract necessary information from index scan node
@@ -148,7 +147,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
if (!node->initialized)
{
+ Relation rel = node->ss.ss_currentRelation;
uint32 extra_flags = 0;
+ bool pf_maximum = 0;
/*
* The leader will immediately come out of the function, but others
@@ -157,6 +158,15 @@ BitmapHeapNext(BitmapHeapScanState *node)
bool init_shared_state = node->pstate ?
BitmapShouldInitializeSharedState(node->pstate) : false;
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(rel->rd_rel->reltablespace);
+#endif
+
+
if (!pstate || init_shared_state)
{
tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
@@ -174,7 +184,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
*/
pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
+ if (pf_maximum > 0)
{
pstate->prefetch_iterator =
tbm_prepare_shared_iterate(tbm);
@@ -200,25 +210,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
* scan descriptor and begin the scan.
*/
scan = table_beginscan_bm(node->ss.ss_currentScanDesc,
- node->ss.ss_currentRelation,
+ rel,
node->ss.ps.state->es_snapshot,
extra_flags,
+ pf_maximum,
node->tbm,
node->pstate,
node->ss.ps.state->es_query_dsa);
node->ss.ss_currentScanDesc = scan;
-#ifdef USE_PREFETCH
- if (node->prefetch_maximum > 0)
- {
- bhs_begin_iterate(&node->pf_iterator, tbm, dsa,
- pstate ?
- pstate->prefetch_iterator :
- InvalidDsaPointer);
- }
-#endif /* USE_PREFETCH */
-
node->initialized = true;
goto new_page;
@@ -230,37 +231,6 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
CHECK_FOR_INTERRUPTS();
-#ifdef USE_PREFETCH
-
- /*
- * Try to prefetch at least a few pages even before we get to the
- * second page if we don't stop reading after the first tuple.
- */
- if (!pstate)
- {
- if (node->prefetch_target < node->prefetch_maximum)
- node->prefetch_target++;
- }
- else if (pstate->prefetch_target < node->prefetch_maximum)
- {
- /* take spinlock while updating shared state */
- SpinLockAcquire(&pstate->mutex);
- if (pstate->prefetch_target < node->prefetch_maximum)
- pstate->prefetch_target++;
- SpinLockRelease(&pstate->mutex);
- }
-#endif /* USE_PREFETCH */
-
- /*
- * We issue prefetch requests *after* fetching the current page to
- * try to avoid having prefetching interfere with the main I/O.
- * Also, this should happen only when we have determined there is
- * still something to do on the current page, else we may
- * uselessly prefetch the same page we are just about to request
- * for real.
- */
- BitmapPrefetch(node, scan);
-
/*
* If we are using lossy info, we have to recheck the qual
* conditions at every tuple.
@@ -283,23 +253,9 @@ BitmapHeapNext(BitmapHeapScanState *node)
new_page:
- BitmapAdjustPrefetchIterator(node);
-
if (!table_scan_bitmap_next_block(scan, &node->blockno, &node->recheck,
&node->lossy_pages, &node->exact_pages))
break;
-
- /*
- * If serial, we can error out if the the prefetch block doesn't stay
- * ahead of the current block.
- */
- if (node->pstate == NULL &&
- !node->pf_iterator.exhausted &&
- node->pfblockno < node->blockno)
- elog(ERROR, "prefetch and main iterators are out of sync");
-
- /* Adjust the prefetch target */
- BitmapAdjustPrefetchTarget(node);
}
/*
@@ -365,21 +321,12 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
PlanState *outerPlan = outerPlanState(node);
/* release bitmaps and buffers if any */
- if (node->pf_iterator.exhausted)
- bhs_end_iterate(&node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
node->tbm = NULL;
node->initialized = false;
- node->pvmbuffer = InvalidBuffer;
node->recheck = true;
node->blockno = InvalidBlockNumber;
- node->pfblockno = InvalidBlockNumber;
- /* Only used for serial BHS */
- node->prefetch_pages = 0;
- node->prefetch_target = -1;
ExecScanReScan(&node->ss);
@@ -418,14 +365,10 @@ ExecEndBitmapHeapScan(BitmapHeapScanState *node)
table_endscan_bm(scanDesc);
/*
- * release bitmaps and buffers if any
+ * release bitmaps if any
*/
- if (!node->pf_iterator.exhausted)
- bhs_end_iterate(&node->pf_iterator);
if (node->tbm)
tbm_free(node->tbm);
- if (node->pvmbuffer != InvalidBuffer)
- ReleaseBuffer(node->pvmbuffer);
}
/* ----------------------------------------------------------------
@@ -458,16 +401,12 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->ss.ps.ExecProcNode = ExecBitmapHeapScan;
scanstate->tbm = NULL;
- scanstate->pvmbuffer = InvalidBuffer;
scanstate->exact_pages = 0;
scanstate->lossy_pages = 0;
- scanstate->prefetch_pages = 0;
- scanstate->prefetch_target = -1;
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
scanstate->blockno = InvalidBlockNumber;
- scanstate->pfblockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
@@ -507,13 +446,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->bitmapqualorig =
ExecInitQual(node->bitmapqualorig, (PlanState *) scanstate);
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
- scanstate->prefetch_maximum =
- get_tablespace_io_concurrency(currentRelation->rd_rel->reltablespace);
-
scanstate->ss.ss_currentRelation = currentRelation;
/*
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index f81a295685..499e739da7 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -93,6 +93,10 @@ typedef struct BitmapHeapScanDescData
* Members common to Parallel and Serial BitmapHeapScan
*/
BitmapHeapIterator iterator;
+ BitmapHeapIterator pf_iterator;
+
+ /* maximum value for prefetch_target */
+ int prefetch_maximum;
/*
* These fields are only used for bitmap scans for the "skip fetch"
@@ -102,7 +106,26 @@ typedef struct BitmapHeapScanDescData
* to return. They are common to parallel and serial BitmapHeapScans
*/
Buffer vmbuffer;
+ /* buffer for visibility-map lookups of prefetched pages */
+ Buffer pvmbuffer;
int empty_tuples_pending;
+
+ /*
+ * Parallel-only members
+ */
+
+ struct ParallelBitmapHeapState *pstate;
+
+ /*
+ * Serial-only members
+ */
+
+ /* Current target for prefetch distance */
+ int prefetch_target;
+ /* # pages prefetch iterator is ahead of current */
+ int prefetch_pages;
+ /* used to validate prefetch block stays ahead of current block */
+ BlockNumber pfblockno;
} BitmapHeapScanDescData;
typedef struct BitmapHeapScanDescData *BitmapHeapScanDesc;
@@ -307,7 +330,7 @@ extern TableScanDesc heap_beginscan_bm(Relation relation, Snapshot snapshot, uin
extern void heap_endscan_bm(TableScanDesc scan);
extern void heap_rescan_bm(TableScanDesc sscan, TIDBitmap *tbm,
- ParallelBitmapHeapState *pstate, dsa_area *dsa);
+ ParallelBitmapHeapState *pstate, dsa_area *dsa, int pf_maximum);
extern bool heap_fetch(Relation relation, Snapshot snapshot,
HeapTuple tuple, Buffer *userbuf, bool keep_buf);
@@ -426,10 +449,6 @@ extern bool heapam_scan_analyze_next_tuple(TableScanDesc scan,
double *liverows, double *deadrows,
TupleTableSlot *slot);
-extern void BitmapAdjustPrefetchIterator(BitmapHeapScanState *node);
-extern void BitmapAdjustPrefetchTarget(BitmapHeapScanState *node);
-extern void BitmapPrefetch(BitmapHeapScanState *node, TableScanDesc scan);
-
/*
* To avoid leaking too much knowledge about reorderbuffer implementation
* details this is implemented in reorderbuffer.c not heapam_visibility.c
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index f344d97f1f..9ec0a1da2b 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -399,7 +399,8 @@ typedef struct TableAmRoutine
uint32 flags);
void (*scan_rescan_bm) (TableScanDesc scan, TIDBitmap *tbm,
- ParallelBitmapHeapState *pstate, dsa_area *dsa);
+ ParallelBitmapHeapState *pstate, dsa_area *dsa,
+ int pf_maximum);
/*
* Release resources and deallocate scan. If TableScanDesc.temp_snap,
@@ -801,17 +802,6 @@ typedef struct TableAmRoutine
* lossy_pages is incremented if the bitmap is lossy for the selected
* block; otherwise, exact_pages is incremented.
*
- * XXX: Currently this may only be implemented if the AM uses md.c as its
- * storage manager, and uses ItemPointer->ip_blkid in a manner that maps
- * blockids directly to the underlying storage. nodeBitmapHeapscan.c
- * performs prefetching directly using that interface. This probably
- * needs to be rectified at a later point.
- *
- * XXX: Currently this may only be implemented if the AM uses the
- * visibilitymap, as nodeBitmapHeapscan.c unconditionally accesses it to
- * perform prefetching. This probably needs to be rectified at a later
- * point.
- *
* Optional callback, but either both scan_bitmap_next_block and
* scan_bitmap_next_tuple need to exist, or neither.
*/
@@ -950,7 +940,7 @@ table_beginscan_strat(Relation rel, Snapshot snapshot,
*/
static inline TableScanDesc
table_beginscan_bm(TableScanDesc scan, Relation rel, Snapshot snapshot,
- uint32 extra_flags, TIDBitmap *tbm,
+ uint32 extra_flags, int pf_maximum, TIDBitmap *tbm,
ParallelBitmapHeapState *pstate, dsa_area *dsa)
{
uint32 flags = SO_TYPE_BITMAPSCAN | SO_ALLOW_PAGEMODE | extra_flags;
@@ -962,7 +952,7 @@ table_beginscan_bm(TableScanDesc scan, Relation rel, Snapshot snapshot,
if (!scan)
scan = rel->rd_tableam->scan_begin_bm(rel, snapshot, flags);
- scan->rs_rd->rd_tableam->scan_rescan_bm(scan, tbm, pstate, dsa);
+ scan->rs_rd->rd_tableam->scan_rescan_bm(scan, tbm, pstate, dsa, pf_maximum);
return scan;
}
diff --git a/src/include/executor/nodeBitmapHeapscan.h b/src/include/executor/nodeBitmapHeapscan.h
index 7064f54686..028081db32 100644
--- a/src/include/executor/nodeBitmapHeapscan.h
+++ b/src/include/executor/nodeBitmapHeapscan.h
@@ -29,6 +29,13 @@ extern void ExecBitmapHeapReInitializeDSM(BitmapHeapScanState *node,
extern void ExecBitmapHeapInitializeWorker(BitmapHeapScanState *node,
ParallelWorkerContext *pwcxt);
+typedef struct BitmapHeapIterator
+{
+ struct TBMIterator *serial;
+ struct TBMSharedIterator *parallel;
+ bool exhausted;
+} BitmapHeapIterator;
+
extern TBMIterateResult *bhs_iterate(BitmapHeapIterator *iterator);
extern void bhs_begin_iterate(BitmapHeapIterator *iterator, TIDBitmap *tbm,
dsa_area *dsa, dsa_pointer dsp);
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 714eb3e534..713de7d50f 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1787,30 +1787,17 @@ typedef struct ParallelBitmapHeapState
ConditionVariable cv;
} ParallelBitmapHeapState;
-typedef struct BitmapHeapIterator
-{
- struct TBMIterator *serial;
- struct TBMSharedIterator *parallel;
- bool exhausted;
-} BitmapHeapIterator;
-
/* ----------------
* BitmapHeapScanState information
*
* bitmapqualorig execution state for bitmapqualorig expressions
* tbm bitmap obtained from child index scan(s)
- * pvmbuffer buffer for visibility-map lookups of prefetched pages
* exact_pages total number of exact pages retrieved
* lossy_pages total number of lossy pages retrieved
- * pf_iterator for prefetching ahead of current page
- * prefetch_pages # pages prefetch iterator is ahead of current
- * prefetch_target current target prefetch distance
- * prefetch_maximum maximum value for prefetch_target
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
* blockno used to validate pf and current block in sync
- * pfblockno used to validate pf stays ahead of current block
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1818,18 +1805,12 @@ typedef struct BitmapHeapScanState
ScanState ss; /* its first field is NodeTag */
ExprState *bitmapqualorig;
TIDBitmap *tbm;
- Buffer pvmbuffer;
long exact_pages;
long lossy_pages;
- int prefetch_pages;
- int prefetch_target;
- int prefetch_maximum;
bool initialized;
- BitmapHeapIterator pf_iterator;
ParallelBitmapHeapState *pstate;
bool recheck;
BlockNumber blockno;
- BlockNumber pfblockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
[text/x-diff] v16-0017-Move-BitmapHeapScan-initialization-to-helper.patch (8.3K, ../../20240405235330.v4didvweb6isodtk@liskov/18-v16-0017-Move-BitmapHeapScan-initialization-to-helper.patch)
download | inline diff:
From d66c526696f6bcb9d6aa0a8b234d51c58866a6ff Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 19:02:38 -0400
Subject: [PATCH v16 17/18] Move BitmapHeapScan initialization to helper
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/executor/nodeBitmapHeapscan.c | 182 +++++++++++-----------
src/include/access/tableam.h | 6 +-
2 files changed, 98 insertions(+), 90 deletions(-)
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index ae36dfd773..ce47cd51bd 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -109,6 +109,97 @@ bhs_end_iterate(BitmapHeapIterator *iterator)
iterator->exhausted = true;
}
+ /*
+ * If we haven't yet performed the underlying index scan, do it, and begin
+ * the iteration over the bitmap.
+ *
+ * For prefetching, we use *two* iterators, one for the pages we are actually
+ * scanning and another that runs ahead of the first for prefetching.
+ * node->prefetch_pages tracks exactly how many pages ahead the prefetch
+ * iterator is. Also, node->prefetch_target tracks the desired prefetch
+ * distance, which starts small and increases up to the
+ * scan->prefetch_maximum. This is to avoid doing a lot of prefetching in a
+ * scan that stops after a few tuples because of a LIMIT.
+ */
+static void
+BitmapHeapInitialize(BitmapHeapScanState *node)
+{
+ TIDBitmap *tbm;
+ ParallelBitmapHeapState *pstate = node->pstate;
+ Relation rel = node->ss.ss_currentRelation;
+ uint32 extra_flags = 0;
+ bool pf_maximum = 0;
+
+ /*
+ * The leader will immediately come out of the function, but others will
+ * be blocked until leader populates the TBM and wakes them up.
+ */
+ bool init_shared_state = node->pstate ?
+ BitmapShouldInitializeSharedState(node->pstate) : false;
+
+ /*
+ * Maximum number of prefetches for the tablespace if configured,
+ * otherwise the current value of the effective_io_concurrency GUC.
+ */
+#ifdef USE_PREFETCH
+ pf_maximum = get_tablespace_io_concurrency(rel->rd_rel->reltablespace);
+#endif
+
+
+ if (!pstate || init_shared_state)
+ {
+ tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
+
+ if (!tbm || !IsA(tbm, TIDBitmap))
+ elog(ERROR, "unrecognized result from subplan");
+ node->tbm = tbm;
+
+ if (init_shared_state)
+ {
+ /*
+ * Prepare to iterate over the TBM. This will return the
+ * dsa_pointer of the iterator state which will be used by
+ * multiple processes to iterate jointly.
+ */
+ pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
+#ifdef USE_PREFETCH
+ if (pf_maximum > 0)
+ {
+ pstate->prefetch_iterator =
+ tbm_prepare_shared_iterate(tbm);
+ }
+#endif
+ /* We have initialized the shared state so wake up others. */
+ BitmapDoneInitializingSharedState(pstate);
+ }
+ }
+
+ /*
+ * We can potentially skip fetching heap pages if we do not need any
+ * columns of the table, either for checking non-indexable quals or for
+ * returning data. This test is a bit simplistic, as it checks the
+ * stronger condition that there's no qual or return tlist at all. But in
+ * most cases it's probably not worth working harder than that.
+ */
+ if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
+ extra_flags |= SO_NEED_TUPLE;
+
+ /*
+ * If this is the first scan of the underlying table, create the table
+ * scan descriptor and begin the scan.
+ */
+ node->ss.ss_currentScanDesc = table_beginscan_bm(node->ss.ss_currentScanDesc,
+ rel,
+ node->ss.ps.state->es_snapshot,
+ extra_flags,
+ pf_maximum,
+ node->tbm,
+ node->pstate,
+ node->ss.ps.state->es_query_dsa);
+
+
+ node->initialized = true;
+}
/* ----------------------------------------------------------------
* BitmapHeapNext
@@ -121,9 +212,7 @@ BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
TableScanDesc scan;
- TIDBitmap *tbm;
TupleTableSlot *slot;
- ParallelBitmapHeapState *pstate = node->pstate;
/*
* extract necessary information from index scan node
@@ -131,97 +220,16 @@ BitmapHeapNext(BitmapHeapScanState *node)
econtext = node->ss.ps.ps_ExprContext;
slot = node->ss.ss_ScanTupleSlot;
scan = node->ss.ss_currentScanDesc;
- tbm = node->tbm;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
- * the iteration over the bitmap.
- *
- * For prefetching, we use *two* iterators, one for the pages we are
- * actually scanning and another that runs ahead of the first for
- * prefetching. node->prefetch_pages tracks exactly how many pages ahead
- * the prefetch iterator is. Also, node->prefetch_target tracks the
- * desired prefetch distance, which starts small and increases up to the
- * node->prefetch_maximum. This is to avoid doing a lot of prefetching in
- * a scan that stops after a few tuples because of a LIMIT.
+ * the iteration over the bitmap. This happens on rescan as well.
*/
if (!node->initialized)
{
- Relation rel = node->ss.ss_currentRelation;
- uint32 extra_flags = 0;
- bool pf_maximum = 0;
-
- /*
- * The leader will immediately come out of the function, but others
- * will be blocked until leader populates the TBM and wakes them up.
- */
- bool init_shared_state = node->pstate ?
- BitmapShouldInitializeSharedState(node->pstate) : false;
-
- /*
- * Maximum number of prefetches for the tablespace if configured,
- * otherwise the current value of the effective_io_concurrency GUC.
- */
-#ifdef USE_PREFETCH
- pf_maximum = get_tablespace_io_concurrency(rel->rd_rel->reltablespace);
-#endif
-
-
- if (!pstate || init_shared_state)
- {
- tbm = (TIDBitmap *) MultiExecProcNode(outerPlanState(node));
-
- if (!tbm || !IsA(tbm, TIDBitmap))
- elog(ERROR, "unrecognized result from subplan");
- node->tbm = tbm;
-
- if (init_shared_state)
- {
- /*
- * Prepare to iterate over the TBM. This will return the
- * dsa_pointer of the iterator state which will be used by
- * multiple processes to iterate jointly.
- */
- pstate->tbmiterator = tbm_prepare_shared_iterate(tbm);
-#ifdef USE_PREFETCH
- if (pf_maximum > 0)
- {
- pstate->prefetch_iterator =
- tbm_prepare_shared_iterate(tbm);
- }
-#endif
- /* We have initialized the shared state so wake up others. */
- BitmapDoneInitializingSharedState(pstate);
- }
- }
-
- /*
- * We can potentially skip fetching heap pages if we do not need any
- * columns of the table, either for checking non-indexable quals or
- * for returning data. This test is a bit simplistic, as it checks
- * the stronger condition that there's no qual or return tlist at all.
- * But in most cases it's probably not worth working harder than that.
- */
- if (node->ss.ps.plan->qual != NIL || node->ss.ps.plan->targetlist != NIL)
- extra_flags |= SO_NEED_TUPLE;
-
- /*
- * If this is the first scan of the underlying table, create the table
- * scan descriptor and begin the scan.
- */
- scan = table_beginscan_bm(node->ss.ss_currentScanDesc,
- rel,
- node->ss.ps.state->es_snapshot,
- extra_flags,
- pf_maximum,
- node->tbm,
- node->pstate,
- node->ss.ps.state->es_query_dsa);
-
- node->ss.ss_currentScanDesc = scan;
-
- node->initialized = true;
-
+ BitmapHeapInitialize(node);
+ /* We may have a new scan descriptor */
+ scan = node->ss.ss_currentScanDesc;
goto new_page;
}
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index 9ec0a1da2b..eea7b7ff20 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -791,9 +791,9 @@ typedef struct TableAmRoutine
*/
/*
- * Prepare to fetch / check / return tuples from `blockno` as part of a
- * bitmap table scan. `scan` was started via table_beginscan_bm(). Return
- * false if the bitmap is exhausted and true otherwise.
+ * Prepare to fetch / check / return tuples as part of a bitmap table
+ * scan. `scan` was started via table_beginscan_bm(). Return false if the
+ * bitmap is exhausted and true otherwise.
*
* This will typically read and pin the target block, and do the necessary
* work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
--
2.40.1
[text/x-diff] v16-0018-Remove-table_scan_bitmap_next_block.patch (14.3K, ../../20240405235330.v4didvweb6isodtk@liskov/19-v16-0018-Remove-table_scan_bitmap_next_block.patch)
download | inline diff:
From 6a2a8b7d147e3c5343512bcf4d1d8389ff71bbe8 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <[email protected]>
Date: Fri, 5 Apr 2024 18:56:45 -0400
Subject: [PATCH v16 18/18] Remove table_scan_bitmap_next_block()
With several of the changes to the control flow of BitmapHeapNext() in
recent commits, table_scan_bitmap_next_tuple() can be responsible for
getting the next block. Do this and remove the table AM API function
table_scan_bitmap_next_block().
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_ZwCwWFeL_H3ia26bP2e7HiKLWt0ZmGXPVwPO6uXq0vaA%40mail.gmail.com
---
src/backend/access/heap/heapam.c | 2 +
src/backend/access/heap/heapam_handler.c | 53 +++++++++-------
src/backend/access/table/tableamapi.c | 2 -
src/backend/executor/nodeBitmapHeapscan.c | 69 +++++++++------------
src/backend/optimizer/util/plancat.c | 2 +-
src/include/access/tableam.h | 73 +++++------------------
src/include/nodes/execnodes.h | 2 -
7 files changed, 78 insertions(+), 125 deletions(-)
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 5728b2ea8a..3ee4d75f17 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -327,6 +327,8 @@ initscan(HeapScanDesc scan, ScanKey key, bool keep_startblock)
ItemPointerSetInvalid(&scan->rs_ctup.t_self);
scan->rs_cbuf = InvalidBuffer;
scan->rs_cblock = InvalidBlockNumber;
+ scan->rs_cindex = 0;
+ scan->rs_ntuples = 0;
/* page-at-a-time fields are always invalid when not rs_inited */
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index fca9c7233e..a6d75cfaa8 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2216,8 +2216,8 @@ BitmapAdjustPrefetchIterator(BitmapHeapScanDesc scan)
/*
* Adjusting the prefetch iterator before invoking
- * table_scan_bitmap_next_block() keeps prefetch distance higher across
- * the parallel workers.
+ * heapam_bitmap_next_block() keeps prefetch distance higher across the
+ * parallel workers.
*/
if (scan->prefetch_maximum > 0)
{
@@ -2397,14 +2397,12 @@ BitmapPrefetch(BitmapHeapScanDesc scan)
-/* ------------------------------------------------------------------------
- * Executor related callbacks for the heap AM
- * ------------------------------------------------------------------------
+/*
+ * Helper for heapam_scan_bitmap_next_tuple()
*/
-
static bool
heapam_scan_bitmap_next_block(TableScanDesc sscan,
- BlockNumber *blockno, bool *recheck,
+ bool *recheck,
long *lossy_pages, long *exact_pages)
{
BitmapHeapScanDesc scan = (BitmapHeapScanDesc) sscan;
@@ -2418,7 +2416,6 @@ heapam_scan_bitmap_next_block(TableScanDesc sscan,
hscan->rs_cindex = 0;
hscan->rs_ntuples = 0;
- *blockno = InvalidBlockNumber;
*recheck = true;
BitmapAdjustPrefetchIterator(scan);
@@ -2447,7 +2444,6 @@ heapam_scan_bitmap_next_block(TableScanDesc sscan,
} while (!IsolationIsSerializable() && tbmres->blockno >= hscan->rs_nblocks);
/* Got a valid block */
- *blockno = tbmres->blockno;
*recheck = tbmres->recheck;
/*
@@ -2585,9 +2581,16 @@ heapam_scan_bitmap_next_block(TableScanDesc sscan,
return true;
}
+
+/* ------------------------------------------------------------------------
+ * Executor related callbacks for the heap AM
+ * ------------------------------------------------------------------------
+ */
+
static bool
heapam_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
BitmapHeapScanDesc bscan = (BitmapHeapScanDesc) scan;
HeapScanDesc hscan = &bscan->heap_common;
@@ -2595,21 +2598,28 @@ heapam_scan_bitmap_next_tuple(TableScanDesc scan,
Page page;
ItemId lp;
- if (bscan->empty_tuples_pending > 0)
+ /*
+ * Out of range? If so, nothing more to look at on this page
+ */
+ while (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
{
/*
- * If we don't have to fetch the tuple, just return nulls.
+ * Emit empty tuples before advancing to the next block
*/
- ExecStoreAllNullTuple(slot);
- bscan->empty_tuples_pending--;
- return true;
- }
+ if (bscan->empty_tuples_pending > 0)
+ {
+ /*
+ * If we don't have to fetch the tuple, just return nulls.
+ */
+ ExecStoreAllNullTuple(slot);
+ bscan->empty_tuples_pending--;
+ return true;
+ }
- /*
- * Out of range? If so, nothing more to look at on this page
- */
- if (hscan->rs_cindex < 0 || hscan->rs_cindex >= hscan->rs_ntuples)
- return false;
+ if (!heapam_scan_bitmap_next_block(scan, recheck,
+ lossy_pages, exact_pages))
+ return false;
+ }
#ifdef USE_PREFETCH
@@ -3010,7 +3020,6 @@ static const TableAmRoutine heapam_methods = {
.relation_estimate_size = heapam_estimate_rel_size,
- .scan_bitmap_next_block = heapam_scan_bitmap_next_block,
.scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple,
.scan_sample_next_block = heapam_scan_sample_next_block,
.scan_sample_next_tuple = heapam_scan_sample_next_tuple
diff --git a/src/backend/access/table/tableamapi.c b/src/backend/access/table/tableamapi.c
index 55b8caeadf..0b87530a9c 100644
--- a/src/backend/access/table/tableamapi.c
+++ b/src/backend/access/table/tableamapi.c
@@ -90,8 +90,6 @@ GetTableAmRoutine(Oid amhandler)
Assert(routine->relation_estimate_size != NULL);
/* optional, but one callback implies presence of the other */
- Assert((routine->scan_bitmap_next_block == NULL) ==
- (routine->scan_bitmap_next_tuple == NULL));
Assert(routine->scan_sample_next_block != NULL);
Assert(routine->scan_sample_next_tuple != NULL);
diff --git a/src/backend/executor/nodeBitmapHeapscan.c b/src/backend/executor/nodeBitmapHeapscan.c
index ce47cd51bd..3ce24156dd 100644
--- a/src/backend/executor/nodeBitmapHeapscan.c
+++ b/src/backend/executor/nodeBitmapHeapscan.c
@@ -211,61 +211,52 @@ static TupleTableSlot *
BitmapHeapNext(BitmapHeapScanState *node)
{
ExprContext *econtext;
- TableScanDesc scan;
TupleTableSlot *slot;
-
- /*
- * extract necessary information from index scan node
- */
- econtext = node->ss.ps.ps_ExprContext;
- slot = node->ss.ss_ScanTupleSlot;
- scan = node->ss.ss_currentScanDesc;
+ TableScanDesc scan;
/*
* If we haven't yet performed the underlying index scan, do it, and begin
* the iteration over the bitmap. This happens on rescan as well.
*/
if (!node->initialized)
- {
BitmapHeapInitialize(node);
- /* We may have a new scan descriptor */
- scan = node->ss.ss_currentScanDesc;
- goto new_page;
- }
- for (;;)
+ /*
+ * BitmapHeapInitialize() may make the scan descriptor so don't get it
+ * from the node until after calling it.
+ */
+ econtext = node->ss.ps.ps_ExprContext;
+ slot = node->ss.ss_ScanTupleSlot;
+ scan = node->ss.ss_currentScanDesc;
+
+
+ while (table_scan_bitmap_next_tuple(scan, slot, &node->recheck,
+ &node->lossy_pages, &node->exact_pages))
{
- while (table_scan_bitmap_next_tuple(scan, slot))
- {
- CHECK_FOR_INTERRUPTS();
+ CHECK_FOR_INTERRUPTS();
- /*
- * If we are using lossy info, we have to recheck the qual
- * conditions at every tuple.
- */
- if (node->recheck)
+
+ /*
+ * If we are using lossy info, we have to recheck the qual conditions
+ * at every tuple.
+ */
+ if (node->recheck)
+ {
+ econtext->ecxt_scantuple = slot;
+ if (!ExecQualAndReset(node->bitmapqualorig, econtext))
{
- econtext->ecxt_scantuple = slot;
- if (!ExecQualAndReset(node->bitmapqualorig, econtext))
- {
- /* Fails recheck, so drop it and loop back for another */
- InstrCountFiltered2(node, 1);
- ExecClearTuple(slot);
- continue;
- }
+ /* Fails recheck, so drop it and loop back for another */
+ InstrCountFiltered2(node, 1);
+ ExecClearTuple(slot);
+ continue;
}
-
- /* OK to return this tuple */
- return slot;
}
-new_page:
-
- if (!table_scan_bitmap_next_block(scan, &node->blockno, &node->recheck,
- &node->lossy_pages, &node->exact_pages))
- break;
+ /* OK to return this tuple */
+ return slot;
}
+
/*
* if we get here it means we are at the end of the scan..
*/
@@ -334,7 +325,6 @@ ExecReScanBitmapHeapScan(BitmapHeapScanState *node)
node->tbm = NULL;
node->initialized = false;
node->recheck = true;
- node->blockno = InvalidBlockNumber;
ExecScanReScan(&node->ss);
@@ -414,7 +404,6 @@ ExecInitBitmapHeapScan(BitmapHeapScan *node, EState *estate, int eflags)
scanstate->initialized = false;
scanstate->pstate = NULL;
scanstate->recheck = true;
- scanstate->blockno = InvalidBlockNumber;
/*
* Miscellaneous initialization
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 6bb53e4346..cf56cc572f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -313,7 +313,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
info->amcanparallel = amroutine->amcanparallel;
info->amhasgettuple = (amroutine->amgettuple != NULL);
info->amhasgetbitmap = amroutine->amgetbitmap != NULL &&
- relation->rd_tableam->scan_bitmap_next_block != NULL;
+ relation->rd_tableam->scan_bitmap_next_tuple != NULL;
info->amcanmarkpos = (amroutine->ammarkpos != NULL &&
amroutine->amrestrpos != NULL);
info->amcostestimate = amroutine->amcostestimate;
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index eea7b7ff20..b2f0a61751 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -790,34 +790,21 @@ typedef struct TableAmRoutine
* ------------------------------------------------------------------------
*/
- /*
- * Prepare to fetch / check / return tuples as part of a bitmap table
- * scan. `scan` was started via table_beginscan_bm(). Return false if the
- * bitmap is exhausted and true otherwise.
- *
- * This will typically read and pin the target block, and do the necessary
- * work to allow scan_bitmap_next_tuple() to return tuples (e.g. it might
- * make sense to perform tuple visibility checks at this time).
- *
- * lossy_pages is incremented if the bitmap is lossy for the selected
- * block; otherwise, exact_pages is incremented.
- *
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
- */
- bool (*scan_bitmap_next_block) (TableScanDesc scan,
- BlockNumber *blockno, bool *recheck,
- long *lossy_pages, long *exact_pages);
/*
* Fetch the next tuple of a bitmap table scan into `slot` and return true
* if a visible tuple was found, false otherwise.
*
- * Optional callback, but either both scan_bitmap_next_block and
- * scan_bitmap_next_tuple need to exist, or neither.
+ * recheck is set if recheck is required.
+ *
+ * The table AM is responsible for reading in blocks and counting (for
+ * EXPLAIN) which of those blocks were represented lossily in the bitmap
+ * using the lossy_pages and exact_pages counters.
*/
bool (*scan_bitmap_next_tuple) (TableScanDesc scan,
- TupleTableSlot *slot);
+ TupleTableSlot *slot,
+ bool *recheck,
+ long *lossy_pages, long *exact_pages);
/*
* Prepare to fetch tuples from the next block in a sample scan. Return
@@ -1983,45 +1970,13 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths,
*/
/*
- * Prepare to fetch / check / return tuples as part of a bitmap table scan.
- * `scan` needs to have been started via table_beginscan_bm(). Returns false if
- * there are no more blocks in the bitmap, true otherwise. lossy_pages is
- * incremented is the block's representation in the bitmap is lossy; otherwise,
- * exact_pages is incremented.
- *
- * Note, this is an optionally implemented function, therefore should only be
- * used after verifying the presence (at plan time or such).
- */
-static inline bool
-table_scan_bitmap_next_block(TableScanDesc scan,
- BlockNumber *blockno, bool *recheck,
- long *lossy_pages,
- long *exact_pages)
-{
- /*
- * We don't expect direct calls to table_scan_bitmap_next_block with valid
- * CheckXidAlive for catalog or regular tables. See detailed comments in
- * xact.c where these variables are declared.
- */
- if (unlikely(TransactionIdIsValid(CheckXidAlive) && !bsysscan))
- elog(ERROR, "unexpected table_scan_bitmap_next_block call during logical decoding");
-
- return scan->rs_rd->rd_tableam->scan_bitmap_next_block(scan,
- blockno, recheck,
- lossy_pages, exact_pages);
-}
-
-/*
- * Fetch the next tuple of a bitmap table scan into `slot` and return true if
- * a visible tuple was found, false otherwise.
- * table_scan_bitmap_next_block() needs to previously have selected a
- * block (i.e. returned true), and no previous
- * table_scan_bitmap_next_tuple() for the same block may have
- * returned false.
+ * Fetch the next tuple of a bitmap table scan into `slot` and return true if a
+ * visible tuple was found, false otherwise.
*/
static inline bool
table_scan_bitmap_next_tuple(TableScanDesc scan,
- TupleTableSlot *slot)
+ TupleTableSlot *slot, bool *recheck,
+ long *lossy_pages, long *exact_pages)
{
/*
* We don't expect direct calls to table_scan_bitmap_next_tuple with valid
@@ -2032,7 +1987,9 @@ table_scan_bitmap_next_tuple(TableScanDesc scan,
elog(ERROR, "unexpected table_scan_bitmap_next_tuple call during logical decoding");
return scan->rs_rd->rd_tableam->scan_bitmap_next_tuple(scan,
- slot);
+ slot, recheck,
+ lossy_pages,
+ exact_pages);
}
/*
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 713de7d50f..49ce4ff9a7 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1797,7 +1797,6 @@ typedef struct ParallelBitmapHeapState
* initialized is node is ready to iterate
* pstate shared state for parallel bitmap scan
* recheck do current page's tuples need recheck
- * blockno used to validate pf and current block in sync
* ----------------
*/
typedef struct BitmapHeapScanState
@@ -1810,7 +1809,6 @@ typedef struct BitmapHeapScanState
bool initialized;
ParallelBitmapHeapState *pstate;
bool recheck;
- BlockNumber blockno;
} BitmapHeapScanState;
/* ----------------
--
2.40.1
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: BitmapHeapScan streaming read user and prelim refactoring
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-31 15:45 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-03 22:57 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-04 14:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-04-05 08:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-05 23:53 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
@ 2024-04-06 00:51 ` Tomas Vondra <[email protected]>
0 siblings, 0 replies; 21+ messages in thread
From: Tomas Vondra @ 2024-04-06 00:51 UTC (permalink / raw)
To: Melanie Plageman <[email protected]>; +Cc: Thomas Munro <[email protected]>; Heikki Linnakangas <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers; Nazir Bilal Yavuz <[email protected]>
On 4/6/24 01:53, Melanie Plageman wrote:
> On Fri, Apr 05, 2024 at 04:06:34AM -0400, Melanie Plageman wrote:
>> On Thu, Apr 04, 2024 at 04:35:45PM +0200, Tomas Vondra wrote:
>>>
>>>
>>> On 4/4/24 00:57, Melanie Plageman wrote:
>>>> On Sun, Mar 31, 2024 at 11:45:51AM -0400, Melanie Plageman wrote:
>>>>> On Fri, Mar 29, 2024 at 12:05:15PM +0100, Tomas Vondra wrote:
>>>>>>
>>>>>>
>>>>>> On 3/29/24 02:12, Thomas Munro wrote:
>>>>>>> On Fri, Mar 29, 2024 at 10:43 AM Tomas Vondra
>>>>>>> <[email protected]> wrote:
>>>>>>>> I think there's some sort of bug, triggering this assert in heapam
>>>>>>>>
>>>>>>>> Assert(BufferGetBlockNumber(hscan->rs_cbuf) == tbmres->blockno);
>>>>>>>
>>>>>>> Thanks for the repro. I can't seem to reproduce it (still trying) but
>>>>>>> I assume this is with Melanie's v11 patch set which had
>>>>>>> v11-0016-v10-Read-Stream-API.patch.
>>>>>>>
>>>>>>> Would you mind removing that commit and instead applying the v13
>>>>>>> stream_read.c patches[1]? v10 stream_read.c was a little confused
>>>>>>> about random I/O combining, which I fixed with a small adjustment to
>>>>>>> the conditions for the "if" statement right at the end of
>>>>>>> read_stream_look_ahead(). Sorry about that. The fixed version, with
>>>>>>> eic=4, with your test query using WHERE a < a, ends its scan with:
>>>>>>>
>>>>>>
>>>>>> I'll give that a try. Unfortunately unfortunately the v11 still has the
>>>>>> problem I reported about a week ago:
>>>>>>
>>>>>> ERROR: prefetch and main iterators are out of sync
>>>>>>
>>>>>> So I can't run the full benchmarks :-( but master vs. streaming read API
>>>>>> should work, I think.
>>>>>
>>>>> Odd, I didn't notice you reporting this ERROR popping up. Now that I
>>>>> take a look, v11 (at least, maybe also v10) had this very sill mistake:
>>>>>
>>>>> if (scan->bm_parallel == NULL &&
>>>>> scan->rs_pf_bhs_iterator &&
>>>>> hscan->pfblockno > hscan->rs_base.blockno)
>>>>> elog(ERROR, "prefetch and main iterators are out of sync");
>>>>>
>>>>> It errors out if the prefetch block is ahead of the current block --
>>>>> which is the opposite of what we want. I've fixed this in attached v12.
>>>>>
>>>>> This version also has v13 of the streaming read API. I noticed one
>>>>> mistake in my bitmapheap scan streaming read user -- it freed the
>>>>> streaming read object at the wrong time. I don't know if this was
>>>>> causing any other issues, but it at least is fixed in this version.
>>>>
>>>> Attached v13 is rebased over master (which includes the streaming read
>>>> API now). I also reset the streaming read object on rescan instead of
>>>> creating a new one each time.
>>>>
>>>> I don't know how much chance any of this has of going in to 17 now, but
>>>> I thought I would start looking into the regression repro Tomas provided
>>>> in [1].
>>>>
>>>
>>> My personal opinion is that we should try to get in as many of the the
>>> refactoring patches as possible, but I think it's probably too late for
>>> the actual switch to the streaming API.
>>
>> Cool. In the attached v15, I have dropped all commits that are related
>> to the streaming read API and included *only* commits that are
>> beneficial to master. A few of the commits are merged or reordered as
>> well.
>>
>> While going through the commits with this new goal in mind (forget about
>> the streaming read API for now), I realized that it doesn't make much
>> sense to just eliminate the layering violation for the current block and
>> leave it there for the prefetch block. I had de-prioritized solving this
>> when I thought we would just delete the prefetch code and replace it
>> with the streaming read.
>>
>> Now that we aren't doing that, I've spent the day trying to resolve the
>> issues with pushing the prefetch code into heapam.c that I cited in [1].
>> 0010 - 0013 are the result of this. They are not very polished yet and
>> need more cleanup and review (especially 0011, which is probably too
>> large), but I am happy with the solution I came up with.
>>
>> Basically, there are too many members needed for bitmap heap scan to put
>> them all in the HeapScanDescData (don't want to bloat it). So, I've made
>> a new BitmapHeapScanDescData and associated begin/rescan/end() functions
>>
>> In the end, with all patches applied, BitmapHeapNext() loops invoking
>> table_scan_bitmap_next_tuple() and table AMs can implement that however
>> they choose.
>>
>>>> I'm also not sure if I should try and group the commits into fewer
>>>> commits now or wait until I have some idea of whether or not the
>>>> approach in 0013 and 0014 is worth pursuing.
>>>>
>>>
>>> You mean whether to pursue the approach in general, or for v17? I think
>>> it looks like the right approach, but for v17 see above :-(
>>>
>>> As for merging, I wouldn't do that. I looked at the commits and while
>>> some of them seem somewhat "trivial", I really like how you organized
>>> the commits, and kept those that just "move" code around, and those that
>>> actually change stuff. It's much easier to understand, IMO.
>>>
>>> I went through the first ~10 commits, and added some review - either as
>>> a separate commit, when possible, in the code as XXX comment, and also
>>> in the commit message. The code tweaks are utterly trivial (whitespace
>>> or indentation to make the line shorter). It shouldn't take much time to
>>> deal with those, I think.
>>
>> Attached v15 incorporates your v14-0002-review.
>>
>> For your v14-0008-review, I actually ended up removing that commit
>> because once I removed everything that was for streaming read API, it
>> became redundant with another commit.
>>
>> For your v14-0010-review, we actually can't easily get rid of those
>> local variables because we make the iterators before we make the scan
>> descriptors and the commit following that commit moves the iterators
>> from the BitmapHeapScanState to the scan descriptor.
>>
>>> I think the main focus should be updating the commit messages. If it was
>>> only a single patch, I'd probably try to write the messages myself, but
>>> with this many patches it'd be great if you could update those and I'll
>>> review that before commit.
>>
>> I did my best to update the commit messages to be less specific and more
>> focused on "why should I care". I found myself wanting to explain why I
>> implemented something the way I did and then getting back into the
>> implementation details again. I'm not sure if I suceeded in having less
>> details and more substance.
>>
>>> I always struggle with writing commit messages myself, and it takes me
>>> ages to write a good one (well, I think the message is good, but who
>>> knows ...). But I think a good message should be concise enough to
>>> explain what and why it's done. It may reference a thread for all the
>>> gory details, but the basic reasoning should be in the commit message.
>>> For example the message for "BitmapPrefetch use prefetch block recheck
>>> for skip fetch" now says that it "makes more sense to do X" but does not
>>> really say why that's the case. The linked message does, but it'd be
>>> good to have that in the message (because how would I know how much of
>>> the thread to read?).
>>
>> I fixed that particular one. I tried to take that feedback and apply it
>> to other commit messages. I don't know how successful I was...
>>
>>> Also, it'd be very helpful if you could update the author & reviewed-by
>>> fields. I'll review those before commit, ofc, but I admit I lost track
>>> of who reviewed which part.
>>
>> I have updated reviewers. I didn't add reviewers on the ones that
>> haven't really been reviewed yet.
>>
>>> I'd focus on the first ~8-9 commits or so for now, we can commit more if
>>> things go reasonably well.
>>
>> Sounds good. I will spend cleanup time on 0010-0013 tomorrow but would
>> love to know if you agree with the direction before I spend more time.
>
> In attached v16, I've split out 0010-0013 into 0011-0017. I think it is
> much easier to understand.
>
Damn it, I went through the whole patch series, adding a couple review
comments and tweaks, and was just about to share my version, but you bet
me to it ;-)
Anyway, I've attached it as .tgz in order to not confuse cfbot. All the
review comments are marked with XXX, so grep for that in the patches.
There's two separate patches - the first one suggests a code change, so
it was better to not merge that with your code. The second has just a
couple XXX comments, I'm not sure why I kept it separate.
A couple review comments:
* I think 0001-0009 are 99% ready to. I reworded some of the commit
messages a bit - I realize it's a bit bold, considering you're native
speaker and I'm not. If you could check I didn't make it worse, that
would be great.
* I'm not sure extra_flags is the right way to pass the flag in 0003.
The "extra_" name is a bit weird, and no other table AM functions do it
this way and pass explicit bool flags instead. So my first "review"
commit does it like that. Do you agree it's better that way?
* The one question I'm somewhat unsure about is why Tom chose to use the
"wrong" recheck flag in the 2017 commit, when the correct recheck flag
is readily available. Surely that had a reason, right? But I can't think
of one ...
> While I was doing that, I realized that I should remove the call to
> table_rescan() from ExecReScanBitmapHeapScan() and just rely on the new
> table_rescan_bm() invoked from BitmapHeapNext(). That is done in the
> attached.
>
> 0010-0018 still need comments updated but I focused on getting the split
> out, reviewable version of them ready. I'll add comments (especially to
> 0011 table AM functions) tomorrow. I also have to double-check if I
> should add any asserts for table AMs about having implemented all of the
> new begin/re/endscan() functions.
>
I added a couple more comments for those patches (10-12). Chances are
the split in v16 clarifies some of my questions, but it'll have to wait
till the morning ...
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
[application/x-compressed-tar] v15-review.tgz (32.5K, ../../[email protected]/2-v15-review.tgz)
download
^ permalink raw reply [nested|flat] 21+ messages in thread
end of thread, other threads:[~2024-04-06 00:51 UTC | newest]
Thread overview: 21+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-12 13:07 [PATCH 1/9] Pass all scan keys to BRIN consistent function at once Tomas Vondra <[email protected]>
2024-03-25 16:07 Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-27 19:37 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-03-28 05:20 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-28 18:01 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-28 21:19 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-28 21:43 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 01:12 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 11:05 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 11:17 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 13:36 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 15:52 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 21:39 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-29 23:40 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-03-29 23:56 ` Re: BitmapHeapScan streaming read user and prelim refactoring Thomas Munro <[email protected]>
2024-03-31 15:45 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-03 22:57 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-04 14:35 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
2024-04-05 08:06 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-05 23:53 ` Re: BitmapHeapScan streaming read user and prelim refactoring Melanie Plageman <[email protected]>
2024-04-06 00:51 ` Re: BitmapHeapScan streaming read user and prelim refactoring Tomas Vondra <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox