public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 8/8] Ignore correlation for new BRIN opclasses
3+ messages / 3 participants
[nested] [flat]
* [PATCH 8/8] Ignore correlation for new BRIN opclasses
@ 2020-09-12 13:07 Tomas Vondra <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: Tomas Vondra @ 2020-09-12 13:07 UTC (permalink / raw)
The new BRIN opclasses (bloom and minmax-multi) are less sensitive to
poorly correlated data, so just assume the data is perfectly correlated
during costing.
Author: Tomas Vondra <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/backend/access/brin/brin_bloom.c | 1 +
src/backend/access/brin/brin_minmax_multi.c | 1 +
src/backend/utils/adt/selfuncs.c | 19 ++++++++++++++++++-
src/include/access/brin_internal.h | 3 +++
4 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c
index c086e83236..22590b2351 100644
--- a/src/backend/access/brin/brin_bloom.c
+++ b/src/backend/access/brin/brin_bloom.c
@@ -412,6 +412,7 @@ brin_bloom_opcinfo(PG_FUNCTION_ARGS)
result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) +
sizeof(BloomOpaque));
+ result->oi_ignore_correlation = true;
result->oi_nstored = 1;
result->oi_regular_nulls = true;
result->oi_opaque = (BloomOpaque *)
diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c
index d2dc38adc8..6b1dd1040c 100644
--- a/src/backend/access/brin/brin_minmax_multi.c
+++ b/src/backend/access/brin/brin_minmax_multi.c
@@ -1737,6 +1737,7 @@ brin_minmax_multi_opcinfo(PG_FUNCTION_ARGS)
result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) +
sizeof(MinmaxMultiOpaque));
+ result->oi_ignore_correlation = true;
result->oi_nstored = 1;
result->oi_regular_nulls = true;
result->oi_opaque = (MinmaxMultiOpaque *)
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 52314d3aa1..0320d128f6 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -98,6 +98,7 @@
#include <math.h>
#include "access/brin.h"
+#include "access/brin_internal.h"
#include "access/brin_page.h"
#include "access/gin.h"
#include "access/table.h"
@@ -7352,7 +7353,8 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
double minimalRanges;
double estimatedRanges;
double selec;
- Relation indexRel;
+ Relation indexRel = NULL;
+ TupleDesc tupdesc = NULL;
ListCell *l;
VariableStatData vardata;
@@ -7374,6 +7376,7 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
*/
indexRel = index_open(index->indexoid, NoLock);
brinGetStats(indexRel, &statsData);
+ tupdesc = RelationGetDescr(indexRel);
index_close(indexRel, NoLock);
/* work out the actual number of ranges in the index */
@@ -7407,6 +7410,17 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
{
IndexClause *iclause = lfirst_node(IndexClause, l);
AttrNumber attnum = index->indexkeys[iclause->indexcol];
+ FmgrInfo *opcInfoFn;
+ BrinOpcInfo *opcInfo;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, iclause->indexcol);
+ bool ignore_correlation;
+
+ opcInfoFn = index_getprocinfo(indexRel, iclause->indexcol + 1, BRIN_PROCNUM_OPCINFO);
+
+ opcInfo = (BrinOpcInfo *)
+ DatumGetPointer(FunctionCall1(opcInfoFn, attr->atttypid));
+
+ ignore_correlation = opcInfo->oi_ignore_correlation;
/* attempt to lookup stats in relation for this index column */
if (attnum != 0)
@@ -7477,6 +7491,9 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count,
if (sslot.nnumbers > 0)
varCorrelation = Abs(sslot.numbers[0]);
+ if (ignore_correlation)
+ varCorrelation = 1.0;
+
if (varCorrelation > *indexCorrelation)
*indexCorrelation = varCorrelation;
diff --git a/src/include/access/brin_internal.h b/src/include/access/brin_internal.h
index 89254b5766..063b703208 100644
--- a/src/include/access/brin_internal.h
+++ b/src/include/access/brin_internal.h
@@ -30,6 +30,9 @@ typedef struct BrinOpcInfo
/* Regular processing of NULLs in BrinValues? */
bool oi_regular_nulls;
+ /* Ignore correlation during cost estimation */
+ bool oi_ignore_correlation;
+
/* Opaque pointer for the opclass' private use */
void *oi_opaque;
--
2.26.2
--------------556A1DC61262AF15641DB204--
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: PATCH: Using BRIN indexes for sorted output
@ 2023-02-24 10:25 Tomas Vondra <[email protected]>
2023-02-24 15:14 ` Re: PATCH: Using BRIN indexes for sorted output Alvaro Herrera <[email protected]>
0 siblings, 1 reply; 3+ messages in thread
From: Tomas Vondra @ 2023-02-24 10:25 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; Matthias van de Meent <[email protected]>; +Cc: Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Greg Stark <[email protected]>; Zhihong Yu <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2/24/23 09:39, Alvaro Herrera wrote:
> On 2023-Feb-23, Matthias van de Meent wrote:
>
>>> + for (heapBlk = 0; heapBlk < nblocks; heapBlk += pagesPerRange)
>>
>> I am not familiar with the frequency of max-sized relations, but this
>> would go into an infinite loop for pagesPerRange values >1 for
>> max-sized relations due to BlockNumber wraparound. I think there
>> should be some additional overflow checks here.
>
> They are definitely not very common -- BlockNumber wraps around at 32 TB
> IIUC. But yeah, I guess it is a possibility, and perhaps we should find
> a way to write these loops in a more robust manner.
>
I guess the easiest fix would be to do the arithmetic in 64 bits. That'd
eliminate the overflow.
Alternatively, we could do something like
prevHeapBlk = 0;
for (heapBlk = 0; (heapBlk < nblocks) && (prevHeapBlk <= heapBlk);
heapBlk += pagesPerRange)
{
...
prevHeapBlk = heapBlk;
}
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: PATCH: Using BRIN indexes for sorted output
2023-02-24 10:25 Re: PATCH: Using BRIN indexes for sorted output Tomas Vondra <[email protected]>
@ 2023-02-24 15:14 ` Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: Alvaro Herrera @ 2023-02-24 15:14 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Matthias van de Meent <[email protected]>; Justin Pryzby <[email protected]>; Andres Freund <[email protected]>; Greg Stark <[email protected]>; Zhihong Yu <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2023-Feb-24, Tomas Vondra wrote:
> I guess the easiest fix would be to do the arithmetic in 64 bits. That'd
> eliminate the overflow.
Yeah, that might be easy to set up. We then don't have to worry about
it until BlockNumber is enlarged to 64 bits ... but by that time surely
we can just grow it again to a 128 bit loop variable.
> Alternatively, we could do something like
>
> prevHeapBlk = 0;
> for (heapBlk = 0; (heapBlk < nblocks) && (prevHeapBlk <= heapBlk);
> heapBlk += pagesPerRange)
> {
> ...
> prevHeapBlk = heapBlk;
> }
I think a formulation of this kind has the benefit that it works after
BlockNumber is enlarged to 64 bits, and doesn't have to be changed ever
again (assuming it is correct).
... if pagesPerRange is not a whole divisor of MaxBlockNumber, I think
this will neglect the last range in the table.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2023-02-24 15:14 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-09-12 13:07 [PATCH 8/8] Ignore correlation for new BRIN opclasses Tomas Vondra <[email protected]>
2023-02-24 10:25 Re: PATCH: Using BRIN indexes for sorted output Tomas Vondra <[email protected]>
2023-02-24 15:14 ` Re: PATCH: Using BRIN indexes for sorted output Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox