public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 8/8] Ignore correlation for new BRIN opclasses 3+ messages / 3 participants [nested] [flat]
* [PATCH 8/8] Ignore correlation for new BRIN opclasses @ 2020-09-12 13:07 Tomas Vondra <[email protected]> 0 siblings, 0 replies; 3+ messages in thread From: Tomas Vondra @ 2020-09-12 13:07 UTC (permalink / raw) The new BRIN opclasses (bloom and minmax-multi) are less sensitive to poorly correlated data, so just assume the data is perfectly correlated during costing. Author: Tomas Vondra <[email protected]> Discussion: https://postgr.es/m/[email protected] --- src/backend/access/brin/brin_bloom.c | 1 + src/backend/access/brin/brin_minmax_multi.c | 1 + src/backend/utils/adt/selfuncs.c | 19 ++++++++++++++++++- src/include/access/brin_internal.h | 3 +++ 4 files changed, 23 insertions(+), 1 deletion(-) diff --git a/src/backend/access/brin/brin_bloom.c b/src/backend/access/brin/brin_bloom.c index c086e83236..22590b2351 100644 --- a/src/backend/access/brin/brin_bloom.c +++ b/src/backend/access/brin/brin_bloom.c @@ -412,6 +412,7 @@ brin_bloom_opcinfo(PG_FUNCTION_ARGS) result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) + sizeof(BloomOpaque)); + result->oi_ignore_correlation = true; result->oi_nstored = 1; result->oi_regular_nulls = true; result->oi_opaque = (BloomOpaque *) diff --git a/src/backend/access/brin/brin_minmax_multi.c b/src/backend/access/brin/brin_minmax_multi.c index d2dc38adc8..6b1dd1040c 100644 --- a/src/backend/access/brin/brin_minmax_multi.c +++ b/src/backend/access/brin/brin_minmax_multi.c @@ -1737,6 +1737,7 @@ brin_minmax_multi_opcinfo(PG_FUNCTION_ARGS) result = palloc0(MAXALIGN(SizeofBrinOpcInfo(1)) + sizeof(MinmaxMultiOpaque)); + result->oi_ignore_correlation = true; result->oi_nstored = 1; result->oi_regular_nulls = true; result->oi_opaque = (MinmaxMultiOpaque *) diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c index 52314d3aa1..0320d128f6 100644 --- a/src/backend/utils/adt/selfuncs.c +++ b/src/backend/utils/adt/selfuncs.c @@ -98,6 +98,7 @@ #include <math.h> #include "access/brin.h" +#include "access/brin_internal.h" #include "access/brin_page.h" #include "access/gin.h" #include "access/table.h" @@ -7352,7 +7353,8 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count, double minimalRanges; double estimatedRanges; double selec; - Relation indexRel; + Relation indexRel = NULL; + TupleDesc tupdesc = NULL; ListCell *l; VariableStatData vardata; @@ -7374,6 +7376,7 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count, */ indexRel = index_open(index->indexoid, NoLock); brinGetStats(indexRel, &statsData); + tupdesc = RelationGetDescr(indexRel); index_close(indexRel, NoLock); /* work out the actual number of ranges in the index */ @@ -7407,6 +7410,17 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count, { IndexClause *iclause = lfirst_node(IndexClause, l); AttrNumber attnum = index->indexkeys[iclause->indexcol]; + FmgrInfo *opcInfoFn; + BrinOpcInfo *opcInfo; + Form_pg_attribute attr = TupleDescAttr(tupdesc, iclause->indexcol); + bool ignore_correlation; + + opcInfoFn = index_getprocinfo(indexRel, iclause->indexcol + 1, BRIN_PROCNUM_OPCINFO); + + opcInfo = (BrinOpcInfo *) + DatumGetPointer(FunctionCall1(opcInfoFn, attr->atttypid)); + + ignore_correlation = opcInfo->oi_ignore_correlation; /* attempt to lookup stats in relation for this index column */ if (attnum != 0) @@ -7477,6 +7491,9 @@ brincostestimate(PlannerInfo *root, IndexPath *path, double loop_count, if (sslot.nnumbers > 0) varCorrelation = Abs(sslot.numbers[0]); + if (ignore_correlation) + varCorrelation = 1.0; + if (varCorrelation > *indexCorrelation) *indexCorrelation = varCorrelation; diff --git a/src/include/access/brin_internal.h b/src/include/access/brin_internal.h index 89254b5766..063b703208 100644 --- a/src/include/access/brin_internal.h +++ b/src/include/access/brin_internal.h @@ -30,6 +30,9 @@ typedef struct BrinOpcInfo /* Regular processing of NULLs in BrinValues? */ bool oi_regular_nulls; + /* Ignore correlation during cost estimation */ + bool oi_ignore_correlation; + /* Opaque pointer for the opclass' private use */ void *oi_opaque; -- 2.26.2 --------------82D677E9F94AC7574E460205-- ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: buildfarm + meson @ 2023-02-23 01:20 Andres Freund <[email protected]> 0 siblings, 1 reply; 3+ messages in thread From: Andres Freund @ 2023-02-23 01:20 UTC (permalink / raw) To: Andrew Dunstan <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> Hi, On 2023-02-22 18:23:44 -0500, Andrew Dunstan wrote: > Here's a progress report on adapting the buildfarm client to meson > > There is a development branch where I'm working on the changes. They can be > seen here: > > <https://github.com/PGBuildFarm/client-code/compare/main...dev/meson; > > On my Linux box (Fedora 37, where crake runs) I can get a complete run. Nice! > There is work to do to make sure we pick up the right log files, and maybe > adjust a module or two. I have adopted a design where instead of trying to > know a lot about the testing regime the client needs to know a lot less. > Instead, it gets meson to tell it the set of tests. I will probably work on > enabling some sort of filter, but I think this makes things more > future-proof. I have stuck with the design of making testing fairly > fine-grained, so each suite runs separately. I don't understand why you'd want to run each suite separately. Serially executing the test takes way longer than doing so in parallel. Why would we want to enforce that? Particularly because with meson the tests log files and the failed tests can directly be correlated? And it should be easy to figure out which log files need to be kept, you can just skip the directories in testrun/ that contain test.success. > On a Windows instance, fairly similar to what's running drongo, I can get a > successful build with meson+VS2019, but I'm getting an error in the > regression tests, which don't like setting lc_time to 'de_DE'. Not sure > what's going on there. Huh, that's odd. > meson apparently wants touch and cp installed, although I can't see why at > first glance. For Windows I just copied them into the path from an msys2 > installation. Those should probably be fixed. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: buildfarm + meson @ 2023-02-23 11:27 Andrew Dunstan <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 0 replies; 3+ messages in thread From: Andrew Dunstan @ 2023-02-23 11:27 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]> On 2023-02-22 We 20:20, Andres Freund wrote: > >> There is work to do to make sure we pick up the right log files, and maybe >> adjust a module or two. I have adopted a design where instead of trying to >> know a lot about the testing regime the client needs to know a lot less. >> Instead, it gets meson to tell it the set of tests. I will probably work on >> enabling some sort of filter, but I think this makes things more >> future-proof. I have stuck with the design of making testing fairly >> fine-grained, so each suite runs separately. > I don't understand why you'd want to run each suite separately. Serially > executing the test takes way longer than doing so in parallel. Why would we > want to enforce that? > > Particularly because with meson the tests log files and the failed tests can > directly be correlated? And it should be easy to figure out which log files > need to be kept, you can just skip the directories in testrun/ that contain > test.success. > We can revisit that later. For now I'm more concerned with getting a working setup. The requirements of the buildfarm are a bit different from those of a developer, though. Running things in parallel can make things faster, but that can also increase the compute load. Also, running things serially makes it easier to report a failure stage that pinpoints the test that encountered an issue. But like I say we can come back to this. >> On a Windows instance, fairly similar to what's running drongo, I can get a >> successful build with meson+VS2019, but I'm getting an error in the >> regression tests, which don't like setting lc_time to 'de_DE'. Not sure >> what's going on there. > Huh, that's odd. See my reply to Michael for details > > >> meson apparently wants touch and cp installed, although I can't see why at >> first glance. For Windows I just copied them into the path from an msys2 >> installation. > Those should probably be fixed. > Yeah. For touch I think we can probably just get rid of this line in the root meson.build: touch = find_program('touch', native: true) For cp there doesn't seem to be a formal requirement, but there is a recipe in src/common/unicode/meson.build that uses it, maybe that's what caused the failure. On Windows/msvc we could just use copy instead, I think. I haven't experimented with any of this. cheers andrew -- Andrew Dunstan EDB:https://www.enterprisedb.com ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2023-02-23 11:27 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-09-12 13:07 [PATCH 8/8] Ignore correlation for new BRIN opclasses Tomas Vondra <[email protected]> 2023-02-23 01:20 Re: buildfarm + meson Andres Freund <[email protected]> 2023-02-23 11:27 ` Re: buildfarm + meson Andrew Dunstan <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox