agora inbox for [email protected]
help / color / mirror / Atom feedWIP: Aggregation push-down
59+ messages / 20 participants
[nested] [flat]
* WIP: Aggregation push-down
@ 2017-04-04 08:41 Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2017-04-04 08:41 UTC (permalink / raw)
To: pgsql-hackers
This is a new version of the patch I presented in [1]. A new thread seems
appropriate because the current version can aggregate both base relations and
joins, so the original subject would no longer match.
There's still work to do but I'd consider the patch complete in terms of
concept. A few things worth attention for those who want to look into the
code:
* I've abandoned the concept of aggmultifn proposed in [1], as it doesn't
appear to be very useful. That implies that a "grouped join" can be formed
in 2 ways: 1) join a grouped relation to a "plain" (i.e. non-grouped) one,
2) join 2 plain relations and aggregate the result. However, w/o the
aggmultifn we can't join 2 grouped relations.
* GroupedVar type is used to propagate the result of partial aggregation from
to the top-level join. It's conceptually very similar to PlaceHolderVar.
* Although I intended to use the "unique join" feature [2], I postponed it so
far. The point is that [2] does conflict with my patch and thus I'd have to
rebase the patch more often. Anyway, the impact of [2] on aggregation
finalization (i.e. possible avoidance of the "finalize aggregate node"
setup) is not really specific to my patch.
* Scan of base relation or join result can be partially aggregated for 2
reasons: 1) it makes the whole plan cheaper because the aggregation takes
place on remote node and thus the amount of data to be transferred via
network is significanlty reduced, 2) aggregate functions are rather
expensive so it makes sense to evaluate them by multiple parallel workers.
The patch contains both of these features as they are hard to
separate from each other.
While 1) needs additional work on postgres_fdw, scripts to simulate 2) are
attached. Planner settings are such that cost of expression evaluation is
significant, so that it's worth to engage multiple parallel workers.
In my environment it yields the following output:
Parallel Finalize HashAggregate
Group Key: a.i
-> Gather Merge
Workers Planned: 4
-> Merge Join
Merge Cond: (b.parent = a.i)
-> Sort
Sort Key: b.parent
-> Parallel Partial HashAggregate
Group Key: b.parent
-> Hash Join
Hash Cond: ((b.parent = c.parent) AND (b.j = c.k))
-> Parallel Seq Scan on b
-> Hash
-> Seq Scan on c
-> Sort
Sort Key: a.i
-> Seq Scan on a
Feedback is appreciated.
[1] https://www.postgresql.org/message-id/29111.1483984605%40localhost
[2] https://commitfest.postgresql.org/13/859/
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
BEGIN;
CREATE TABLE a (
i int primary key);
CREATE TABLE b (
j int primary key,
parent int references a,
v double precision);
CREATE TABLE c (
k int primary key,
parent int references a,
v double precision);
INSERT INTO a(i)
SELECT n
FROM generate_series(0, 255) AS s(n);
INSERT INTO b(j, parent, v)
SELECT 1024 * i + n, i, random()
FROM generate_series(0, 1023) AS s(n), a;
INSERT INTO c(k, parent, v)
SELECT 1024 * i + n, i, random()
FROM generate_series(0, 1023) AS s(n), a;
CREATE OR REPLACE FUNCTION expensive(x double precision)
RETURNS double precision
LANGUAGE sql
AS $$
-- CTE is there to avoid inlining.
WITH tmp AS (SELECT x) SELECT x FROM tmp;
$$
PARALLEL SAFE
COST 10000;
COMMIT;
ANALYZE;
SET seq_page_cost TO 0;
SET cpu_operator_cost TO 1;
SET min_parallel_table_scan_size TO 0;
SET max_parallel_workers_per_gather TO 4;
EXPLAIN (COSTS false)
SELECT a.i, avg(expensive(b.v + c.v))
FROM a
JOIN
b ON b.parent = a.i
JOIN
c ON c.parent = a.i
WHERE b.j = c.k
GROUP BY a.i;
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[text/x-diff] agg_pushdown.diff (208.6K, ../../9666.1491295317@localhost/2-agg_pushdown.diff)
download | inline diff:
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index 5a84742..d7afcea
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, PlanState *p
*** 717,722 ****
--- 717,749 ----
break;
}
+ case T_GroupedVar:
+ /*
+ * GroupedVar is treated as an aggregate if it appears in the
+ * targetlist of Agg node, but as a normal variable elsewhere.
+ */
+ if (parent && (IsA(parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * Currently GroupedVar can only represent partial aggregate.
+ */
+ Assert(gvar->agg_partial != NULL);
+
+ ExecInitExprRec((Expr *) gvar->agg_partial, parent, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ */
+ elog(ERROR, "parent of GroupedVar is not Agg node");
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
new file mode 100644
index ef35da6..70d2367
*** a/src/backend/executor/nodeAgg.c
--- b/src/backend/executor/nodeAgg.c
*************** find_unaggregated_cols_walker(Node *node
*** 1826,1831 ****
--- 1826,1842 ----
/* do not descend into aggregate exprs */
return false;
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * GroupedVar is currently used only for partial aggregation, so treat
+ * it like an Aggref above.
+ */
+ Assert(gvar->agg_partial != NULL);
+ return false;
+ }
return expression_tree_walker(node, find_unaggregated_cols_walker,
(void *) colnos);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 61bc502..5f5bb4f
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2189,2194 ****
--- 2189,2209 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** copyObjectImpl(const void *from)
*** 4958,4963 ****
--- 4973,4981 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index 5941b7a..4fe3aa8
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 865,870 ****
--- 865,878 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3130,3135 ****
--- 3138,3146 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index d5293a1..da517ce
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 256,261 ****
--- 256,264 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprCollation(const Node *expr)
*** 925,930 ****
--- 928,936 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2188,2193 ****
--- 2194,2201 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2978,2983 ****
--- 2986,3001 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 766ca49..81da091
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2181,2186 ****
--- 2181,2187 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outParamPathInfo(StringInfo str, const
*** 2401,2406 ****
--- 2402,2417 ----
}
static void
+ _outGroupedPathInfo(StringInfo str, const GroupedPathInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDPATHINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(pathlist);
+ WRITE_NODE_FIELD(partial_pathlist);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2444,2449 ****
--- 2455,2470 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** outNode(StringInfo str, const void *obj)
*** 3980,3991 ****
--- 4001,4018 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_GroupedPathInfo:
+ _outGroupedPathInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index 766f2d8..fdf4edd
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 521,526 ****
--- 521,541 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** parseNodeString(void)
*** 2436,2441 ****
--- 2451,2458 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index b5cab0c..f89406d
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 265,271 ****
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
--- 265,271 ----
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 343b35a..fca4727
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 486,492 ****
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! generate_gather_paths(root, rel);
/*
* Allow a plugin to editorialize on the set of Paths for this base
--- 486,495 ----
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! {
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
! }
/*
* Allow a plugin to editorialize on the set of Paths for this base
*************** static void
*** 687,692 ****
--- 690,696 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 695,709 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel);
/* Consider TID scans */
create_tidscan_paths(root, rel);
--- 699,726 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! add_path(rel, seq_path, false);
! if (rel->gpi != NULL && required_outer == NULL)
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel, false);
! if (rel->gpi != NULL)
! {
! /*
! * TODO Instead of calling the whole clause-matching machinery twice
! * (there should be no difference between plain and grouped paths from
! * this point of view), consider returning a separate list of paths
! * usable as grouped ones.
! */
! create_index_paths(root, rel, true);
! }
/* Consider TID scans */
create_tidscan_paths(root, rel);
*************** static void
*** 717,722 ****
--- 734,740 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
parallel_workers = compute_parallel_worker(rel, rel->pages, -1);
*************** create_plain_partial_paths(PlannerInfo *
*** 725,731 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 743,850 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
! add_partial_path(rel, path, false);
!
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it.
! */
! if (rel->gpi != NULL)
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the
! * appropriate pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck().
! *
! * If "partial" is true, the resulting path is considered partial in terms of
! * parallel execution.
! *
! * The path we create here shouldn't be parameterized because of supposedly
! * high startup cost of aggregation (whether due to build of hash table for
! * AGG_HASHED strategy or due to explicit sort for AGG_SORTED).
! *
! * XXX IndexPath as an input for AGG_SORTED might seem to be an exception, but
! * aggregation of its output is only beneficial if it's performed by multiple
! * workers, i.e. the resulting path is partial (Besides parallel aggregation,
! * the other use case of aggregation push-down is aggregation performed on
! * remote database, but that has nothing to do with IndexScan). And partial
! * path cannot be parameterized because it's semantically wrong to use it on
! * the inner side of NL join.
! */
! void
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! List *group_clauses = NIL;
! List *group_exprs = NIL;
! List *agg_exprs = NIL;
! Path *agg_path;
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Grouped path should never be parameterized, so we're not supposed to
! * receive parameterized subpath.
! */
! Assert(subpath->param_info == NULL);
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL,
! true))
! return;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost, pathkeys,
! true))
! return;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path, true);
! else
! add_partial_path(rel, (Path *) agg_path, true);
! }
}
/*
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 811,817 ****
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path);
/* For the moment, at least, there are no other paths to consider */
}
--- 930,936 ----
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path, false);
/* For the moment, at least, there are no other paths to consider */
}
*************** set_append_rel_size(PlannerInfo *root, R
*** 1066,1071 ****
--- 1185,1233 ----
appinfo);
/*
+ * If grouping is applicable to the parent relation, it should be
+ * applicable to the children too. Make sure the child rel has valid
+ * sortgrouprefs.
+ *
+ * TODO Consider if this is really needed --- child rel is not joined
+ * to grouped rel itself, so it might not participate on creation of
+ * the grouped path target that upper joins will see.
+ */
+ if (rel->reltarget->sortgrouprefs)
+ {
+ Assert(childrel->reltarget->sortgrouprefs == NULL);
+ childrel->reltarget->sortgrouprefs = rel->reltarget->sortgrouprefs;
+ }
+
+ /*
+ * Also the grouped target needs to be adjusted, if one exists.
+ */
+ if (rel->gpi != NULL)
+ {
+ PathTarget *target = rel->gpi->target;
+
+ Assert(target->sortgrouprefs != NULL);
+
+ Assert(childrel->gpi == NULL);
+ childrel->gpi = makeNode(GroupedPathInfo);
+ memcpy(childrel->gpi, rel->gpi, sizeof(GroupedPathInfo));
+
+ /*
+ * add_grouping_info_to_base_rels was not sure if grouping makes
+ * sense for the parent rel, so create a separate copy of the
+ * target now.
+ */
+ childrel->gpi->target = copy_pathtarget(childrel->gpi->target);
+
+ /* Translate vars of the grouping target. */
+ Assert(childrel->gpi->target->exprs != NIL);
+ childrel->gpi->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) childrel->gpi->target->exprs,
+ appinfo);
+ }
+
+ /*
* We have to make child entries in the EquivalenceClass data
* structures as well. This is needed either if the parent
* participates in some eclass joins (because we will want to consider
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1280,1285 ****
--- 1442,1449 ----
bool subpaths_valid = true;
List *partial_subpaths = NIL;
bool partial_subpaths_valid = true;
+ List *grouped_subpaths = NIL;
+ bool grouped_subpaths_valid = true;
List *all_child_pathkeys = NIL;
List *all_child_outers = NIL;
ListCell *l;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1323,1328 ****
--- 1487,1515 ----
partial_subpaths_valid = false;
/*
+ * For grouped paths, use only the unparameterized subpaths.
+ *
+ * XXX Consider if the parameterized subpaths should be processed
+ * below. It's probably not useful for sequential scans (due to
+ * repeated aggregation), but might be worthwhile for other child
+ * nodes.
+ */
+ if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
+ {
+ Path *path;
+
+ path = (Path *) linitial(childrel->gpi->pathlist);
+ if (path->param_info == NULL)
+ grouped_subpaths = accumulate_append_subpath(grouped_subpaths,
+ path);
+ else
+ grouped_subpaths_valid = false;
+ }
+ else
+ grouped_subpaths_valid = false;
+
+
+ /*
* Collect lists of all the available path orderings and
* parameterizations for all the children. We use these as a
* heuristic to indicate which sort orderings and parameterizations we
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1394,1400 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels));
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1581,1588 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels),
! false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1421,1428 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1609,1629 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers,
! partitioned_rels);
! add_partial_path(rel, (Path *) appendpath, false);
! }
!
! /* TODO Also partial grouped paths? */
! if (grouped_subpaths_valid)
! {
! Path *path;
!
! path = (Path *) create_append_path(rel, grouped_subpaths, NULL, 0,
! partitioned_rels);
! /* pathtarget will produce the grouped relation.. */
! path->pathtarget = rel->gpi->target;
! add_path(rel, path, true);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1475,1481 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels));
}
}
--- 1676,1683 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels),
! false);
}
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1571,1584 ****
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels));
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels));
}
}
--- 1773,1788 ----
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
}
}
*************** set_dummy_rel_pathlist(RelOptInfo *rel)
*** 1711,1717 ****
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
--- 1915,1921 ----
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
*************** set_subquery_pathlist(PlannerInfo *root,
*** 1925,1931 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer));
}
}
--- 2129,2135 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer), false);
}
}
*************** set_function_pathlist(PlannerInfo *root,
*** 1994,2000 ****
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer));
}
/*
--- 2198,2204 ----
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer), false);
}
/*
*************** set_values_pathlist(PlannerInfo *root, R
*** 2014,2020 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer));
}
/*
--- 2218,2224 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer), false);
}
/*
*************** set_tablefunc_pathlist(PlannerInfo *root
*** 2035,2041 ****
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer));
}
/*
--- 2239,2245 ----
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer), false);
}
/*
*************** set_cte_pathlist(PlannerInfo *root, RelO
*** 2101,2107 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer));
}
/*
--- 2305,2311 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer), false);
}
/*
*************** set_namedtuplestore_pathlist(PlannerInfo
*** 2128,2134 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
--- 2332,2339 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
*************** set_worktable_pathlist(PlannerInfo *root
*** 2181,2187 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer));
}
/*
--- 2386,2393 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer),
! false);
}
/*
*************** set_worktable_pathlist(PlannerInfo *root
*** 2194,2207 ****
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
ListCell *lc;
/* If there are no partial paths, there's nothing to do here. */
! if (rel->partial_pathlist == NIL)
return;
/*
--- 2400,2420 ----
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
+ List *pathlist = NIL;
+ PathTarget *partial_target;
ListCell *lc;
+ if (!grouped)
+ pathlist = rel->partial_pathlist;
+ else if (rel->gpi != NULL)
+ pathlist = rel->gpi->partial_pathlist;
+
/* If there are no partial paths, there's nothing to do here. */
! if (pathlist == NIL)
return;
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2209,2225 ****
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(rel->partial_pathlist);
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, rel->partial_pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
--- 2422,2444 ----
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(pathlist);
!
! if (!grouped)
! partial_target = rel->reltarget;
! else if (rel->gpi != NULL)
! partial_target = rel->gpi->target;
!
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, partial_target,
NULL, NULL);
! add_path(rel, simple_gather_path, grouped);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
*************** generate_gather_paths(PlannerInfo *root,
*** 2227,2235 ****
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path);
}
}
--- 2446,2454 ----
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, partial_target,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, grouped);
}
}
*************** standard_join_search(PlannerInfo *root,
*** 2395,2401 ****
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
--- 2614,2621 ----
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
*************** create_partial_bitmap_paths(PlannerInfo
*** 3046,3052 ****
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3266,3272 ----
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers), false);
}
/*
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index a5d19f9..89f4308
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** static bool eclass_already_used(Equivale
*** 107,119 ****
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
--- 108,121 ----
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! bool grouped);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
*************** static Const *string_to_const(const char
*** 229,235 ****
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel)
{
List *indexpaths;
List *bitindexpaths;
--- 231,237 ----
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
List *indexpaths;
List *bitindexpaths;
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,283 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
! grouped);
/*
* Identify the join clauses that can match the index. For the moment
*************** create_index_paths(PlannerInfo *root, Re
*** 338,344 ****
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
--- 340,346 ----
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath, false);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
*************** create_index_paths(PlannerInfo *root, Re
*** 415,421 ****
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath);
}
}
}
--- 417,423 ----
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath, false);
}
}
}
*************** get_join_index_paths(PlannerInfo *root,
*** 667,673 ****
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
--- 669,675 ----
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths, false);
/*
* Remember we considered paths for this set of relids. We use lcons not
*************** bms_equal_any(Relids relids, List *relid
*** 736,742 ****
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths)
{
List *indexpaths;
bool skip_nonnative_saop = false;
--- 738,744 ----
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped)
{
List *indexpaths;
bool skip_nonnative_saop = false;
*************** get_index_paths(PlannerInfo *root, RelOp
*** 754,760 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
--- 756,762 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop, grouped);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
*************** get_index_paths(PlannerInfo *root, RelOp
*** 769,775 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL));
}
/*
--- 771,777 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL, grouped));
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,797 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath);
! if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
--- 791,799 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, grouped);
! if (!grouped && index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
*************** get_index_paths(PlannerInfo *root, RelOp
*** 802,815 ****
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
--- 804,818 ----
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (!grouped && skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 861,867 ****
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop)
{
List *result = NIL;
IndexPath *ipath;
--- 864,870 ----
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop, bool grouped)
{
List *result = NIL;
IndexPath *ipath;
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 881,890 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool can_agg_sorted;
+ List *group_clauses, *group_exprs, *agg_exprs;
+ AggPath *agg_path;
+ double agg_input_rows;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 891,896 ****
--- 898,906 ----
case ST_BITMAPSCAN:
if (!index->amhasgetbitmap)
return NIL;
+
+ if (grouped)
+ return NIL;
break;
case ST_ANYSCAN:
/* either or both are OK */
*************** build_index_paths(PlannerInfo *root, Rel
*** 1032,1037 ****
--- 1042,1051 ----
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
*/
+ can_agg_sorted = true;
+ group_clauses = NIL;
+ group_exprs = NIL;
+ agg_exprs = NIL;
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1062,1086 ----
outer_relids,
loop_count,
false);
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root, (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1109,1139 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1161,1185 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else if (can_agg_sorted)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! Assert(agg_path != NULL);
! result = lappend(result, agg_path);
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1203,1227 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1244,1250 ****
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL);
result = list_concat(result, indexpaths);
}
--- 1336,1343 ----
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index de7044d..212b40c
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***************
*** 21,26 ****
--- 21,27 ----
#include "optimizer/cost.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
+ #include "optimizer/tlist.h"
/* Hook for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
*************** static void try_partial_mergejoin_path(P
*** 37,65 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
--- 38,86 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static void sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static bool is_grouped_join_target_complete(PlannerInfo *root,
! PathTarget *jointarget,
! Path *outer_path,
! Path *inner_path);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
*************** static void generate_mergejoin_paths(Pla
*** 76,82 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 97,106 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
/*
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 197,204 ****
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 2. Consider paths where the outer relation need not be explicitly
--- 221,232 ----
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
+ {
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 2. Consider paths where the outer relation need not be explicitly
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 208,215 ****
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
#ifdef NOT_USED
--- 236,247 ----
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
+ {
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
#ifdef NOT_USED
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 235,242 ****
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 267,278 ----
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
+ {
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 300,309 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
--- 336,355 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_nestloop_path(PlannerInfo *root,
*** 311,329 ****
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! ((!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path))))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 357,379 ----
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
+ *
+ * Grouped path should never be parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped ||
! (!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path)))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_nestloop_path(PlannerInfo *root,
*** 339,360 ****
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer));
}
else
{
--- 389,425 ----
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! required_outer, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate && required_outer == NULL)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, join_path, grouped);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 375,383 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 440,456 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_nestloop_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 401,422 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL));
}
/*
--- 474,555 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! NULL, join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_nestloop_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_nestloop_path(root, joinrel, outer_path, inner_path, pathkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_nestloop_path(root, joinrel, outer_path, inner_path,
! pathkeys, jointype, extra, true,
! do_aggregate);
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 435,444 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
if (is_partial)
{
--- 568,587 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
if (is_partial)
{
*************** try_mergejoin_path(PlannerInfo *root,
*** 451,472 ****
outersortkeys,
innersortkeys,
jointype,
! extra);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 594,618 ----
outersortkeys,
innersortkeys,
jointype,
! extra,
! grouped,
! do_aggregate);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 488,511 ****
outersortkeys, innersortkeys,
extra->sjinfo);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
else
{
--- 634,679 ----
outersortkeys, innersortkeys,
extra->sjinfo);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 529,537 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 697,713 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_mergejoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 564,587 ****
outersortkeys, innersortkeys,
extra->sjinfo);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
/*
--- 740,910 ----
outersortkeys, innersortkeys,
extra->sjinfo);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_mergejoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_mergejoin_path(root, joinrel, outer_path, inner_path, pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, false, true, do_aggregate);
! else
! try_partial_mergejoin_path(root, joinrel, outer_path, inner_path,
! pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! }
!
! static void
! try_mergejoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false, false);
! }
! else if (grouped_outer || grouped_inner)
! {
! Assert(!do_aggregate);
!
! /*
! * Exactly one of the input paths is grouped, so create a grouped join
! * path.
! */
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false);
! }
! /* Preform explicit aggregation only if suitable target exists. */
! else if (joinrel->gpi != NULL)
! {
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial, true);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 596,644 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
! outer_path, inner_path,
! extra->sjinfo, &extra->semifactors);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! required_outer,
! hashclauses));
}
else
{
--- 919,994 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
+ *
+ * TODO Need to consider aggregation here?
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
! outer_path, inner_path, extra->sjinfo, &extra->semifactors);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist,
! required_outer, hashclauses,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 659,667 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1009,1025 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_hashjoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 685,706 ****
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! NULL,
! hashclauses));
}
/*
--- 1043,1139 ----
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist, NULL,
! hashclauses, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! NIL, grouped))
! {
! add_partial_path(joinrel, (Path *) join_path , grouped);
! }
! }
!
! /*
! * Create a new grouped hash join path by joining a grouped path to plain
! * (non-grouped) one, or by joining 2 plain relations and applying grouping on
! * the result.
! *
! * Joining of 2 grouped paths is not supported. If a grouped relation A was
! * joined to grouped relation B, then the grouping of B reduces the number of
! * times each group of A is appears in the join output. This makes difference
! * for some aggregates, e.g. sum().
! *
! * If do_aggregate is true, neither input rel is grouped so we need to
! * aggregate the join result explicitly.
! *
! * partial argument tells whether the join path should be considered partial.
! */
! static void
! try_grouped_hashjoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_hashjoin_path(root, joinrel, outer_path, inner_path, hashclauses,
! jointype, extra, true, do_aggregate);
! else
! try_partial_hashjoin_path(root, joinrel, outer_path, inner_path,
! hashclauses, jointype, extra, true,
! do_aggregate);
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 751,757 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
Path *outer_path;
--- 1184,1223 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
! {
! if (!grouped)
! {
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, false);
! }
! else
! {
! /* Use all the supported strategies to generate grouped join. */
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, true, false, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, true, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, true);
! }
! }
!
! /*
! * TODO As merge_pathkeys shouldn't differ across execution, use a separate
! * function to derive them and pass them here in a list.
! */
! static void
! sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
Path *outer_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 760,765 ****
--- 1226,1232 ----
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped_result;
/*
* We only consider the cheapest-total-cost input paths, since we are
*************** sort_inner_and_outer(PlannerInfo *root,
*** 774,781 ****
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! outer_path = outerrel->cheapest_total_path;
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 1241,1267 ----
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->pathlist != NIL)
! outer_path = linitial(outerrel->gpi->pathlist);
! else
! return;
! }
! else
! outer_path = outerrel->cheapest_total_path;
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
!
! grouped_result = grouped_outer || grouped_inner || do_aggregate;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** sort_inner_and_outer(PlannerInfo *root,
*** 821,833 ****
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! cheapest_partial_outer = (Path *) linitial(outerrel->partial_pathlist);
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
/*
--- 1307,1356 ----
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->partial_pathlist != NIL)
! cheapest_partial_outer = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
! else
! return;
! }
! else
! cheapest_partial_outer = (Path *)
! linitial(outerrel->partial_pathlist);
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else
+ {
+ Assert(innerrel->gpi != NULL);
+ inner_pathlist = innerrel->gpi->pathlist;
+ }
+
+ /*
+ * All the grouped paths should be unparameterized, so the
+ * function is overly stringent in the grouped_inner case, but
+ * still useful.
+ */
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 903,935 ****
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra);
}
}
--- 1426,1484 ----
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! if (!grouped_result)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false, false, false);
! else
! {
! try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! {
! if (!grouped_result)
! {
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra, false, false);
! }
! else
! {
! try_mergejoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! true,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
! }
}
}
*************** sort_inner_and_outer(PlannerInfo *root,
*** 946,951 ****
--- 1495,1508 ----
* some sort key requirements). So, we consider truncations of the
* mergeclause list as well as the full list. (Ideally we'd consider all
* subsets of the mergeclause list, but that seems way too expensive.)
+ *
+ * grouped_outer - is outerpath grouped?
+ * grouped_inner - use grouped paths of innerrel?
+ * do_aggregate - apply (partial) aggregation to the output?
+ *
+ * TODO If subsequent calls often differ only by the 3 arguments above,
+ * consider a workspace structure to share useful info (eg merge clauses)
+ * across calls.
*/
static void
generate_mergejoin_paths(PlannerInfo *root,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 957,963 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1514,1523 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1008,1024 ****
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1568,1585 ----
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner, do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1074,1089 ****
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
List *newclauses = NIL;
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
--- 1635,1656 ----
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
+ List *inner_pathlist = NIL;
Path *innerpath;
List *newclauses = NIL;
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else if (innerrel->gpi != NULL)
+ inner_pathlist = innerrel->gpi->pathlist;
+
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1106,1126 ****
}
else
newclauses = mergeclauses;
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
--- 1673,1697 ----
}
else
newclauses = mergeclauses;
!
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
!
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1151,1167 ****
else
newclauses = mergeclauses;
}
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1722,1740 ----
else
newclauses = mergeclauses;
}
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1196,1201 ****
--- 1769,1776 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'grouped' indicates that the at least one relation in the join has been
+ * aggregated.
*/
static void
match_unsorted_outer(PlannerInfo *root,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1203,1209 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
--- 1778,1785 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1213,1218 ****
--- 1789,1816 ----
ListCell *lc1;
/*
+ * If grouped join path is requested, we ignore cases where either input
+ * path needs to be unique. For each side we should expect either grouped
+ * or plain relation, which differ quite a bit.
+ *
+ * XXX Although unique-ification of grouped path might result in too
+ * expensive input path (note that grouped input relation is not
+ * necessarily unique, regardless the grouping keys --- one or more plain
+ * relation could already have been joined to it), we might want to
+ * unique-ify the input relation in the future at least in the case it's a
+ * plain relation.
+ *
+ * (Materialization is not involved in grouped paths for similar reasons.)
+ */
+ if (grouped &&
+ (jointype == JOIN_UNIQUE_OUTER || jointype == JOIN_UNIQUE_INNER))
+ return;
+
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
+ /*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
* are doing a right or full mergejoin, we must use *all* the mergeclauses
* as join clauses, else we will not have a valid plan. (Although these
*************** match_unsorted_outer(PlannerInfo *root,
*** 1268,1274 ****
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK)
{
/*
* Consider materializing the cheapest inner path, unless
--- 1866,1872 ----
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK && !grouped)
{
/*
* Consider materializing the cheapest inner path, unless
*************** match_unsorted_outer(PlannerInfo *root,
*** 1299,1304 ****
--- 1897,1904 ----
*/
if (save_jointype == JOIN_UNIQUE_OUTER)
{
+ Assert(!grouped);
+
if (outerpath != outerrel->cheapest_total_path)
continue;
outerpath = (Path *) create_unique_path(root, outerrel,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1326,1332 ****
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra);
}
else if (nestjoinOK)
{
--- 1926,1933 ----
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1342,1365 ****
{
Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra);
}
! /* Also consider materialized form of the cheapest inner path */
! if (matpath != NULL)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 1943,1988 ----
{
Path *innerpath = (Path *) lfirst(lc2);
! if (!grouped)
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra, false, false);
! else
! {
! /*
! * Since both input paths are plain, request explicit
! * aggregation.
! */
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! true,
! false);
! }
}
! /*
! * Also consider materialized form of the cheapest inner path.
! *
! * (There's no matpath for grouped join.)
! */
! if (matpath != NULL && !grouped)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1374,1380 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 1997,2073 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, false, false, grouped);
!
! /* Try to join the plain outer relation to grouped inner. */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER &&
! innerrel->gpi != NULL && outerrel->gpi == NULL)
! {
! Path *inner_cheapest_grouped = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (PATH_PARAM_BY_REL(inner_cheapest_grouped, outerrel))
! continue;
!
! /* XXX Shouldn't Assert() be used here instead? */
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! /*
! * Only outer grouped path is interesting in this case: grouped
! * path on the inner side of NL join would imply repeated
! * aggregation somewhere in the inner path.
! */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_grouped, merge_pathkeys,
! false, false, true, false);
! }
! }
!
! /*
! * Combine grouped outer and plain inner paths.
! */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER)
! {
! /*
! * If the inner rel had a grouped target, its plain paths should be
! * ignored. Otherwise we could create grouped paths with different
! * targets.
! */
! if (outerrel->gpi != NULL && innerrel->gpi == NULL &&
! inner_cheapest_total != NULL)
! {
! /* Nested loop paths. */
! foreach(lc1, outerrel->gpi->pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
!
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra,
! false,
! false);
!
! /* Merge join paths. */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_total, merge_pathkeys,
! false, true, false, false);
! }
! }
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1394,1401 ****
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 2087,2107 ----
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! {
! if (!grouped)
! /* Plain partial paths. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, false, false);
! else
! {
! /* Grouped partial paths with explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, true);
! /* Grouped partial paths w/o explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, false);
! }
! }
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1415,1421 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
--- 2121,2127 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total, grouped);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1438,1447 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
--- 2144,2162 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped)
{
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (jointype == JOIN_UNIQUE_OUTER ||
+ jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
*************** consider_parallel_mergejoin(PlannerInfo
*** 1454,1462 ****
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
! extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 2169,2224 ----
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! if (!grouped)
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true,
! false, false, false);
! else
! {
! /*
! * Create grouped join by joining plain rels and aggregating the
! * result.
! */
! Assert(joinrel->gpi != NULL);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, false, false, true);
!
! /* Combine the plain outer with grouped inner one(s). */
! if (outerrel->gpi == NULL && innerrel->gpi != NULL)
! {
! Path *inner_cheapest_grouped = (Path *)
! linitial(innerrel->gpi->pathlist);
!
! if (inner_cheapest_grouped != NULL &&
! inner_cheapest_grouped->parallel_safe)
! generate_mergejoin_paths(root, joinrel, innerrel,
! outerpath, jointype, extra,
! false, inner_cheapest_grouped,
! merge_pathkeys,
! true, false, true, false);
! }
! }
! }
!
! /* In addition, try to join grouped outer to plain inner one(s). */
! if (grouped && outerrel->gpi != NULL && innerrel->gpi == NULL)
! {
! foreach(lc1, outerrel->gpi->partial_pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys;
!
! merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, true, false, false);
! }
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1477,1491 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
--- 2239,2283 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate)
{
JoinType save_jointype = jointype;
+ List *outer_pathlist;
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (save_jointype == JOIN_UNIQUE_OUTER ||
+ save_jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! if (!grouped || do_aggregate)
! {
! /*
! * If creating grouped paths by explicit aggregation, the input paths
! * must be plain.
! */
! outer_pathlist = outerrel->partial_pathlist;
! }
! else if (outerrel->gpi != NULL)
! {
! /*
! * Only the outer paths are accepted as grouped when we try to combine
! * grouped and plain ones. Grouped inner path implies repeated
! * aggregation, which doesn't sound as a good idea.
! */
! outer_pathlist = outerrel->gpi->partial_pathlist;
! }
! else
! return;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1516,1522 ****
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
--- 2308,2314 ----
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER && !grouped)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1526,1533 ****
Assert(innerpath);
}
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
--- 2318,2343 ----
Assert(innerpath);
}
! if (!grouped)
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, false);
! else if (do_aggregate)
! {
! /* Request aggregation as both input rels are plain. */
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! true, true);
! }
! /*
! * Only combine the grouped outer path with the plain inner if the
! * inner relation cannot produce grouped paths. Otherwise we could
! * generate grouped paths with different targets.
! */
! else if (innerrel->gpi == NULL)
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, true);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1549,1561 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
--- 2359,2376 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1605,1610 ****
--- 2420,2428 ----
* can't use a hashjoin. (There's no use looking for alternative
* input paths, since these should already be the least-parameterized
* available paths.)
+ *
+ * (The same check should work for grouped paths, as these don't
+ * differ in parameterization.)
*/
if (PATH_PARAM_BY_REL(cheapest_total_outer, innerrel) ||
PATH_PARAM_BY_REL(cheapest_total_inner, outerrel))
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1624,1630 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
--- 2442,2449 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1640,1646 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
--- 2459,2466 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1649,1711 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
}
else
{
! /*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized ones.
! * There is no use in generating parameterized paths on the basis
! * of possibly cheap startup cost, so this is sufficient.
! */
! ListCell *lc1;
! ListCell *lc2;
!
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
- Path *outerpath = (Path *) lfirst(lc1);
-
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
! Path *innerpath = (Path *) lfirst(lc2);
/*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
*/
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
continue;
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra);
}
}
}
--- 2469,2622 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
}
else
{
! if (!grouped)
{
/*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized
! * ones. There is no use in generating parameterized paths on
! * the basis of possibly cheap startup cost, so this is
! * sufficient.
*/
! ListCell *lc1;
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false, false);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
! Path *outerpath = (Path *) lfirst(lc1);
! ListCell *lc2;
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
! /*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
! */
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
! continue;
!
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
!
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra,
! false, false);
! }
! }
! }
! else
! {
! /* Create grouped paths if possible. */
! /*
! * TODO
! *
! * Consider processing JOIN_UNIQUE_INNER and JOIN_UNIQUE_OUTER
! * join types, ie perform grouping of the inner / outer rel if
! * it's not unique yet and if the grouping is legal.
! */
! if (jointype == JOIN_UNIQUE_OUTER ||
! jointype == JOIN_UNIQUE_INNER)
! return;
!
! /*
! * Join grouped relation to non-grouped one.
! *
! * Do not use plain path of the input rel whose target does
! * have GroupedPahtInfo. For example (assuming that join of
! * two grouped rels is not supported), the only way to
! * evaluate SELECT sum(a.x), sum(b.y) ... is to join "a" and
! * "b" and aggregate the result. Otherwise the path target
! * wouldn't match joinrel->gpi->target. TODO Move this comment
! * elsewhere as it seems common to all join kinds.
! */
! /*
! * TODO Allow outer join if the grouped rel is on the
! * non-nullable side.
! */
! if (jointype == JOIN_INNER)
! {
! Path *grouped_path, *plain_path;
!
! if (outerrel->gpi != NULL &&
! outerrel->gpi->pathlist != NIL &&
! innerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(outerrel->gpi->pathlist);
! plain_path = cheapest_total_inner;
! try_grouped_hashjoin_path(root, joinrel,
! grouped_path, plain_path,
! hashclauses, jointype,
! extra, false, false);
! }
! else if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(innerrel->gpi->pathlist);
! plain_path = cheapest_total_outer;
! try_grouped_hashjoin_path(root, joinrel, plain_path,
! grouped_path, hashclauses,
! jointype, extra,
! false, false);
!
! if (cheapest_startup_outer != NULL &&
! cheapest_startup_outer != cheapest_total_outer)
! {
! plain_path = cheapest_startup_outer;
! try_grouped_hashjoin_path(root, joinrel,
! plain_path,
! grouped_path,
! hashclauses,
! jointype, extra,
! false, false);
! }
! }
}
+
+ /*
+ * Try to join plain relations and make a grouped rel out of
+ * the join.
+ *
+ * Since aggregation needs the whole relation, we are only
+ * interested in total costs.
+ */
+ try_grouped_hashjoin_path(root, joinrel,
+ cheapest_total_outer,
+ cheapest_total_inner,
+ hashclauses,
+ jointype, extra, true, false);
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1743,1758 ****
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra);
}
}
}
/*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
--- 2654,2880 ----
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (!grouped)
! {
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra,
! false, false);
! }
! else if (joinrel->gpi != NULL)
! {
! /*
! * Grouped partial path.
! *
! * 1. Apply aggregation to the plain partial join path.
! */
! if (cheapest_safe_inner != NULL)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, true, true);
!
! /*
! * 2. Join the cheapest partial grouped outer path (if one
! * exists) to cheapest_safe_inner (there's no reason to look
! * for another inner path than what we used for non-grouped
! * partial join path).
! */
! if (outerrel->gpi != NULL &&
! outerrel->gpi->partial_pathlist != NIL &&
! innerrel->gpi == NULL &&
! cheapest_safe_inner != NULL)
! {
! Path *outer_path;
!
! outer_path = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
!
! try_grouped_hashjoin_path(root, joinrel, outer_path,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, false, true);
! }
!
! /*
! * 3. Join the cheapest_partial_outer path (again, no reason
! * to use different outer path than the one we used for plain
! * partial join) to the cheapest grouped inner path if the
! * latter exists and is parallel-safe.
! */
! if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! Path *inner_path;
!
! inner_path = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (inner_path->parallel_safe)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! inner_path,
! hashclauses,
! jointype, extra,
! false, true);
! }
!
! /*
! * Other combinations seem impossible because: 1. At most 1
! * input relation of the join can be grouped, 2. the inner
! * path must not be partial.
! */
! }
}
}
}
/*
+ * Do the input paths emit all the aggregates contained in the grouped target
+ * of the join?
+ *
+ * The point is that one input relation might be unable to evaluate some
+ * aggregate(s), so it'll only generate plain paths. It's wrong to combine
+ * such plain paths with grouped ones that the other input rel might be able
+ * to generate because the result would miss the aggregate(s) the first
+ * relation failed to evaluate.
+ *
+ * TODO For better efficiency, consider storing Bitmapset of
+ * GroupedVarInfo.gvid in GroupedPathInfo.
+ */
+ static bool
+ is_grouped_join_target_complete(PlannerInfo *root, PathTarget *jointarget,
+ Path *outer_path, Path *inner_path)
+ {
+ RelOptInfo *outer_rel = outer_path->parent;
+ RelOptInfo *inner_rel = inner_path->parent;
+ ListCell *l1;
+
+ /*
+ * Join of two grouped relations is not supported.
+ *
+ * This actually isn't check of target completeness --- can it be located
+ * elsewhere?
+ */
+ if (outer_rel->gpi != NULL && inner_rel->gpi != NULL)
+ return false;
+
+ foreach(l1, jointarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+ GroupedVar *gvar;
+ GroupedVarInfo *gvi = NULL;
+ ListCell *l2;
+ bool found = false;
+
+ /* Only interested in aggregates. */
+ if (!IsA(expr, GroupedVar))
+ continue;
+
+ gvar = castNode(GroupedVar, expr);
+
+ /* Find the corresponding GroupedVarInfo. */
+ foreach(l2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi_tmp = castNode(GroupedVarInfo, lfirst(l2));
+
+ if (gvi_tmp->gvid == gvar->gvid)
+ {
+ gvi = gvi_tmp;
+ break;
+ }
+ }
+ Assert(gvi != NULL);
+
+ /*
+ * If any aggregate references both input relations, something went
+ * wrong during construction of one of the input targets: one input
+ * rel is grouped, but no grouping target should have been created for
+ * it if some aggregate required more than that input rel.
+ */
+ Assert(gvi->gv_eval_at == NULL ||
+ !(bms_overlap(gvi->gv_eval_at, outer_rel->relids) &&
+ bms_overlap(gvi->gv_eval_at, inner_rel->relids)));
+
+ /*
+ * If the aggregate belongs to the plain relation, it probably
+ * means that non-grouping expression made aggregation of that
+ * input relation impossible. Since that expression is not
+ * necessarily emitted by the current join, aggregation might be
+ * possible here. On the other hand, aggregation of a join which
+ * already contains a grouped relation does not seem too
+ * beneficial.
+ *
+ * XXX The condition below is also met if the query contains both
+ * "star aggregate" and a normal one. Since the earlier can be
+ * added to any base relation, and since we don't support join of
+ * 2 grouped relations, join of arbitrary 2 relations will always
+ * result in a plain relation.
+ *
+ * XXX If we conclude that aggregation is worth, only consider
+ * this test failed if target usable for aggregation cannot be
+ * created (i.e. the non-grouping expression is in the output of
+ * the current join).
+ */
+ if ((outer_rel->gpi == NULL &&
+ bms_overlap(gvi->gv_eval_at, outer_rel->relids))
+ || (inner_rel->gpi == NULL &&
+ bms_overlap(gvi->gv_eval_at, inner_rel->relids)))
+ return false;
+
+ /* Look for the aggregate in the input targets. */
+ if (outer_rel->gpi != NULL)
+ {
+ /* No more than one input path should be grouped. */
+ Assert(inner_rel->gpi == NULL);
+
+ foreach(l2, outer_path->pathtarget->exprs)
+ {
+ expr = (Expr *) lfirst(l2);
+
+ if (!IsA(expr, GroupedVar))
+ continue;
+
+ gvar = castNode(GroupedVar, expr);
+ if (gvar->gvid == gvi->gvid)
+ {
+ found = true;
+ break;
+ }
+ }
+ }
+ else if (!found && inner_rel->gpi != NULL)
+ {
+ Assert(outer_rel->gpi == NULL);
+
+ foreach(l2, inner_path->pathtarget->exprs)
+ {
+ expr = (Expr *) lfirst(l2);
+
+ if (!IsA(expr, GroupedVar))
+ continue;
+
+ gvar = castNode(GroupedVar, expr);
+ if (gvar->gvid == gvi->gvid)
+ {
+ found = true;
+ break;
+ }
+ }
+ }
+
+ /* Even a single missing aggregate causes the whole test to fail. */
+ if (!found)
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 6a0c67b..58aea01
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** mark_dummy_rel(RelOptInfo *rel)
*** 1217,1223 ****
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
--- 1217,1223 ----
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index a2fe661..91d855c
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** create_tidscan_paths(PlannerInfo *root,
*** 266,270 ****
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 266,270 ----
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer), false);
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 53aefbd..f19e18c
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,19 ****
--- 14,20 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
***************
*** 26,31 ****
--- 27,33 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 45,50 ****
--- 47,53 ----
} PostponedQual;
+ static void create_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** add_vars_to_targetlist(PlannerInfo *root
*** 240,245 ****
--- 243,532 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate and setup
+ * GroupedPathInfo for each base relation that can product grouped paths.
+ *
+ * XXX In the future we might want to create GroupedVarInfo for grouping
+ * expressions too, so that grouping key is not limited to plain Var if the
+ * grouping takes place below the top-level join.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouping_info_to_base_rels(PlannerInfo *root)
+ {
+ int i;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause || root->group_pathkeys == NIL)
+ return;
+
+ /* TODO This is just for PoC. Relax the limitation later. */
+ if (root->parse->havingQual)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_grouped_var_infos(root);
+
+ /* Is no grouping is possible below the top-level join? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+
+ /*
+ * "other rels" will have their targets built later, by translation of
+ * the target of the parent rel - see set_append_rel_size. If we
+ * wanted to prepare the child rels here, we'd need another iteration
+ * of simple_rel_array_size.
+ */
+ if (rel != NULL && rel->reloptkind == RELOPT_BASEREL)
+ prepare_rel_for_grouping(root, rel);
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ *
+ * TODO Include aggregates from HAVING clause.
+ */
+ static void
+ create_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ /*
+ * TODO Check if processed_tlist contains the HAVING aggregates. If not,
+ * get them elsewhere.
+ */
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ Expr *expr = (Expr *) lfirst(lc2);
+
+ gvi = castNode(GroupedVarInfo, expr);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ /* TODO Initialize gv_width. */
+ gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ {
+ Assert(aggref->aggstar);
+ gvi->gv_eval_at = NULL;
+ }
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Check if all the expressions of rel->reltarget can be used as grouping
+ * expressions and create target for grouped paths.
+ *
+ * If we succeed to create the grouping target, also replace rel->reltarget
+ * with a new one that has sortgrouprefs initialized -- this is necessary for
+ * create_agg_plan to match the grouping clauses against the input target
+ * expressions.
+ *
+ * rel_agg_attrs is a set attributes of the relation referenced by aggregate
+ * arguments. These can exist in the (plain) target without being grouping
+ * expressions.
+ *
+ * rel_agg_vars should be passed instead if rel is a join.
+ *
+ * TODO How about PHVs?
+ *
+ * TODO Make sure cost / width of both "result" and "plain" are correct.
+ */
+ PathTarget *
+ create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs, List *rel_agg_vars)
+ {
+ PathTarget *result, *plain;
+ ListCell *lc;
+
+ /* The plan to be returned. */
+ result = create_empty_pathtarget();
+ /* The one to replace rel->reltarget. */
+ plain = create_empty_pathtarget();
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *texpr;
+ Index sortgroupref;
+ bool agg_arg_only = false;
+
+ texpr = (Expr *) lfirst(lc);
+
+ sortgroupref = get_expr_sortgroupref(root, texpr);
+ if (sortgroupref > 0)
+ {
+ /* It's o.k. to use the target expression for grouping. */
+ add_column_to_pathtarget(result, texpr, sortgroupref);
+
+ /*
+ * As for the plain target, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(plain, texpr, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * It may still be o.k. if the expression is only contained in Aggref
+ * - then it's not expected in the grouped output.
+ *
+ * TODO Try to handle generic expression, not only Var. That might
+ * require us to create rel->reltarget of the grouping rel in
+ * parallel to that of the plain rel, and adding whole expressions
+ * instead of individual vars.
+ */
+ if (IsA(texpr, Var))
+ {
+ Var *arg_var = castNode(Var, texpr);
+
+ if (rel->relid > 0)
+ {
+ AttrNumber varattno;
+
+ /*
+ * For a single relation we only need to check attribute
+ * number.
+ *
+ * Apply the same offset that pull_varattnos() did.
+ */
+ varattno = arg_var->varattno - FirstLowInvalidHeapAttributeNumber;
+
+ if (bms_is_member(varattno, rel_agg_attrs))
+ agg_arg_only = true;
+ }
+ else
+ {
+ ListCell *lc2;
+
+ /* Join case. */
+ foreach(lc2, rel_agg_vars)
+ {
+ Var *var = castNode(Var, lfirst(lc2));
+
+ if (var->varno == arg_var->varno &&
+ var->varattno == arg_var->varattno)
+ {
+ agg_arg_only = true;
+ break;
+ }
+ }
+ }
+
+ if (agg_arg_only)
+ {
+ /*
+ * This expression is not suitable for grouping, but the
+ * aggregation input target ought to stay complete.
+ */
+ add_column_to_pathtarget(plain, texpr, 0);
+ }
+ }
+
+ /*
+ * A single mismatched expression makes the whole relation useless
+ * for grouping.
+ */
+ if (!agg_arg_only)
+ {
+ /*
+ * TODO This seems possible to happen multiple times per relation,
+ * so result might be worth freeing. Implement free_pathtarget()?
+ * Or mark the relation as inappropriate for grouping?
+ */
+ /* TODO Free both result and plain. */
+ return NULL;
+ }
+ }
+
+ if (list_length(result->exprs) == 0)
+ {
+ /* TODO free_pathtarget(result); free_pathtarget(plain) */
+ result = NULL;
+ }
+
+ /* Apply the adjusted input target as the replacement is complete now.q */
+ rel->reltarget = plain;
+
+ return result;
+ }
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 5565736..058af2c
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,229 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual));
}
/*
--- 223,229 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual), false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 3c58d05..5db7dec
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,89 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,89 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals), false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 114,120 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 177,182 ****
--- 178,191 ----
(*qp_callback) (root, qp_extra);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, Initialize GroupedPathInfo
+ * of base relations capable to do the grouping and setup
+ * root->grouped_var_list.
+ */
+ add_grouping_info_to_base_rels(root);
+
+ /*
* Examine any "placeholder" expressions generated during subquery pullup.
* Make sure that the Vars they need are marked as needed at the relevant
* join level. This must be done before join removal because it might
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index f99257b..8245ce0
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 130,138 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
--- 130,135 ----
*************** inheritance_planner(PlannerInfo *root)
*** 1420,1426 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)));
}
/*--------------------
--- 1417,1423 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)), false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 2041,2047 ****
}
/* And shove it into final_rel */
! add_path(final_rel, path);
}
/*
--- 2038,2044 ----
}
/* And shove it into final_rel */
! add_path(final_rel, path, false);
}
/*
*************** get_number_of_groups(PlannerInfo *root,
*** 3445,3484 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3442,3447 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3599,3605 ****
(List *) parse->havingQual);
}
! add_path(grouped_rel, path);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
--- 3562,3568 ----
(List *) parse->havingQual);
}
! add_path(grouped_rel, path, false);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
*************** create_grouping_paths(PlannerInfo *root,
*** 3776,3782 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
--- 3739,3746 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3785,3791 ****
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups));
}
}
}
--- 3749,3756 ----
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3816,3822 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
}
}
}
--- 3781,3788 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3868,3874 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
else if (parse->groupClause)
{
--- 3834,3840 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
else if (parse->groupClause)
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3883,3889 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
else
{
--- 3849,3855 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
else
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3932,3938 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3898,3904 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3941,3947 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
/*
* The point of using Gather Merge rather than Gather is that it
--- 3907,3913 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
/*
* The point of using Gather Merge rather than Gather is that it
*************** create_grouping_paths(PlannerInfo *root,
*** 3994,4000 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3960,3966 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4003,4009 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
}
}
--- 3969,3975 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4048,4054 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
}
--- 4014,4020 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4086,4094 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
}
}
}
/* Give a helpful error if we failed to find any implementation */
--- 4052,4128 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
}
}
+
+ /*
+ * If input_rel has partially aggregated partial paths, gather them
+ * and perform the final aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL &&
+ input_rel->gpi->partial_pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->partial_pathlist);
+ double total_groups = path->rows * path->parallel_workers;
+
+ path = (Path *) create_gather_path(root,
+ input_rel,
+ path,
+ path->pathtarget,
+ NULL,
+ &total_groups);
+
+ /*
+ * The input path is partially aggregated and the final
+ * aggregation - if the path wins - will be done below. So we're
+ * done with it for now.
+ *
+ * The top-level grouped_rel needs to receive the path into
+ * regular pathlist, as opposed grouped_rel->gpi->pathlist.
+ */
+
+ add_path(input_rel, path, false);
+ }
+
+ /*
+ * If input_rel has partially aggregated paths, perform the final
+ * aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *pre_agg = (Path *) linitial(input_rel->gpi->pathlist);
+
+ dNumGroups = get_number_of_groups(root, pre_agg->rows, gd);
+
+ MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+
+ add_path(grouped_rel,
+ (Path *) create_agg_path(root, grouped_rel,
+ pre_agg,
+ target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ &agg_final_costs,
+ dNumGroups),
+ false);
+ }
}
/* Give a helpful error if we failed to find any implementation */
*************** consider_groupingsets_paths(PlannerInfo
*** 4288,4294 ****
strat,
new_rollups,
agg_costs,
! dNumGroups));
return;
}
--- 4322,4328 ----
strat,
new_rollups,
agg_costs,
! dNumGroups), false);
return;
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4446,4452 ****
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups));
}
}
--- 4480,4486 ----
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups), false);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4463,4469 ****
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups));
}
/*
--- 4497,4503 ----
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups), false);
}
/*
*************** create_one_window_path(PlannerInfo *root
*** 4648,4654 ****
window_pathkeys);
}
! add_path(window_rel, path);
}
/*
--- 4682,4688 ----
window_pathkeys);
}
! add_path(window_rel, path, false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4754,4760 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
}
--- 4788,4794 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4781,4787 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
/*
--- 4815,4821 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4828,4834 ****
parse->distinctClause,
NIL,
NULL,
! numDistinctRows));
}
/* Give a helpful error if we failed to find any implementation */
--- 4862,4868 ----
parse->distinctClause,
NIL,
NULL,
! numDistinctRows), false);
}
/* Give a helpful error if we failed to find any implementation */
*************** create_ordered_paths(PlannerInfo *root,
*** 4926,4932 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 4960,4966 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
*************** create_ordered_paths(PlannerInfo *root,
*** 4976,4982 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5010,5016 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index cdb8e95..fca3c0b
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
tlist_vinfo vars[FLEXIBLE_ARRAY_MEMBER]; /* has num_vars entries */
} indexed_tlist;
*************** set_upper_references(PlannerInfo *root,
*** 1725,1733 ****
--- 1726,1777 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the original list.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1937,1942 ****
--- 1981,1987 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
/* Find the Vars and fill in the index array */
*************** build_tlist_index(List *tlist)
*** 1956,1961 ****
--- 2001,2008 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else
itlist->has_non_vars = true;
}
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2233,2238 ****
--- 2280,2310 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2389,2395 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
--- 2461,2468 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
new file mode 100644
index e327e66..e90d72f
*** a/src/backend/optimizer/prep/prepunion.c
--- b/src/backend/optimizer/prep/prepunion.c
*************** plan_set_operations(PlannerInfo *root)
*** 207,213 ****
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
--- 207,213 ----
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path, false);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 8536212..39813b8
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 24,29 ****
--- 24,31 ----
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if get_grouping_expressions ends up in another module. */
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
*************** set_cheapest(RelOptInfo *parent_rel)
*** 409,416 ****
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
--- 411,419 ----
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
+ List *pathlist;
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 427,432 ****
--- 430,443 ----
/* Pretend parameterized paths have no pathkeys, per comment above */
new_path_pathkeys = new_path->param_info ? NIL : new_path->pathkeys;
+ if (!grouped)
+ pathlist = parent_rel->pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->pathlist;
+ }
+
/*
* Loop to check proposed new path against old paths. Note it is possible
* for more than one old path to be tossed out because new_path dominates
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 436,442 ****
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
--- 447,453 ----
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 582,589 ****
*/
if (remove_old)
{
! parent_rel->pathlist = list_delete_cell(parent_rel->pathlist,
! p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
--- 593,599 ----
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 614,622 ****
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(parent_rel->pathlist, insert_after, new_path);
else
! parent_rel->pathlist = lcons(new_path, parent_rel->pathlist);
}
else
{
--- 624,637 ----
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->pathlist = pathlist;
! else
! parent_rel->gpi->pathlist = pathlist;
}
else
{
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 646,653 ****
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer)
{
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
--- 661,669 ----
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped)
{
+ List *pathlist;
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 656,664 ****
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup : parent_rel->consider_startup;
! foreach(p1, parent_rel->pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 672,689 ----
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup :
! parent_rel->consider_startup;
! if (!grouped)
! pathlist = parent_rel->pathlist;
! else
! {
! Assert(parent_rel->gpi != NULL);
! pathlist = parent_rel->gpi->pathlist;
! }
!
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 749,771 ****
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->partial_pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
--- 774,805 ----
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
+ List *pathlist;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
+
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
*************** add_partial_path(RelOptInfo *parent_rel,
*** 819,830 ****
}
/*
! * Remove current element from partial_pathlist if dominated by new.
*/
if (remove_old)
{
! parent_rel->partial_pathlist =
! list_delete_cell(parent_rel->partial_pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
--- 853,863 ----
}
/*
! * Remove current element from pathlist if dominated by new.
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
*************** add_partial_path(RelOptInfo *parent_rel,
*** 839,845 ****
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the partial_pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
--- 872,878 ----
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
*************** add_partial_path(RelOptInfo *parent_rel,
*** 850,859 ****
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(parent_rel->partial_pathlist, insert_after, new_path);
else
! parent_rel->partial_pathlist =
! lcons(new_path, parent_rel->partial_pathlist);
}
else
{
--- 883,896 ----
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->partial_pathlist = pathlist;
! else
! parent_rel->gpi->partial_pathlist = pathlist;
}
else
{
*************** add_partial_path(RelOptInfo *parent_rel,
*** 874,882 ****
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys)
{
ListCell *p1;
/*
* Our goal here is twofold. First, we want to find out whether this path
--- 911,928 ----
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys, bool grouped)
{
ListCell *p1;
+ List *pathlist;
+
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
/*
* Our goal here is twofold. First, we want to find out whether this path
*************** add_partial_path_precheck(RelOptInfo *pa
*** 886,895 ****
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist to be very short, and getting a definitive
! * answer at this stage avoids the need to call add_path_precheck.
*/
! foreach(p1, parent_rel->partial_pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 932,942 ----
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist / grouped_pathlist to be very short, and
! * getting a definitive answer at this stage avoids the need to call
! * add_path_precheck.
*/
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_partial_path_precheck(RelOptInfo *pa
*** 918,924 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL))
return false;
return true;
--- 965,971 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, grouped))
return false;
return true;
*************** calc_non_nestloop_required_outer(Path *o
*** 2056,2061 ****
--- 2103,2109 ----
* 'restrict_clauses' are the RestrictInfo nodes to apply at the join
* 'pathkeys' are the path keys of the new join path
* 'required_outer' is the set of required outer rels
+ * 'target' can be passed to override that of joinrel.
*
* Returns the resulting path node.
*/
*************** create_nestloop_path(PlannerInfo *root,
*** 2070,2076 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
--- 2118,2125 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
*************** create_nestloop_path(PlannerInfo *root,
*** 2103,2109 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2152,2158 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target == NULL ? joinrel->reltarget : target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2160,2172 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2209,2223 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2210,2215 ****
--- 2261,2267 ----
* 'required_outer' is the set of required outer rels
* 'hashclauses' are the RestrictInfo nodes to use as hash clauses
* (this should be a subset of the restrict_clauses list)
+ * 'target' can be passed to override that of joinrel.
*/
HashPath *
create_hashjoin_path(PlannerInfo *root,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2222,2234 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2274,2288 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_agg_path(PlannerInfo *root,
*** 2682,2688 ****
pathnode->path.pathtarget = target;
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
! pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
--- 2736,2742 ----
pathnode->path.pathtarget = target;
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
! pathnode->path.parallel_aware = true;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
*************** create_agg_path(PlannerInfo *root,
*** 2713,2718 ****
--- 2767,2942 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * first_call indicates whether the function is being called first time for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * group_clauses, group_exprs and agg_exprs are pointers to lists we populate
+ * when called first time for particular index, and that user passes for
+ * subsequent calls.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ if (first_call)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ if (first_call)
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows, NULL);
+
+ /* TODO HAVING qual. */
+ Assert(*group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, rel->gpi->target, AGG_SORTED,
+ AGGSPLIT_INITIAL_SERIAL, *group_clauses, NIL,
+ &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Appy partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ *
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (first_call)
+ {
+ /*
+ * Find one grouping clause per grouping column.
+ *
+ * All that create_agg_plan eventually needs of the clause is
+ * tleSortGroupRef, so we don't have to care that the clause
+ * expression might differ from texpr, in case texpr was derived from
+ * EC.
+ */
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows,
+ NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(*group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ rel->gpi->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 7912df0..cc7f6d3
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 25,30 ****
--- 25,31 ----
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
#include "utils/hsearch.h"
*************** typedef struct JoinHashEntry
*** 35,41 ****
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
--- 36,42 ----
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
*************** build_simple_rel(PlannerInfo *root, int
*** 120,125 ****
--- 121,127 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->gpi = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** build_join_rel(PlannerInfo *root,
*** 478,483 ****
--- 480,486 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 516,525 ****
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
--- 519,535 ----
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel, false);
! build_joinrel_tlist(root, joinrel, inner_rel, false);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ /* Try to build grouped target. */
+ /*
+ * TODO Consider if placeholders make sense here. If not, also make the
+ * related code below conditional.
+ */
+ prepare_rel_for_grouping(root, joinrel);
+
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
*************** min_join_parameterization(PlannerInfo *r
*** 647,663 ****
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel)
{
Relids relids = joinrel->relids;
ListCell *vars;
! foreach(vars, input_rel->reltarget->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
--- 657,699 ----
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped)
{
Relids relids = joinrel->relids;
+ PathTarget *input_target, *result;
ListCell *vars;
+ int i = -1;
! if (!grouped)
! {
! input_target = input_rel->reltarget;
! result = joinrel->reltarget;
! }
! else
! {
! if (input_rel->gpi != NULL)
! {
! input_target = input_rel->gpi->target;
! Assert(input_target != NULL);
! }
! else
! input_target = input_rel->reltarget;
!
! /* Caller should have initialized this. */
! Assert(joinrel->gpi != NULL);
!
! /* Default to the plain target. */
! result = joinrel->gpi->target;
! }
!
! foreach(vars, input_target->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
+ i++;
+
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
*************** build_joinrel_tlist(PlannerInfo *root, R
*** 681,690 ****
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
/* Yup, add it to the output */
! joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs, var);
/* Vars have cost zero, so no need to adjust reltarget->cost */
! joinrel->reltarget->width += baserel->attr_widths[ndx];
}
}
}
--- 717,740 ----
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
+ Index sortgroupref = 0;
+
/* Yup, add it to the output */
! if (input_target->sortgrouprefs)
! sortgroupref = input_target->sortgrouprefs[i];
!
! /*
! * Even if not used for grouping in the input path (the input path
! * is not necessarily grouped), it might be useful for grouping
! * higher in the join tree.
! */
! if (sortgroupref == 0)
! sortgroupref = get_expr_sortgroupref(root, (Expr *) var);
!
! add_column_to_pathtarget(result, (Expr *) var, sortgroupref);
!
/* Vars have cost zero, so no need to adjust reltarget->cost */
! result->width += baserel->attr_widths[ndx];
}
}
}
*************** get_appendrel_parampathinfo(RelOptInfo *
*** 1360,1362 ****
--- 1410,1561 ----
return ppi;
}
+
+ /*
+ * If the relation can produce grouped paths, create GroupedPathInfo for it
+ * and create target for the grouped paths.
+ */
+ void
+ prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *rel_aggregates;
+ Relids rel_agg_attrs = NULL;
+ List *rel_agg_vars = NIL;
+ bool found_higher;
+ ListCell *lc;
+ PathTarget *target_grouped;
+
+ if (rel->relid > 0)
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the aggregate
+ * would receive different input at the base rel level.
+ *
+ * TODO For RELOPT_JOINREL, do not return if all the joins that can set
+ * any entry of the grouped target (do we need to postpone this check
+ * until the grouped target is available, and should create_grouped_target
+ * take care?) of this rel to NULL are provably below rel. (It's ok if rel
+ * is one of these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return;
+
+ /*
+ * Check if some aggregates can be evaluated in this relation's target,
+ * and collect all vars referenced by these aggregates.
+ */
+ rel_aggregates = NIL;
+ found_higher = false;
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which typically means
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ Aggref *aggref = castNode(Aggref, gvi->gvexpr);
+
+ /*
+ * Accept the aggregate.
+ *
+ * GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.
+ */
+ rel_aggregates = lappend(rel_aggregates, gvi);
+
+ if (rel->relid > 0)
+ {
+ /*
+ * Simple relation. Collect attributes referenced by the
+ * aggregate arguments.
+ */
+ pull_varattnos((Node *) aggref, rel->relid, &rel_agg_attrs);
+ }
+ else
+ {
+ List *agg_vars;
+
+ /*
+ * Join. Collect vars referenced by the aggregate
+ * arguments.
+ */
+ /*
+ * TODO Can any argument contain PHVs? And if so, does it matter?
+ * Consider PVC_INCLUDE_PLACEHOLDERS | PVC_RECURSE_PLACEHOLDERS.
+ */
+ agg_vars = pull_var_clause((Node *) aggref,
+ PVC_RECURSE_AGGREGATES);
+ rel_agg_vars = list_concat(rel_agg_vars, agg_vars);
+ }
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * Remember that there is at least one aggregate that needs more
+ * than this rel.
+ */
+ found_higher = true;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function.
+ */
+ if (rel_aggregates == NIL)
+ {
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ if (found_higher)
+ {
+ /*
+ * If some aggregate(s) need only this rel but some other need
+ * multiple relations including the the current one, grouping of the
+ * current rel could steal some input variables from the "higher
+ * aggregate" (besides decreasing the number of input rows).
+ */
+ list_free(rel_aggregates);
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ /*
+ * If rel->reltarget can be used for aggregation, mark the relation as
+ * capable of grouping.
+ */
+ Assert(rel->gpi == NULL);
+ target_grouped = create_grouped_target(root, rel, rel_agg_attrs,
+ rel_agg_vars);
+ if (target_grouped != NULL)
+ {
+ GroupedPathInfo *gpi;
+
+ gpi = makeNode(GroupedPathInfo);
+ gpi->target = copy_pathtarget(target_grouped);
+ gpi->pathlist = NIL;
+ gpi->partial_pathlist = NIL;
+ rel->gpi = gpi;
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the target.
+ */
+ add_aggregates_to_target(root, gpi->target, rel_aggregates, rel);
+ }
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 0952385..dd962b7
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,413 ****
--- 408,487 ----
return result;
}
+ /*
+ * get_sortgrouplist_clauses
+ *
+ * Given a "grouped target" (i.e. target where each non-GroupedVar
+ * element must have sortgroupref set), build a list of the referencing
+ * SortGroupClauses, a list of the corresponding grouping expressions and
+ * a list of aggregate expressions.
+ */
+ /* Refine the function name. */
+ void
+ get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses, List **grouping_exprs,
+ List **agg_exprs)
+ {
+ ListCell *l;
+ int i = 0;
+
+ foreach(l, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(l);
+
+ /* The target should contain at least one grouping column. */
+ Assert(target->sortgrouprefs != NULL);
+
+ if (IsA(texpr, GroupedVar))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides aggregates, the target should contain no expressions w/o
+ * sortgroupref. Plain relation being joined to grouped can have
+ * sortgroupref equal to zero for expressions contained neither in
+ * grouping expression nor in aggregate arguments, but if the target
+ * contains such an expression, it shouldn't be used for aggregation
+ * --- see can_aggregate field of GroupedPathInfo.
+ */
+ Assert(sortgroupref > 0);
+
+ cl = get_sortgroupref_clause(sortgroupref, root->parse->groupClause);
+ *grouping_clauses = list_append_unique(*grouping_clauses, cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ *grouping_exprs = list_append_unique(*grouping_exprs, texpr);
+ }
+
+ /* Now collect the aggregates. */
+ while (l != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(l));
+
+ /* Currently, GroupedVarInfo can only represent aggregate. */
+ Assert(gvar->agg_partial != NULL);
+ *agg_exprs = lappend(*agg_exprs, gvar->agg_partial);
+ l = lnext(l);
+ }
+ }
+
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 857,1081 ----
}
/*
+ * Replace each "grouped var" in the source targetlist with the original
+ * expression.
+ *
+ * TODO Think of more suitable name. Although "grouped var" may substitute for
+ * grouping expressions in the future, currently Aggref is the only outcome of
+ * the replacement. undo_grouped_var_substitutions?
+ */
+ List *
+ restore_grouping_expressions(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te, *te_new;
+ Aggref *expr_new = NULL;
+
+ te = castNode(TargetEntry, lfirst(l));
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ expr_new = gvar->agg_partial;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
+ * For each aggregate add GroupedVar to target if "vars" is true, or the
+ * Aggref (marked as partial) if "vars" is false.
+ *
+ * If caller passes the aggregates, he must do so in the form of
+ * GroupedVarInfos so that we don't have to look for gvid. If NULL is passed,
+ * the function retrieves the suitable aggregates itself.
+ *
+ * List of the aggregates added is returned. This is only useful if the
+ * function had to retrieve the aggregates itself (i.e. NIL was passed for
+ * aggregates) -- caller is expected to do extra checks in that case (and to
+ * also free the list).
+ */
+ List *
+ add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel)
+ {
+ ListCell *lc;
+ GroupedVarInfo *gvi;
+
+ if (aggregates == NIL)
+ {
+ /* Caller should pass the aggregates for base relation. */
+ Assert(rel->reloptkind != RELOPT_BASEREL);
+
+ /* Collect all aggregates that this rel can evaluate. */
+ foreach(lc, root->grouped_var_list)
+ {
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * Overlap is not guarantee of correctness alone, but caller needs
+ * to do additional checks, so we're optimistic here.
+ *
+ * If gv_eval_at is NULL, the underlying Aggref should have
+ * aggstar set.
+ */
+ if (bms_overlap(gvi->gv_eval_at, rel->relids) ||
+ gvi->gv_eval_at == NULL)
+ aggregates = lappend(aggregates, gvi);
+ }
+
+ if (aggregates == NIL)
+ return NIL;
+ }
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, aggregates)
+ {
+ GroupedVar *gvar;
+
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_new_column_to_pathtarget(target, (Expr *) gvar);
+ }
+
+ return aggregates;
+ }
+
+ /*
+ * Return ressortgroupref of the target entry that is either equal to the
+ * expression or exists in the same equivalence class.
+ */
+ Index
+ get_expr_sortgroupref(PlannerInfo *root, Expr *expr)
+ {
+ ListCell *lc;
+ Index sortgroupref;
+
+ /*
+ * First, check if the query group clause contains exactly this
+ * expression.
+ */
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = castNode(TargetEntry, lfirst(lc));
+
+ if (equal(expr, te->expr) && te->ressortgroupref > 0)
+ return te->ressortgroupref;
+ }
+
+ /*
+ * If exactly this expression is not there, check if a grouping clause
+ * exists that belongs to the same equivalence class as the expression.
+ */
+ foreach(lc, root->group_pathkeys)
+ {
+ PathKey *pk = castNode(PathKey, lfirst(lc));
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *lm;
+ EquivalenceMember *em;
+ Expr *em_expr = NULL;
+ Query *query = root->parse;
+
+ /*
+ * Single-member EC cannot provide us with additional expression.
+ */
+ if (list_length(ec->ec_members) < 2)
+ continue;
+
+ /* We need equality anywhere in the join tree. */
+ if (ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * TODO Reconsider this restriction. As the grouping expression is
+ * only evaluated at the relation level (and only the result will be
+ * propagated to the final targetlist), volatile function might be
+ * o.k. Need to think what volatile EC exactly means.
+ */
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(lm, ec->ec_members)
+ {
+ em = (EquivalenceMember *) lfirst(lm);
+
+ /* The EC has !ec_below_outer_join. */
+ Assert(!em->em_nullable_relids);
+ if (equal(em->em_expr, expr))
+ {
+ em_expr = (Expr *) em->em_expr;
+ break;
+ }
+ }
+
+ if (em_expr == NULL)
+ /* Go for the next EC. */
+ continue;
+
+ /*
+ * Find the corresponding SortGroupClause, which provides us with
+ * sortgroupref. (It can belong to any EC member.)
+ */
+ sortgroupref = 0;
+ foreach(lm, ec->ec_members)
+ {
+ ListCell *lsg;
+
+ em = (EquivalenceMember *) lfirst(lm);
+ foreach(lsg, query->groupClause)
+ {
+ SortGroupClause *sgc;
+ Expr *expr;
+
+ sgc = (SortGroupClause *) lfirst(lsg);
+ expr = (Expr *) get_sortgroupclause_expr(sgc,
+ query->targetList);
+ if (equal(em->em_expr, expr))
+ {
+ Assert(sgc->tleSortGroupRef > 0);
+ sortgroupref = sgc->tleSortGroupRef;
+ break;
+ }
+ }
+
+ if (sortgroupref > 0)
+ break;
+ }
+
+ /*
+ * Since we searched in group_pathkeys, at least one EM of this EC
+ * should correspond to a SortGroupClause, otherwise the EC could
+ * not exist at all.
+ */
+ Assert(sortgroupref > 0);
+
+ return sortgroupref;
+ }
+
+ /* No EC found in group_pathkeys. */
+ return 0;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index 0c1a201..f4639c4
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7499,7504 ****
--- 7499,7512 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 8933,8942 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 8941,8958 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index 5c382a2..1dd2d73
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 113,118 ****
--- 113,119 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
*************** estimate_num_groups(PlannerInfo *root, L
*** 3473,3479 ****
/*
* Sanity check --- don't divide by zero if empty relation.
*/
! Assert(rel->reloptkind == RELOPT_BASEREL);
if (rel->tuples > 0)
{
/*
--- 3474,3481 ----
/*
* Sanity check --- don't divide by zero if empty relation.
*/
! Assert(rel->reloptkind == RELOPT_BASEREL ||
! rel->reloptkind == RELOPT_OTHER_MEMBER_REL);
if (rel->tuples > 0)
{
/*
*************** estimate_hash_bucketsize(PlannerInfo *ro
*** 3704,3709 ****
--- 3706,3744 ----
return (Selectivity) estfract;
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index 177853b..df98ef7
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 217,222 ****
--- 217,223 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_GroupedPathInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 257,266 ****
--- 258,269 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index ebf9480..90588d9
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 256,261 ****
--- 256,263 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 401,406 ****
--- 403,410 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 518,523 ****
--- 522,530 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to produce grouped paths. */
+ struct GroupedPathInfo *gpi;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 878,883 ****
--- 885,912 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * GroupedPathInfo
+ *
+ * If RelOptInfo points to this structure, grouped paths can be created for
+ * it.
+ *
+ * "target" will be used as pathtarget of grouped paths produced by this
+ * relation. Grouped path is either a result of aggregation of the relation
+ * that owns this structure or, if the owning relation is a join, a join path
+ * whose one side is a grouped path and the other is a plain (i.e. not
+ * grouped) one. (Two grouped paths cannot be joined in general because
+ * grouping of one side of the join essentially reduces occurrence of groups
+ * of the other side in the input of the final aggregation.)
+ */
+ typedef struct GroupedPathInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* output of grouped paths. */
+ List *pathlist; /* List of grouped paths. */
+ List *partial_pathlist; /* List of partial grouped paths. */
+ } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct PlaceHolderVar
*** 1806,1811 ****
--- 1835,1873 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. The reason is that aggregates having start as the argument can be
+ * evaluated at various places in the join tree (i.e. cannot be assigned to
+ * target list of exactly one relation). Also this concept seems to be less
+ * invasive than adding the grouped vars to reltarget (in which case
+ * attr_needed and attr_widths arrays of RelOptInfo) would also need
+ * additional changes.
+ *
+ * gvexpr is a pointer to gvexpr field of the corresponding instance
+ * GroupedVarInfo. It's there for the sake of exprType(), exprCollation(),
+ * etc.
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if the
+ * GroupedVar is in the target of a parent relation (RELOPT_BASEREL). However
+ * within a child relation's (RELOPT_OTHER_MEMBER_REL) target it points to a
+ * copy which has argument expressions translated, so they no longer reference
+ * the parent.
+ *
+ * XXX Currently we only create GroupedVar for aggregates, but sometime we can
+ * do it for grouping keys as well. That would allow grouping below the
+ * top-level join by keys other than plain Var.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PlaceHolderInfo
*** 2021,2026 ****
--- 2083,2104 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is
+ * the corresponding partial aggregate */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 82d4e87..91f0a57
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern int compare_path_costs(Path *path
*** 25,37 ****
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
--- 25,39 ----
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path,
! bool grouped);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys,
! bool grouped);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
*************** extern NestPath *create_nestloop_path(Pl
*** 125,131 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 127,134 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern MergePath *create_mergejoin_path(
*** 139,145 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 142,149 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern HashPath *create_hashjoin_path(Pl
*** 151,157 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
--- 155,162 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
*************** extern AggPath *create_agg_path(PlannerI
*** 192,197 ****
--- 197,216 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern ParamPathInfo *get_joinrel_paramp
*** 288,292 ****
--- 307,312 ----
List **restrict_clauses);
extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
+ extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 25fe78c..38967da
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void set_dummy_rel_pathlist(RelOp
*** 53,59 ****
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
--- 53,64 ----
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
!
! extern void create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
! Path *subpath, bool precheck, bool partial,
! AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
*************** extern void debug_print_rel(PlannerInfo
*** 67,73 ****
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
--- 72,79 ----
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index 94ef84b..159ccff
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** extern int join_collapse_limit;
*** 74,80 ****
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
--- 74,82 ----
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
! extern void add_grouping_info_to_base_rels(PlannerInfo *root);
! extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index ccb93d8..ddea03c
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern Node *get_sortgroupclause_expr(So
*** 41,46 ****
--- 41,49 ----
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
+ extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses,
+ List **grouping_exprs, List **agg_exprs);
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 68,84 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
+ extern List *add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel);
+ extern Index get_expr_sortgroupref(PlannerInfo *root, Expr *expr);
+ /* TODO Move definition from initsplan.c to tlist.c. */
+ extern PathTarget *create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs,
+ List *rel_agg_vars);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 9f9d2dc..e05e6f6
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern double estimate_num_groups(Planne
*** 206,211 ****
--- 206,214 ----
extern Selectivity estimate_hash_bucketsize(PlannerInfo *root, Node *hashkey,
double nbuckets);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
[text/plain] test_setup.sql (725B, ../../9666.1491295317@localhost/3-test_setup.sql)
download | inline:
BEGIN;
CREATE TABLE a (
i int primary key);
CREATE TABLE b (
j int primary key,
parent int references a,
v double precision);
CREATE TABLE c (
k int primary key,
parent int references a,
v double precision);
INSERT INTO a(i)
SELECT n
FROM generate_series(0, 255) AS s(n);
INSERT INTO b(j, parent, v)
SELECT 1024 * i + n, i, random()
FROM generate_series(0, 1023) AS s(n), a;
INSERT INTO c(k, parent, v)
SELECT 1024 * i + n, i, random()
FROM generate_series(0, 1023) AS s(n), a;
CREATE OR REPLACE FUNCTION expensive(x double precision)
RETURNS double precision
LANGUAGE sql
AS $$
-- CTE is there to avoid inlining.
WITH tmp AS (SELECT x) SELECT x FROM tmp;
$$
PARALLEL SAFE
COST 10000;
COMMIT;
ANALYZE;
[text/plain] query.sql (373B, ../../9666.1491295317@localhost/4-query.sql)
download | inline:
SET seq_page_cost TO 0;
SET cpu_operator_cost TO 1;
SET min_parallel_table_scan_size TO 0;
SET max_parallel_workers_per_gather TO 4;
EXPLAIN (COSTS false)
SELECT a.i, avg(expensive(b.v + c.v))
FROM a
JOIN
b ON b.parent = a.i
JOIN
c ON c.parent = a.i
WHERE b.j = c.k
GROUP BY a.i;
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-04-28 08:46 Antonin Houska <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2017-04-28 08:46 UTC (permalink / raw)
To: pgsql-hackers
Antonin Houska <[email protected]> wrote:
> This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7b
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[text/x-diff] agg_pushdown_v2.diff (206.9K, ../../8160.1493369169@localhost/2-agg_pushdown_v2.diff)
download | inline diff:
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index 5a34a46..717763d
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, PlanState *p
*** 723,728 ****
--- 723,755 ----
break;
}
+ case T_GroupedVar:
+ /*
+ * GroupedVar is treated as an aggregate if it appears in the
+ * targetlist of Agg node, but as a normal variable elsewhere.
+ */
+ if (parent && (IsA(parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * Currently GroupedVar can only represent partial aggregate.
+ */
+ Assert(gvar->agg_partial != NULL);
+
+ ExecInitExprRec((Expr *) gvar->agg_partial, parent, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ */
+ elog(ERROR, "parent of GroupedVar is not Agg node");
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
new file mode 100644
index c2b8618..c4cb4c0
*** a/src/backend/executor/nodeAgg.c
--- b/src/backend/executor/nodeAgg.c
*************** find_unaggregated_cols_walker(Node *node
*** 1829,1834 ****
--- 1829,1845 ----
/* do not descend into aggregate exprs */
return false;
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * GroupedVar is currently used only for partial aggregation, so treat
+ * it like an Aggref above.
+ */
+ Assert(gvar->agg_partial != NULL);
+ return false;
+ }
return expression_tree_walker(node, find_unaggregated_cols_walker,
(void *) colnos);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 00a0fed..7d188ea
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2206,2211 ****
--- 2206,2226 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** copyObjectImpl(const void *from)
*** 4984,4989 ****
--- 4999,5007 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index 46573ae..f1dacd5
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 874,879 ****
--- 874,887 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3148,3153 ****
--- 3156,3164 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index 3e8189c..5c00e55
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 259,264 ****
--- 259,267 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprCollation(const Node *expr)
*** 931,936 ****
--- 934,942 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2198,2203 ****
--- 2204,2211 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2989,2994 ****
--- 2997,3012 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 28cef85..4b6ee30
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2186,2191 ****
--- 2186,2192 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outParamPathInfo(StringInfo str, const
*** 2408,2413 ****
--- 2409,2424 ----
}
static void
+ _outGroupedPathInfo(StringInfo str, const GroupedPathInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDPATHINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(pathlist);
+ WRITE_NODE_FIELD(partial_pathlist);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2451,2456 ****
--- 2462,2477 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** outNode(StringInfo str, const void *obj)
*** 3996,4007 ****
--- 4017,4034 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_GroupedPathInfo:
+ _outGroupedPathInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index a883220..138f71c
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 522,527 ****
--- 522,542 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** parseNodeString(void)
*** 2440,2445 ****
--- 2455,2462 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index b5cab0c..f89406d
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 265,271 ****
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
--- 265,271 ----
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index b93b4fc..ad14578
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 486,492 ****
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! generate_gather_paths(root, rel);
/*
* Allow a plugin to editorialize on the set of Paths for this base
--- 486,495 ----
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! {
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
! }
/*
* Allow a plugin to editorialize on the set of Paths for this base
*************** static void
*** 686,691 ****
--- 689,695 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 694,708 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel);
/* Consider TID scans */
create_tidscan_paths(root, rel);
--- 698,725 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! add_path(rel, seq_path, false);
! if (rel->gpi != NULL && required_outer == NULL)
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel, false);
! if (rel->gpi != NULL)
! {
! /*
! * TODO Instead of calling the whole clause-matching machinery twice
! * (there should be no difference between plain and grouped paths from
! * this point of view), consider returning a separate list of paths
! * usable as grouped ones.
! */
! create_index_paths(root, rel, true);
! }
/* Consider TID scans */
create_tidscan_paths(root, rel);
*************** static void
*** 716,721 ****
--- 733,739 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
parallel_workers = compute_parallel_worker(rel, rel->pages, -1);
*************** create_plain_partial_paths(PlannerInfo *
*** 724,730 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 742,849 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
! add_partial_path(rel, path, false);
!
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it.
! */
! if (rel->gpi != NULL)
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the
! * appropriate pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck().
! *
! * If "partial" is true, the resulting path is considered partial in terms of
! * parallel execution.
! *
! * The path we create here shouldn't be parameterized because of supposedly
! * high startup cost of aggregation (whether due to build of hash table for
! * AGG_HASHED strategy or due to explicit sort for AGG_SORTED).
! *
! * XXX IndexPath as an input for AGG_SORTED might seem to be an exception, but
! * aggregation of its output is only beneficial if it's performed by multiple
! * workers, i.e. the resulting path is partial (Besides parallel aggregation,
! * the other use case of aggregation push-down is aggregation performed on
! * remote database, but that has nothing to do with IndexScan). And partial
! * path cannot be parameterized because it's semantically wrong to use it on
! * the inner side of NL join.
! */
! void
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! List *group_clauses = NIL;
! List *group_exprs = NIL;
! List *agg_exprs = NIL;
! Path *agg_path;
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Grouped path should never be parameterized, so we're not supposed to
! * receive parameterized subpath.
! */
! Assert(subpath->param_info == NULL);
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL,
! true))
! return;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost, pathkeys,
! true))
! return;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path, true);
! else
! add_partial_path(rel, (Path *) agg_path, true);
! }
}
/*
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 810,816 ****
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path);
/* For the moment, at least, there are no other paths to consider */
}
--- 929,935 ----
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path, false);
/* For the moment, at least, there are no other paths to consider */
}
*************** set_append_rel_size(PlannerInfo *root, R
*** 1067,1072 ****
--- 1186,1234 ----
appinfo);
/*
+ * If grouping is applicable to the parent relation, it should be
+ * applicable to the children too. Make sure the child rel has valid
+ * sortgrouprefs.
+ *
+ * TODO Consider if this is really needed --- child rel is not joined
+ * to grouped rel itself, so it might not participate on creation of
+ * the grouped path target that upper joins will see.
+ */
+ if (rel->reltarget->sortgrouprefs)
+ {
+ Assert(childrel->reltarget->sortgrouprefs == NULL);
+ childrel->reltarget->sortgrouprefs = rel->reltarget->sortgrouprefs;
+ }
+
+ /*
+ * Also the grouped target needs to be adjusted, if one exists.
+ */
+ if (rel->gpi != NULL)
+ {
+ PathTarget *target = rel->gpi->target;
+
+ Assert(target->sortgrouprefs != NULL);
+
+ Assert(childrel->gpi == NULL);
+ childrel->gpi = makeNode(GroupedPathInfo);
+ memcpy(childrel->gpi, rel->gpi, sizeof(GroupedPathInfo));
+
+ /*
+ * add_grouping_info_to_base_rels was not sure if grouping makes
+ * sense for the parent rel, so create a separate copy of the
+ * target now.
+ */
+ childrel->gpi->target = copy_pathtarget(childrel->gpi->target);
+
+ /* Translate vars of the grouping target. */
+ Assert(childrel->gpi->target->exprs != NIL);
+ childrel->gpi->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) childrel->gpi->target->exprs,
+ appinfo);
+ }
+
+ /*
* We have to make child entries in the EquivalenceClass data
* structures as well. This is needed either if the parent
* participates in some eclass joins (because we will want to consider
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1281,1286 ****
--- 1443,1450 ----
bool subpaths_valid = true;
List *partial_subpaths = NIL;
bool partial_subpaths_valid = true;
+ List *grouped_subpaths = NIL;
+ bool grouped_subpaths_valid = true;
List *all_child_pathkeys = NIL;
List *all_child_outers = NIL;
ListCell *l;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1324,1329 ****
--- 1488,1516 ----
partial_subpaths_valid = false;
/*
+ * For grouped paths, use only the unparameterized subpaths.
+ *
+ * XXX Consider if the parameterized subpaths should be processed
+ * below. It's probably not useful for sequential scans (due to
+ * repeated aggregation), but might be worthwhile for other child
+ * nodes.
+ */
+ if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
+ {
+ Path *path;
+
+ path = (Path *) linitial(childrel->gpi->pathlist);
+ if (path->param_info == NULL)
+ grouped_subpaths = accumulate_append_subpath(grouped_subpaths,
+ path);
+ else
+ grouped_subpaths_valid = false;
+ }
+ else
+ grouped_subpaths_valid = false;
+
+
+ /*
* Collect lists of all the available path orderings and
* parameterizations for all the children. We use these as a
* heuristic to indicate which sort orderings and parameterizations we
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1395,1401 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels));
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1582,1589 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels),
! false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1422,1429 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1610,1630 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers,
! partitioned_rels);
! add_partial_path(rel, (Path *) appendpath, false);
! }
!
! /* TODO Also partial grouped paths? */
! if (grouped_subpaths_valid)
! {
! Path *path;
!
! path = (Path *) create_append_path(rel, grouped_subpaths, NULL, 0,
! partitioned_rels);
! /* pathtarget will produce the grouped relation.. */
! path->pathtarget = rel->gpi->target;
! add_path(rel, path, true);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1476,1482 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels));
}
}
--- 1677,1684 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels),
! false);
}
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1572,1585 ****
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels));
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels));
}
}
--- 1774,1789 ----
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
}
}
*************** set_dummy_rel_pathlist(RelOptInfo *rel)
*** 1712,1718 ****
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
--- 1916,1922 ----
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
*************** set_subquery_pathlist(PlannerInfo *root,
*** 1926,1932 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer));
}
}
--- 2130,2136 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer), false);
}
}
*************** set_function_pathlist(PlannerInfo *root,
*** 1995,2001 ****
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer));
}
/*
--- 2199,2205 ----
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer), false);
}
/*
*************** set_values_pathlist(PlannerInfo *root, R
*** 2015,2021 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer));
}
/*
--- 2219,2225 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer), false);
}
/*
*************** set_tablefunc_pathlist(PlannerInfo *root
*** 2036,2042 ****
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer));
}
/*
--- 2240,2246 ----
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer), false);
}
/*
*************** set_cte_pathlist(PlannerInfo *root, RelO
*** 2102,2108 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer));
}
/*
--- 2306,2312 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer), false);
}
/*
*************** set_namedtuplestore_pathlist(PlannerInfo
*** 2129,2135 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
--- 2333,2340 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
*************** set_worktable_pathlist(PlannerInfo *root
*** 2182,2188 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer));
}
/*
--- 2387,2394 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer),
! false);
}
/*
*************** set_worktable_pathlist(PlannerInfo *root
*** 2195,2208 ****
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
ListCell *lc;
/* If there are no partial paths, there's nothing to do here. */
! if (rel->partial_pathlist == NIL)
return;
/*
--- 2401,2421 ----
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
+ List *pathlist = NIL;
+ PathTarget *partial_target;
ListCell *lc;
+ if (!grouped)
+ pathlist = rel->partial_pathlist;
+ else if (rel->gpi != NULL)
+ pathlist = rel->gpi->partial_pathlist;
+
/* If there are no partial paths, there's nothing to do here. */
! if (pathlist == NIL)
return;
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2210,2226 ****
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(rel->partial_pathlist);
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, rel->partial_pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
--- 2423,2445 ----
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(pathlist);
!
! if (!grouped)
! partial_target = rel->reltarget;
! else if (rel->gpi != NULL)
! partial_target = rel->gpi->target;
!
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, partial_target,
NULL, NULL);
! add_path(rel, simple_gather_path, grouped);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
*************** generate_gather_paths(PlannerInfo *root,
*** 2228,2236 ****
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path);
}
}
--- 2447,2455 ----
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, partial_target,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, grouped);
}
}
*************** standard_join_search(PlannerInfo *root,
*** 2396,2402 ****
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
--- 2615,2622 ----
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
*************** create_partial_bitmap_paths(PlannerInfo
*** 3047,3053 ****
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3267,3273 ----
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers), false);
}
/*
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 6e4bae8..a6fa713
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** static bool eclass_already_used(Equivale
*** 107,119 ****
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
--- 108,121 ----
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! bool grouped);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
*************** static Const *string_to_const(const char
*** 229,235 ****
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel)
{
List *indexpaths;
List *bitindexpaths;
--- 231,237 ----
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
List *indexpaths;
List *bitindexpaths;
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,283 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
! grouped);
/*
* Identify the join clauses that can match the index. For the moment
*************** create_index_paths(PlannerInfo *root, Re
*** 338,344 ****
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
--- 340,346 ----
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath, false);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
*************** create_index_paths(PlannerInfo *root, Re
*** 415,421 ****
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath);
}
}
}
--- 417,423 ----
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath, false);
}
}
}
*************** get_join_index_paths(PlannerInfo *root,
*** 667,673 ****
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
--- 669,675 ----
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths, false);
/*
* Remember we considered paths for this set of relids. We use lcons not
*************** bms_equal_any(Relids relids, List *relid
*** 736,742 ****
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths)
{
List *indexpaths;
bool skip_nonnative_saop = false;
--- 738,744 ----
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped)
{
List *indexpaths;
bool skip_nonnative_saop = false;
*************** get_index_paths(PlannerInfo *root, RelOp
*** 754,760 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
--- 756,762 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop, grouped);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
*************** get_index_paths(PlannerInfo *root, RelOp
*** 769,775 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL));
}
/*
--- 771,777 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL, grouped));
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,797 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath);
! if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
--- 791,799 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, grouped);
! if (!grouped && index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
*************** get_index_paths(PlannerInfo *root, RelOp
*** 802,815 ****
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
--- 804,818 ----
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (!grouped && skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 861,867 ****
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop)
{
List *result = NIL;
IndexPath *ipath;
--- 864,870 ----
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop, bool grouped)
{
List *result = NIL;
IndexPath *ipath;
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 881,890 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool can_agg_sorted;
+ List *group_clauses, *group_exprs, *agg_exprs;
+ AggPath *agg_path;
+ double agg_input_rows;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 891,896 ****
--- 898,906 ----
case ST_BITMAPSCAN:
if (!index->amhasgetbitmap)
return NIL;
+
+ if (grouped)
+ return NIL;
break;
case ST_ANYSCAN:
/* either or both are OK */
*************** build_index_paths(PlannerInfo *root, Rel
*** 1032,1037 ****
--- 1042,1051 ----
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
*/
+ can_agg_sorted = true;
+ group_clauses = NIL;
+ group_exprs = NIL;
+ agg_exprs = NIL;
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1062,1086 ----
outer_relids,
loop_count,
false);
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root, (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1109,1139 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1161,1185 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else if (can_agg_sorted)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! Assert(agg_path != NULL);
! result = lappend(result, agg_path);
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1203,1227 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1244,1250 ****
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL);
result = list_concat(result, indexpaths);
}
--- 1336,1343 ----
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 5aedcd1..12f0356
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***************
*** 22,27 ****
--- 22,28 ----
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
+ #include "optimizer/tlist.h"
/* Hook for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
*************** static void try_partial_mergejoin_path(P
*** 38,66 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
--- 39,87 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static void sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static bool is_grouped_join_target_complete(PlannerInfo *root,
! PathTarget *jointarget,
! Path *outer_path,
! Path *inner_path);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
*************** static void generate_mergejoin_paths(Pla
*** 77,83 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 98,107 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
/*
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 227,234 ****
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 2. Consider paths where the outer relation need not be explicitly
--- 251,262 ----
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
+ {
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 2. Consider paths where the outer relation need not be explicitly
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 238,245 ****
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
#ifdef NOT_USED
--- 266,277 ----
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
+ {
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
#ifdef NOT_USED
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 265,272 ****
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 297,308 ----
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
+ {
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 330,339 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
--- 366,385 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_nestloop_path(PlannerInfo *root,
*** 341,359 ****
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! ((!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path))))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 387,409 ----
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
+ *
+ * Grouped path should never be parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped ||
! (!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path)))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_nestloop_path(PlannerInfo *root,
*** 368,388 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer));
}
else
{
--- 418,453 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! required_outer, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate && required_outer == NULL)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, join_path, grouped);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 403,411 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 468,484 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_nestloop_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 428,448 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL));
}
/*
--- 501,581 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! NULL, join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_nestloop_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_nestloop_path(root, joinrel, outer_path, inner_path, pathkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_nestloop_path(root, joinrel, outer_path, inner_path,
! pathkeys, jointype, extra, true,
! do_aggregate);
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 461,470 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
if (is_partial)
{
--- 594,613 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
if (is_partial)
{
*************** try_mergejoin_path(PlannerInfo *root,
*** 477,498 ****
outersortkeys,
innersortkeys,
jointype,
! extra);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 620,644 ----
outersortkeys,
innersortkeys,
jointype,
! extra,
! grouped,
! do_aggregate);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 511,537 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
else
{
--- 657,704 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 555,563 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 722,738 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_mergejoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 587,613 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
/*
--- 762,934 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_mergejoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_mergejoin_path(root, joinrel, outer_path, inner_path, pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, false, true, do_aggregate);
! else
! try_partial_mergejoin_path(root, joinrel, outer_path, inner_path,
! pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! }
!
! static void
! try_mergejoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false, false);
! }
! else if (grouped_outer || grouped_inner)
! {
! Assert(!do_aggregate);
!
! /*
! * Exactly one of the input paths is grouped, so create a grouped join
! * path.
! */
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false);
! }
! /* Preform explicit aggregation only if suitable target exists. */
! else if (joinrel->gpi != NULL)
! {
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial, true);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 622,668 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! required_outer,
! hashclauses));
}
else
{
--- 943,1017 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
+ *
+ * TODO Need to consider aggregation here?
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist,
! required_outer, hashclauses,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 683,691 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1032,1048 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_hashjoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 708,728 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! NULL,
! hashclauses));
}
/*
--- 1065,1160 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist, NULL,
! hashclauses, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! NIL, grouped))
! {
! add_partial_path(joinrel, (Path *) join_path , grouped);
! }
! }
!
! /*
! * Create a new grouped hash join path by joining a grouped path to plain
! * (non-grouped) one, or by joining 2 plain relations and applying grouping on
! * the result.
! *
! * Joining of 2 grouped paths is not supported. If a grouped relation A was
! * joined to grouped relation B, then the grouping of B reduces the number of
! * times each group of A is appears in the join output. This makes difference
! * for some aggregates, e.g. sum().
! *
! * If do_aggregate is true, neither input rel is grouped so we need to
! * aggregate the join result explicitly.
! *
! * partial argument tells whether the join path should be considered partial.
! */
! static void
! try_grouped_hashjoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_hashjoin_path(root, joinrel, outer_path, inner_path, hashclauses,
! jointype, extra, true, do_aggregate);
! else
! try_partial_hashjoin_path(root, joinrel, outer_path, inner_path,
! hashclauses, jointype, extra, true,
! do_aggregate);
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 773,779 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
Path *outer_path;
--- 1205,1244 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
! {
! if (!grouped)
! {
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, false);
! }
! else
! {
! /* Use all the supported strategies to generate grouped join. */
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, true, false, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, true, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, true);
! }
! }
!
! /*
! * TODO As merge_pathkeys shouldn't differ across execution, use a separate
! * function to derive them and pass them here in a list.
! */
! static void
! sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
Path *outer_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 782,787 ****
--- 1247,1253 ----
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped_result;
/*
* We only consider the cheapest-total-cost input paths, since we are
*************** sort_inner_and_outer(PlannerInfo *root,
*** 796,803 ****
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! outer_path = outerrel->cheapest_total_path;
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 1262,1288 ----
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->pathlist != NIL)
! outer_path = linitial(outerrel->gpi->pathlist);
! else
! return;
! }
! else
! outer_path = outerrel->cheapest_total_path;
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
!
! grouped_result = grouped_outer || grouped_inner || do_aggregate;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** sort_inner_and_outer(PlannerInfo *root,
*** 843,855 ****
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! cheapest_partial_outer = (Path *) linitial(outerrel->partial_pathlist);
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
/*
--- 1328,1377 ----
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->partial_pathlist != NIL)
! cheapest_partial_outer = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
! else
! return;
! }
! else
! cheapest_partial_outer = (Path *)
! linitial(outerrel->partial_pathlist);
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else
+ {
+ Assert(innerrel->gpi != NULL);
+ inner_pathlist = innerrel->gpi->pathlist;
+ }
+
+ /*
+ * All the grouped paths should be unparameterized, so the
+ * function is overly stringent in the grouped_inner case, but
+ * still useful.
+ */
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 925,957 ****
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra);
}
}
--- 1447,1505 ----
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! if (!grouped_result)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false, false, false);
! else
! {
! try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! {
! if (!grouped_result)
! {
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra, false, false);
! }
! else
! {
! try_mergejoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! true,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
! }
}
}
*************** sort_inner_and_outer(PlannerInfo *root,
*** 968,973 ****
--- 1516,1529 ----
* some sort key requirements). So, we consider truncations of the
* mergeclause list as well as the full list. (Ideally we'd consider all
* subsets of the mergeclause list, but that seems way too expensive.)
+ *
+ * grouped_outer - is outerpath grouped?
+ * grouped_inner - use grouped paths of innerrel?
+ * do_aggregate - apply (partial) aggregation to the output?
+ *
+ * TODO If subsequent calls often differ only by the 3 arguments above,
+ * consider a workspace structure to share useful info (eg merge clauses)
+ * across calls.
*/
static void
generate_mergejoin_paths(PlannerInfo *root,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 979,985 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1535,1544 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1030,1046 ****
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1589,1606 ----
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner, do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1096,1111 ****
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
List *newclauses = NIL;
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
--- 1656,1677 ----
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
+ List *inner_pathlist = NIL;
Path *innerpath;
List *newclauses = NIL;
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else if (innerrel->gpi != NULL)
+ inner_pathlist = innerrel->gpi->pathlist;
+
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1128,1148 ****
}
else
newclauses = mergeclauses;
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
--- 1694,1718 ----
}
else
newclauses = mergeclauses;
!
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
!
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1173,1189 ****
else
newclauses = mergeclauses;
}
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1743,1761 ----
else
newclauses = mergeclauses;
}
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1218,1223 ****
--- 1790,1797 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'grouped' indicates that the at least one relation in the join has been
+ * aggregated.
*/
static void
match_unsorted_outer(PlannerInfo *root,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1225,1231 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
--- 1799,1806 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1235,1240 ****
--- 1810,1837 ----
ListCell *lc1;
/*
+ * If grouped join path is requested, we ignore cases where either input
+ * path needs to be unique. For each side we should expect either grouped
+ * or plain relation, which differ quite a bit.
+ *
+ * XXX Although unique-ification of grouped path might result in too
+ * expensive input path (note that grouped input relation is not
+ * necessarily unique, regardless the grouping keys --- one or more plain
+ * relation could already have been joined to it), we might want to
+ * unique-ify the input relation in the future at least in the case it's a
+ * plain relation.
+ *
+ * (Materialization is not involved in grouped paths for similar reasons.)
+ */
+ if (grouped &&
+ (jointype == JOIN_UNIQUE_OUTER || jointype == JOIN_UNIQUE_INNER))
+ return;
+
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
+ /*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
* are doing a right or full mergejoin, we must use *all* the mergeclauses
* as join clauses, else we will not have a valid plan. (Although these
*************** match_unsorted_outer(PlannerInfo *root,
*** 1290,1296 ****
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK)
{
/*
* Consider materializing the cheapest inner path, unless
--- 1887,1893 ----
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK && !grouped)
{
/*
* Consider materializing the cheapest inner path, unless
*************** match_unsorted_outer(PlannerInfo *root,
*** 1321,1326 ****
--- 1918,1925 ----
*/
if (save_jointype == JOIN_UNIQUE_OUTER)
{
+ Assert(!grouped);
+
if (outerpath != outerrel->cheapest_total_path)
continue;
outerpath = (Path *) create_unique_path(root, outerrel,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1348,1354 ****
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra);
}
else if (nestjoinOK)
{
--- 1947,1954 ----
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1364,1387 ****
{
Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra);
}
! /* Also consider materialized form of the cheapest inner path */
! if (matpath != NULL)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 1964,2009 ----
{
Path *innerpath = (Path *) lfirst(lc2);
! if (!grouped)
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra, false, false);
! else
! {
! /*
! * Since both input paths are plain, request explicit
! * aggregation.
! */
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! true,
! false);
! }
}
! /*
! * Also consider materialized form of the cheapest inner path.
! *
! * (There's no matpath for grouped join.)
! */
! if (matpath != NULL && !grouped)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1396,1402 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 2018,2094 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, false, false, grouped);
!
! /* Try to join the plain outer relation to grouped inner. */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER &&
! innerrel->gpi != NULL && outerrel->gpi == NULL)
! {
! Path *inner_cheapest_grouped = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (PATH_PARAM_BY_REL(inner_cheapest_grouped, outerrel))
! continue;
!
! /* XXX Shouldn't Assert() be used here instead? */
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! /*
! * Only outer grouped path is interesting in this case: grouped
! * path on the inner side of NL join would imply repeated
! * aggregation somewhere in the inner path.
! */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_grouped, merge_pathkeys,
! false, false, true, false);
! }
! }
!
! /*
! * Combine grouped outer and plain inner paths.
! */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER)
! {
! /*
! * If the inner rel had a grouped target, its plain paths should be
! * ignored. Otherwise we could create grouped paths with different
! * targets.
! */
! if (outerrel->gpi != NULL && innerrel->gpi == NULL &&
! inner_cheapest_total != NULL)
! {
! /* Nested loop paths. */
! foreach(lc1, outerrel->gpi->pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
!
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra,
! false,
! false);
!
! /* Merge join paths. */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_total, merge_pathkeys,
! false, true, false, false);
! }
! }
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1416,1423 ****
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 2108,2128 ----
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! {
! if (!grouped)
! /* Plain partial paths. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, false, false);
! else
! {
! /* Grouped partial paths with explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, true);
! /* Grouped partial paths w/o explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, false);
! }
! }
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1437,1443 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
--- 2142,2148 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total, grouped);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1460,1469 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
--- 2165,2183 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped)
{
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (jointype == JOIN_UNIQUE_OUTER ||
+ jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
*************** consider_parallel_mergejoin(PlannerInfo
*** 1476,1484 ****
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
! extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 2190,2245 ----
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! if (!grouped)
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true,
! false, false, false);
! else
! {
! /*
! * Create grouped join by joining plain rels and aggregating the
! * result.
! */
! Assert(joinrel->gpi != NULL);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, false, false, true);
!
! /* Combine the plain outer with grouped inner one(s). */
! if (outerrel->gpi == NULL && innerrel->gpi != NULL)
! {
! Path *inner_cheapest_grouped = (Path *)
! linitial(innerrel->gpi->pathlist);
!
! if (inner_cheapest_grouped != NULL &&
! inner_cheapest_grouped->parallel_safe)
! generate_mergejoin_paths(root, joinrel, innerrel,
! outerpath, jointype, extra,
! false, inner_cheapest_grouped,
! merge_pathkeys,
! true, false, true, false);
! }
! }
! }
!
! /* In addition, try to join grouped outer to plain inner one(s). */
! if (grouped && outerrel->gpi != NULL && innerrel->gpi == NULL)
! {
! foreach(lc1, outerrel->gpi->partial_pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys;
!
! merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, true, false, false);
! }
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1499,1513 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
--- 2260,2304 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate)
{
JoinType save_jointype = jointype;
+ List *outer_pathlist;
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (save_jointype == JOIN_UNIQUE_OUTER ||
+ save_jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! if (!grouped || do_aggregate)
! {
! /*
! * If creating grouped paths by explicit aggregation, the input paths
! * must be plain.
! */
! outer_pathlist = outerrel->partial_pathlist;
! }
! else if (outerrel->gpi != NULL)
! {
! /*
! * Only the outer paths are accepted as grouped when we try to combine
! * grouped and plain ones. Grouped inner path implies repeated
! * aggregation, which doesn't sound as a good idea.
! */
! outer_pathlist = outerrel->gpi->partial_pathlist;
! }
! else
! return;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1538,1544 ****
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
--- 2329,2335 ----
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER && !grouped)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1548,1555 ****
Assert(innerpath);
}
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
--- 2339,2364 ----
Assert(innerpath);
}
! if (!grouped)
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, false);
! else if (do_aggregate)
! {
! /* Request aggregation as both input rels are plain. */
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! true, true);
! }
! /*
! * Only combine the grouped outer path with the plain inner if the
! * inner relation cannot produce grouped paths. Otherwise we could
! * generate grouped paths with different targets.
! */
! else if (innerrel->gpi == NULL)
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, true);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1571,1583 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
--- 2380,2397 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1627,1632 ****
--- 2441,2449 ----
* can't use a hashjoin. (There's no use looking for alternative
* input paths, since these should already be the least-parameterized
* available paths.)
+ *
+ * (The same check should work for grouped paths, as these don't
+ * differ in parameterization.)
*/
if (PATH_PARAM_BY_REL(cheapest_total_outer, innerrel) ||
PATH_PARAM_BY_REL(cheapest_total_inner, outerrel))
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1646,1652 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
--- 2463,2470 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1662,1668 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
--- 2480,2487 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1671,1733 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
}
else
{
! /*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized ones.
! * There is no use in generating parameterized paths on the basis
! * of possibly cheap startup cost, so this is sufficient.
! */
! ListCell *lc1;
! ListCell *lc2;
!
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
- Path *outerpath = (Path *) lfirst(lc1);
-
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
! Path *innerpath = (Path *) lfirst(lc2);
/*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
*/
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
continue;
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra);
}
}
}
--- 2490,2643 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
}
else
{
! if (!grouped)
{
/*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized
! * ones. There is no use in generating parameterized paths on
! * the basis of possibly cheap startup cost, so this is
! * sufficient.
*/
! ListCell *lc1;
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false, false);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
! Path *outerpath = (Path *) lfirst(lc1);
! ListCell *lc2;
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
! /*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
! */
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
! continue;
!
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
!
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra,
! false, false);
! }
! }
! }
! else
! {
! /* Create grouped paths if possible. */
! /*
! * TODO
! *
! * Consider processing JOIN_UNIQUE_INNER and JOIN_UNIQUE_OUTER
! * join types, ie perform grouping of the inner / outer rel if
! * it's not unique yet and if the grouping is legal.
! */
! if (jointype == JOIN_UNIQUE_OUTER ||
! jointype == JOIN_UNIQUE_INNER)
! return;
!
! /*
! * Join grouped relation to non-grouped one.
! *
! * Do not use plain path of the input rel whose target does
! * have GroupedPahtInfo. For example (assuming that join of
! * two grouped rels is not supported), the only way to
! * evaluate SELECT sum(a.x), sum(b.y) ... is to join "a" and
! * "b" and aggregate the result. Otherwise the path target
! * wouldn't match joinrel->gpi->target. TODO Move this comment
! * elsewhere as it seems common to all join kinds.
! */
! /*
! * TODO Allow outer join if the grouped rel is on the
! * non-nullable side.
! */
! if (jointype == JOIN_INNER)
! {
! Path *grouped_path, *plain_path;
!
! if (outerrel->gpi != NULL &&
! outerrel->gpi->pathlist != NIL &&
! innerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(outerrel->gpi->pathlist);
! plain_path = cheapest_total_inner;
! try_grouped_hashjoin_path(root, joinrel,
! grouped_path, plain_path,
! hashclauses, jointype,
! extra, false, false);
! }
! else if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(innerrel->gpi->pathlist);
! plain_path = cheapest_total_outer;
! try_grouped_hashjoin_path(root, joinrel, plain_path,
! grouped_path, hashclauses,
! jointype, extra,
! false, false);
!
! if (cheapest_startup_outer != NULL &&
! cheapest_startup_outer != cheapest_total_outer)
! {
! plain_path = cheapest_startup_outer;
! try_grouped_hashjoin_path(root, joinrel,
! plain_path,
! grouped_path,
! hashclauses,
! jointype, extra,
! false, false);
! }
! }
}
+
+ /*
+ * Try to join plain relations and make a grouped rel out of
+ * the join.
+ *
+ * Since aggregation needs the whole relation, we are only
+ * interested in total costs.
+ */
+ try_grouped_hashjoin_path(root, joinrel,
+ cheapest_total_outer,
+ cheapest_total_inner,
+ hashclauses,
+ jointype, extra, true, false);
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1765,1777 ****
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra);
}
}
}
/*
--- 2675,2898 ----
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (!grouped)
! {
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra,
! false, false);
! }
! else if (joinrel->gpi != NULL)
! {
! /*
! * Grouped partial path.
! *
! * 1. Apply aggregation to the plain partial join path.
! */
! if (cheapest_safe_inner != NULL)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, true, true);
!
! /*
! * 2. Join the cheapest partial grouped outer path (if one
! * exists) to cheapest_safe_inner (there's no reason to look
! * for another inner path than what we used for non-grouped
! * partial join path).
! */
! if (outerrel->gpi != NULL &&
! outerrel->gpi->partial_pathlist != NIL &&
! innerrel->gpi == NULL &&
! cheapest_safe_inner != NULL)
! {
! Path *outer_path;
!
! outer_path = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
!
! try_grouped_hashjoin_path(root, joinrel, outer_path,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, false, true);
! }
!
! /*
! * 3. Join the cheapest_partial_outer path (again, no reason
! * to use different outer path than the one we used for plain
! * partial join) to the cheapest grouped inner path if the
! * latter exists and is parallel-safe.
! */
! if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! Path *inner_path;
!
! inner_path = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (inner_path->parallel_safe)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! inner_path,
! hashclauses,
! jointype, extra,
! false, true);
! }
!
! /*
! * Other combinations seem impossible because: 1. At most 1
! * input relation of the join can be grouped, 2. the inner
! * path must not be partial.
! */
! }
! }
! }
! }
!
! /*
! * Do the input paths emit all the aggregates contained in the grouped target
! * of the join?
! *
! * The point is that one input relation might be unable to evaluate some
! * aggregate(s), so it'll only generate plain paths. It's wrong to combine
! * such plain paths with grouped ones that the other input rel might be able
! * to generate because the result would miss the aggregate(s) the first
! * relation failed to evaluate.
! *
! * TODO For better efficiency, consider storing Bitmapset of
! * GroupedVarInfo.gvid in GroupedPathInfo.
! */
! static bool
! is_grouped_join_target_complete(PlannerInfo *root, PathTarget *jointarget,
! Path *outer_path, Path *inner_path)
! {
! RelOptInfo *outer_rel = outer_path->parent;
! RelOptInfo *inner_rel = inner_path->parent;
! ListCell *l1;
!
! /*
! * Join of two grouped relations is not supported.
! *
! * This actually isn't check of target completeness --- can it be located
! * elsewhere?
! */
! if (outer_rel->gpi != NULL && inner_rel->gpi != NULL)
! return false;
!
! foreach(l1, jointarget->exprs)
! {
! Expr *expr = (Expr *) lfirst(l1);
! GroupedVar *gvar;
! GroupedVarInfo *gvi = NULL;
! ListCell *l2;
! bool found = false;
!
! /* Only interested in aggregates. */
! if (!IsA(expr, GroupedVar))
! continue;
!
! gvar = castNode(GroupedVar, expr);
!
! /* Find the corresponding GroupedVarInfo. */
! foreach(l2, root->grouped_var_list)
! {
! GroupedVarInfo *gvi_tmp = castNode(GroupedVarInfo, lfirst(l2));
!
! if (gvi_tmp->gvid == gvar->gvid)
! {
! gvi = gvi_tmp;
! break;
! }
! }
! Assert(gvi != NULL);
!
! /*
! * If any aggregate references both input relations, something went
! * wrong during construction of one of the input targets: one input
! * rel is grouped, but no grouping target should have been created for
! * it if some aggregate required more than that input rel.
! */
! Assert(gvi->gv_eval_at == NULL ||
! !(bms_overlap(gvi->gv_eval_at, outer_rel->relids) &&
! bms_overlap(gvi->gv_eval_at, inner_rel->relids)));
!
! /*
! * If the aggregate belongs to the plain relation, it probably
! * means that non-grouping expression made aggregation of that
! * input relation impossible. Since that expression is not
! * necessarily emitted by the current join, aggregation might be
! * possible here. On the other hand, aggregation of a join which
! * already contains a grouped relation does not seem too
! * beneficial.
! *
! * XXX The condition below is also met if the query contains both
! * "star aggregate" and a normal one. Since the earlier can be
! * added to any base relation, and since we don't support join of
! * 2 grouped relations, join of arbitrary 2 relations will always
! * result in a plain relation.
! *
! * XXX If we conclude that aggregation is worth, only consider
! * this test failed if target usable for aggregation cannot be
! * created (i.e. the non-grouping expression is in the output of
! * the current join).
! */
! if ((outer_rel->gpi == NULL &&
! bms_overlap(gvi->gv_eval_at, outer_rel->relids))
! || (inner_rel->gpi == NULL &&
! bms_overlap(gvi->gv_eval_at, inner_rel->relids)))
! return false;
!
! /* Look for the aggregate in the input targets. */
! if (outer_rel->gpi != NULL)
! {
! /* No more than one input path should be grouped. */
! Assert(inner_rel->gpi == NULL);
!
! foreach(l2, outer_path->pathtarget->exprs)
! {
! expr = (Expr *) lfirst(l2);
!
! if (!IsA(expr, GroupedVar))
! continue;
!
! gvar = castNode(GroupedVar, expr);
! if (gvar->gvid == gvi->gvid)
! {
! found = true;
! break;
! }
! }
! }
! else if (!found && inner_rel->gpi != NULL)
! {
! Assert(outer_rel->gpi == NULL);
!
! foreach(l2, inner_path->pathtarget->exprs)
! {
! expr = (Expr *) lfirst(l2);
!
! if (!IsA(expr, GroupedVar))
! continue;
!
! gvar = castNode(GroupedVar, expr);
! if (gvar->gvid == gvi->gvid)
! {
! found = true;
! break;
! }
! }
}
+
+ /* Even a single missing aggregate causes the whole test to fail. */
+ if (!found)
+ return false;
}
+
+ return true;
}
/*
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 5a68de3..eadf3ee
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** mark_dummy_rel(RelOptInfo *rel)
*** 1217,1223 ****
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
--- 1217,1223 ----
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index a2fe661..91d855c
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** create_tidscan_paths(PlannerInfo *root,
*** 266,270 ****
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 266,270 ----
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer), false);
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index ebd442a..a7be1b2
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,19 ****
--- 14,20 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
***************
*** 26,31 ****
--- 27,33 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 45,50 ****
--- 47,53 ----
} PostponedQual;
+ static void create_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** add_vars_to_targetlist(PlannerInfo *root
*** 240,245 ****
--- 243,532 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate and setup
+ * GroupedPathInfo for each base relation that can product grouped paths.
+ *
+ * XXX In the future we might want to create GroupedVarInfo for grouping
+ * expressions too, so that grouping key is not limited to plain Var if the
+ * grouping takes place below the top-level join.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouping_info_to_base_rels(PlannerInfo *root)
+ {
+ int i;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause || root->group_pathkeys == NIL)
+ return;
+
+ /* TODO This is just for PoC. Relax the limitation later. */
+ if (root->parse->havingQual)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_grouped_var_infos(root);
+
+ /* Is no grouping is possible below the top-level join? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+
+ /*
+ * "other rels" will have their targets built later, by translation of
+ * the target of the parent rel - see set_append_rel_size. If we
+ * wanted to prepare the child rels here, we'd need another iteration
+ * of simple_rel_array_size.
+ */
+ if (rel != NULL && rel->reloptkind == RELOPT_BASEREL)
+ prepare_rel_for_grouping(root, rel);
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ *
+ * TODO Include aggregates from HAVING clause.
+ */
+ static void
+ create_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ /*
+ * TODO Check if processed_tlist contains the HAVING aggregates. If not,
+ * get them elsewhere.
+ */
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ Expr *expr = (Expr *) lfirst(lc2);
+
+ gvi = castNode(GroupedVarInfo, expr);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ /* TODO Initialize gv_width. */
+ gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ {
+ Assert(aggref->aggstar);
+ gvi->gv_eval_at = NULL;
+ }
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Check if all the expressions of rel->reltarget can be used as grouping
+ * expressions and create target for grouped paths.
+ *
+ * If we succeed to create the grouping target, also replace rel->reltarget
+ * with a new one that has sortgrouprefs initialized -- this is necessary for
+ * create_agg_plan to match the grouping clauses against the input target
+ * expressions.
+ *
+ * rel_agg_attrs is a set attributes of the relation referenced by aggregate
+ * arguments. These can exist in the (plain) target without being grouping
+ * expressions.
+ *
+ * rel_agg_vars should be passed instead if rel is a join.
+ *
+ * TODO How about PHVs?
+ *
+ * TODO Make sure cost / width of both "result" and "plain" are correct.
+ */
+ PathTarget *
+ create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs, List *rel_agg_vars)
+ {
+ PathTarget *result, *plain;
+ ListCell *lc;
+
+ /* The plan to be returned. */
+ result = create_empty_pathtarget();
+ /* The one to replace rel->reltarget. */
+ plain = create_empty_pathtarget();
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *texpr;
+ Index sortgroupref;
+ bool agg_arg_only = false;
+
+ texpr = (Expr *) lfirst(lc);
+
+ sortgroupref = get_expr_sortgroupref(root, texpr);
+ if (sortgroupref > 0)
+ {
+ /* It's o.k. to use the target expression for grouping. */
+ add_column_to_pathtarget(result, texpr, sortgroupref);
+
+ /*
+ * As for the plain target, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(plain, texpr, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * It may still be o.k. if the expression is only contained in Aggref
+ * - then it's not expected in the grouped output.
+ *
+ * TODO Try to handle generic expression, not only Var. That might
+ * require us to create rel->reltarget of the grouping rel in
+ * parallel to that of the plain rel, and adding whole expressions
+ * instead of individual vars.
+ */
+ if (IsA(texpr, Var))
+ {
+ Var *arg_var = castNode(Var, texpr);
+
+ if (rel->relid > 0)
+ {
+ AttrNumber varattno;
+
+ /*
+ * For a single relation we only need to check attribute
+ * number.
+ *
+ * Apply the same offset that pull_varattnos() did.
+ */
+ varattno = arg_var->varattno - FirstLowInvalidHeapAttributeNumber;
+
+ if (bms_is_member(varattno, rel_agg_attrs))
+ agg_arg_only = true;
+ }
+ else
+ {
+ ListCell *lc2;
+
+ /* Join case. */
+ foreach(lc2, rel_agg_vars)
+ {
+ Var *var = castNode(Var, lfirst(lc2));
+
+ if (var->varno == arg_var->varno &&
+ var->varattno == arg_var->varattno)
+ {
+ agg_arg_only = true;
+ break;
+ }
+ }
+ }
+
+ if (agg_arg_only)
+ {
+ /*
+ * This expression is not suitable for grouping, but the
+ * aggregation input target ought to stay complete.
+ */
+ add_column_to_pathtarget(plain, texpr, 0);
+ }
+ }
+
+ /*
+ * A single mismatched expression makes the whole relation useless
+ * for grouping.
+ */
+ if (!agg_arg_only)
+ {
+ /*
+ * TODO This seems possible to happen multiple times per relation,
+ * so result might be worth freeing. Implement free_pathtarget()?
+ * Or mark the relation as inappropriate for grouping?
+ */
+ /* TODO Free both result and plain. */
+ return NULL;
+ }
+ }
+
+ if (list_length(result->exprs) == 0)
+ {
+ /* TODO free_pathtarget(result); free_pathtarget(plain) */
+ result = NULL;
+ }
+
+ /* Apply the adjusted input target as the replacement is complete now.q */
+ rel->reltarget = plain;
+
+ return result;
+ }
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 5565736..058af2c
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,229 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual));
}
/*
--- 223,229 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual), false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index ef0de3f..f70b445
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,89 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,89 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals), false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 114,120 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 177,182 ****
--- 178,191 ----
(*qp_callback) (root, qp_extra);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, Initialize GroupedPathInfo
+ * of base relations capable to do the grouping and setup
+ * root->grouped_var_list.
+ */
+ add_grouping_info_to_base_rels(root);
+
+ /*
* Examine any "placeholder" expressions generated during subquery pullup.
* Make sure that the Vars they need are marked as needed at the relevant
* join level. This must be done before join removal because it might
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 649a233..2c44f42
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 130,138 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
--- 130,135 ----
*************** inheritance_planner(PlannerInfo *root)
*** 1419,1425 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)));
}
/*--------------------
--- 1416,1422 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)), false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 2040,2046 ****
}
/* And shove it into final_rel */
! add_path(final_rel, path);
}
/*
--- 2037,2043 ----
}
/* And shove it into final_rel */
! add_path(final_rel, path, false);
}
/*
*************** get_number_of_groups(PlannerInfo *root,
*** 3446,3485 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3443,3448 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3600,3606 ****
(List *) parse->havingQual);
}
! add_path(grouped_rel, path);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
--- 3563,3569 ----
(List *) parse->havingQual);
}
! add_path(grouped_rel, path, false);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
*************** create_grouping_paths(PlannerInfo *root,
*** 3777,3783 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
--- 3740,3747 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3786,3792 ****
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups));
}
}
}
--- 3750,3757 ----
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3817,3823 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
}
}
}
--- 3782,3789 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3869,3875 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
else if (parse->groupClause)
{
--- 3835,3841 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
else if (parse->groupClause)
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3884,3890 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
else
{
--- 3850,3856 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
else
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3933,3939 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3899,3905 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3942,3948 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
/*
* The point of using Gather Merge rather than Gather is that it
--- 3908,3914 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
/*
* The point of using Gather Merge rather than Gather is that it
*************** create_grouping_paths(PlannerInfo *root,
*** 3995,4001 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3961,3967 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4004,4010 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
}
}
--- 3970,3976 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4049,4055 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
}
--- 4015,4021 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4087,4095 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
}
}
}
/* Give a helpful error if we failed to find any implementation */
--- 4053,4129 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
}
}
+
+ /*
+ * If input_rel has partially aggregated partial paths, gather them
+ * and perform the final aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL &&
+ input_rel->gpi->partial_pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->partial_pathlist);
+ double total_groups = path->rows * path->parallel_workers;
+
+ path = (Path *) create_gather_path(root,
+ input_rel,
+ path,
+ path->pathtarget,
+ NULL,
+ &total_groups);
+
+ /*
+ * The input path is partially aggregated and the final
+ * aggregation - if the path wins - will be done below. So we're
+ * done with it for now.
+ *
+ * The top-level grouped_rel needs to receive the path into
+ * regular pathlist, as opposed grouped_rel->gpi->pathlist.
+ */
+
+ add_path(input_rel, path, false);
+ }
+
+ /*
+ * If input_rel has partially aggregated paths, perform the final
+ * aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *pre_agg = (Path *) linitial(input_rel->gpi->pathlist);
+
+ dNumGroups = get_number_of_groups(root, pre_agg->rows, gd);
+
+ MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+
+ add_path(grouped_rel,
+ (Path *) create_agg_path(root, grouped_rel,
+ pre_agg,
+ target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ &agg_final_costs,
+ dNumGroups),
+ false);
+ }
}
/* Give a helpful error if we failed to find any implementation */
*************** consider_groupingsets_paths(PlannerInfo
*** 4289,4295 ****
strat,
new_rollups,
agg_costs,
! dNumGroups));
return;
}
--- 4323,4329 ----
strat,
new_rollups,
agg_costs,
! dNumGroups), false);
return;
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4447,4453 ****
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups));
}
}
--- 4481,4487 ----
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups), false);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4464,4470 ****
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups));
}
/*
--- 4498,4504 ----
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups), false);
}
/*
*************** create_one_window_path(PlannerInfo *root
*** 4649,4655 ****
window_pathkeys);
}
! add_path(window_rel, path);
}
/*
--- 4683,4689 ----
window_pathkeys);
}
! add_path(window_rel, path, false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4755,4761 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
}
--- 4789,4795 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4782,4788 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
/*
--- 4816,4822 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4829,4835 ****
parse->distinctClause,
NIL,
NULL,
! numDistinctRows));
}
/* Give a helpful error if we failed to find any implementation */
--- 4863,4869 ----
parse->distinctClause,
NIL,
NULL,
! numDistinctRows), false);
}
/* Give a helpful error if we failed to find any implementation */
*************** create_ordered_paths(PlannerInfo *root,
*** 4927,4933 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 4961,4967 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
*************** create_ordered_paths(PlannerInfo *root,
*** 4977,4983 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5011,5017 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index 1278371..548b372
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
tlist_vinfo vars[FLEXIBLE_ARRAY_MEMBER]; /* has num_vars entries */
} indexed_tlist;
*************** set_upper_references(PlannerInfo *root,
*** 1725,1733 ****
--- 1726,1777 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the original list.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1937,1942 ****
--- 1981,1987 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
/* Find the Vars and fill in the index array */
*************** build_tlist_index(List *tlist)
*** 1956,1961 ****
--- 2001,2008 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else
itlist->has_non_vars = true;
}
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2233,2238 ****
--- 2280,2310 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2389,2395 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
--- 2461,2468 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
new file mode 100644
index a1be858..87a74be
*** a/src/backend/optimizer/prep/prepunion.c
--- b/src/backend/optimizer/prep/prepunion.c
*************** plan_set_operations(PlannerInfo *root)
*** 207,213 ****
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
--- 207,213 ----
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path, false);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 2d5caae..98e5d6a
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 24,29 ****
--- 24,31 ----
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if get_grouping_expressions ends up in another module. */
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
*************** set_cheapest(RelOptInfo *parent_rel)
*** 409,416 ****
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
--- 411,419 ----
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
+ List *pathlist;
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 427,432 ****
--- 430,443 ----
/* Pretend parameterized paths have no pathkeys, per comment above */
new_path_pathkeys = new_path->param_info ? NIL : new_path->pathkeys;
+ if (!grouped)
+ pathlist = parent_rel->pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->pathlist;
+ }
+
/*
* Loop to check proposed new path against old paths. Note it is possible
* for more than one old path to be tossed out because new_path dominates
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 436,442 ****
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
--- 447,453 ----
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 582,589 ****
*/
if (remove_old)
{
! parent_rel->pathlist = list_delete_cell(parent_rel->pathlist,
! p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
--- 593,599 ----
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 614,622 ****
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(parent_rel->pathlist, insert_after, new_path);
else
! parent_rel->pathlist = lcons(new_path, parent_rel->pathlist);
}
else
{
--- 624,637 ----
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->pathlist = pathlist;
! else
! parent_rel->gpi->pathlist = pathlist;
}
else
{
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 646,653 ****
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer)
{
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
--- 661,669 ----
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped)
{
+ List *pathlist;
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 656,664 ****
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup : parent_rel->consider_startup;
! foreach(p1, parent_rel->pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 672,689 ----
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup :
! parent_rel->consider_startup;
! if (!grouped)
! pathlist = parent_rel->pathlist;
! else
! {
! Assert(parent_rel->gpi != NULL);
! pathlist = parent_rel->gpi->pathlist;
! }
!
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 749,771 ****
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->partial_pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
--- 774,805 ----
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
+ List *pathlist;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
+
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
*************** add_partial_path(RelOptInfo *parent_rel,
*** 819,830 ****
}
/*
! * Remove current element from partial_pathlist if dominated by new.
*/
if (remove_old)
{
! parent_rel->partial_pathlist =
! list_delete_cell(parent_rel->partial_pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
--- 853,863 ----
}
/*
! * Remove current element from pathlist if dominated by new.
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
*************** add_partial_path(RelOptInfo *parent_rel,
*** 839,845 ****
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the partial_pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
--- 872,878 ----
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
*************** add_partial_path(RelOptInfo *parent_rel,
*** 850,859 ****
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(parent_rel->partial_pathlist, insert_after, new_path);
else
! parent_rel->partial_pathlist =
! lcons(new_path, parent_rel->partial_pathlist);
}
else
{
--- 883,896 ----
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->partial_pathlist = pathlist;
! else
! parent_rel->gpi->partial_pathlist = pathlist;
}
else
{
*************** add_partial_path(RelOptInfo *parent_rel,
*** 874,882 ****
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys)
{
ListCell *p1;
/*
* Our goal here is twofold. First, we want to find out whether this path
--- 911,928 ----
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys, bool grouped)
{
ListCell *p1;
+ List *pathlist;
+
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
/*
* Our goal here is twofold. First, we want to find out whether this path
*************** add_partial_path_precheck(RelOptInfo *pa
*** 886,895 ****
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist to be very short, and getting a definitive
! * answer at this stage avoids the need to call add_path_precheck.
*/
! foreach(p1, parent_rel->partial_pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 932,942 ----
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist / grouped_pathlist to be very short, and
! * getting a definitive answer at this stage avoids the need to call
! * add_path_precheck.
*/
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_partial_path_precheck(RelOptInfo *pa
*** 918,924 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL))
return false;
return true;
--- 965,971 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, grouped))
return false;
return true;
*************** calc_non_nestloop_required_outer(Path *o
*** 2055,2060 ****
--- 2102,2108 ----
* 'restrict_clauses' are the RestrictInfo nodes to apply at the join
* 'pathkeys' are the path keys of the new join path
* 'required_outer' is the set of required outer rels
+ * 'target' can be passed to override that of joinrel.
*
* Returns the resulting path node.
*/
*************** create_nestloop_path(PlannerInfo *root,
*** 2068,2074 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
--- 2116,2123 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
*************** create_nestloop_path(PlannerInfo *root,
*** 2101,2107 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2150,2156 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target == NULL ? joinrel->reltarget : target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2159,2171 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2208,2222 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2210,2215 ****
--- 2261,2267 ----
* 'required_outer' is the set of required outer rels
* 'hashclauses' are the RestrictInfo nodes to use as hash clauses
* (this should be a subset of the restrict_clauses list)
+ * 'target' can be passed to override that of joinrel.
*/
HashPath *
create_hashjoin_path(PlannerInfo *root,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2221,2233 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2273,2287 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_agg_path(PlannerInfo *root,
*** 2713,2718 ****
--- 2767,2942 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * first_call indicates whether the function is being called first time for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * group_clauses, group_exprs and agg_exprs are pointers to lists we populate
+ * when called first time for particular index, and that user passes for
+ * subsequent calls.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ if (first_call)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ if (first_call)
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows, NULL);
+
+ /* TODO HAVING qual. */
+ Assert(*group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, rel->gpi->target, AGG_SORTED,
+ AGGSPLIT_INITIAL_SERIAL, *group_clauses, NIL,
+ &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Appy partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ *
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (first_call)
+ {
+ /*
+ * Find one grouping clause per grouping column.
+ *
+ * All that create_agg_plan eventually needs of the clause is
+ * tleSortGroupRef, so we don't have to care that the clause
+ * expression might differ from texpr, in case texpr was derived from
+ * EC.
+ */
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows,
+ NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(*group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ rel->gpi->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 342d884..3cd7093
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 25,30 ****
--- 25,31 ----
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
#include "utils/hsearch.h"
*************** typedef struct JoinHashEntry
*** 35,41 ****
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
--- 36,42 ----
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
*************** build_simple_rel(PlannerInfo *root, int
*** 120,125 ****
--- 121,127 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->gpi = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** build_join_rel(PlannerInfo *root,
*** 497,502 ****
--- 499,505 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 539,548 ****
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
--- 542,558 ----
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel, false);
! build_joinrel_tlist(root, joinrel, inner_rel, false);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ /* Try to build grouped target. */
+ /*
+ * TODO Consider if placeholders make sense here. If not, also make the
+ * related code below conditional.
+ */
+ prepare_rel_for_grouping(root, joinrel);
+
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
*************** min_join_parameterization(PlannerInfo *r
*** 670,686 ****
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel)
{
Relids relids = joinrel->relids;
ListCell *vars;
! foreach(vars, input_rel->reltarget->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
--- 680,722 ----
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped)
{
Relids relids = joinrel->relids;
+ PathTarget *input_target, *result;
ListCell *vars;
+ int i = -1;
! if (!grouped)
! {
! input_target = input_rel->reltarget;
! result = joinrel->reltarget;
! }
! else
! {
! if (input_rel->gpi != NULL)
! {
! input_target = input_rel->gpi->target;
! Assert(input_target != NULL);
! }
! else
! input_target = input_rel->reltarget;
!
! /* Caller should have initialized this. */
! Assert(joinrel->gpi != NULL);
!
! /* Default to the plain target. */
! result = joinrel->gpi->target;
! }
!
! foreach(vars, input_target->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
+ i++;
+
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
*************** build_joinrel_tlist(PlannerInfo *root, R
*** 704,713 ****
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
/* Yup, add it to the output */
! joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs, var);
/* Vars have cost zero, so no need to adjust reltarget->cost */
! joinrel->reltarget->width += baserel->attr_widths[ndx];
}
}
}
--- 740,763 ----
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
+ Index sortgroupref = 0;
+
/* Yup, add it to the output */
! if (input_target->sortgrouprefs)
! sortgroupref = input_target->sortgrouprefs[i];
!
! /*
! * Even if not used for grouping in the input path (the input path
! * is not necessarily grouped), it might be useful for grouping
! * higher in the join tree.
! */
! if (sortgroupref == 0)
! sortgroupref = get_expr_sortgroupref(root, (Expr *) var);
!
! add_column_to_pathtarget(result, (Expr *) var, sortgroupref);
!
/* Vars have cost zero, so no need to adjust reltarget->cost */
! result->width += baserel->attr_widths[ndx];
}
}
}
*************** get_appendrel_parampathinfo(RelOptInfo *
*** 1359,1361 ****
--- 1409,1560 ----
return ppi;
}
+
+ /*
+ * If the relation can produce grouped paths, create GroupedPathInfo for it
+ * and create target for the grouped paths.
+ */
+ void
+ prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *rel_aggregates;
+ Relids rel_agg_attrs = NULL;
+ List *rel_agg_vars = NIL;
+ bool found_higher;
+ ListCell *lc;
+ PathTarget *target_grouped;
+
+ if (rel->relid > 0)
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the aggregate
+ * would receive different input at the base rel level.
+ *
+ * TODO For RELOPT_JOINREL, do not return if all the joins that can set
+ * any entry of the grouped target (do we need to postpone this check
+ * until the grouped target is available, and should create_grouped_target
+ * take care?) of this rel to NULL are provably below rel. (It's ok if rel
+ * is one of these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return;
+
+ /*
+ * Check if some aggregates can be evaluated in this relation's target,
+ * and collect all vars referenced by these aggregates.
+ */
+ rel_aggregates = NIL;
+ found_higher = false;
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which typically means
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ Aggref *aggref = castNode(Aggref, gvi->gvexpr);
+
+ /*
+ * Accept the aggregate.
+ *
+ * GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.
+ */
+ rel_aggregates = lappend(rel_aggregates, gvi);
+
+ if (rel->relid > 0)
+ {
+ /*
+ * Simple relation. Collect attributes referenced by the
+ * aggregate arguments.
+ */
+ pull_varattnos((Node *) aggref, rel->relid, &rel_agg_attrs);
+ }
+ else
+ {
+ List *agg_vars;
+
+ /*
+ * Join. Collect vars referenced by the aggregate
+ * arguments.
+ */
+ /*
+ * TODO Can any argument contain PHVs? And if so, does it matter?
+ * Consider PVC_INCLUDE_PLACEHOLDERS | PVC_RECURSE_PLACEHOLDERS.
+ */
+ agg_vars = pull_var_clause((Node *) aggref,
+ PVC_RECURSE_AGGREGATES);
+ rel_agg_vars = list_concat(rel_agg_vars, agg_vars);
+ }
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * Remember that there is at least one aggregate that needs more
+ * than this rel.
+ */
+ found_higher = true;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function.
+ */
+ if (rel_aggregates == NIL)
+ {
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ if (found_higher)
+ {
+ /*
+ * If some aggregate(s) need only this rel but some other need
+ * multiple relations including the the current one, grouping of the
+ * current rel could steal some input variables from the "higher
+ * aggregate" (besides decreasing the number of input rows).
+ */
+ list_free(rel_aggregates);
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ /*
+ * If rel->reltarget can be used for aggregation, mark the relation as
+ * capable of grouping.
+ */
+ Assert(rel->gpi == NULL);
+ target_grouped = create_grouped_target(root, rel, rel_agg_attrs,
+ rel_agg_vars);
+ if (target_grouped != NULL)
+ {
+ GroupedPathInfo *gpi;
+
+ gpi = makeNode(GroupedPathInfo);
+ gpi->target = copy_pathtarget(target_grouped);
+ gpi->pathlist = NIL;
+ gpi->partial_pathlist = NIL;
+ rel->gpi = gpi;
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the target.
+ */
+ add_aggregates_to_target(root, gpi->target, rel_aggregates, rel);
+ }
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 0952385..dd962b7
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,413 ****
--- 408,487 ----
return result;
}
+ /*
+ * get_sortgrouplist_clauses
+ *
+ * Given a "grouped target" (i.e. target where each non-GroupedVar
+ * element must have sortgroupref set), build a list of the referencing
+ * SortGroupClauses, a list of the corresponding grouping expressions and
+ * a list of aggregate expressions.
+ */
+ /* Refine the function name. */
+ void
+ get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses, List **grouping_exprs,
+ List **agg_exprs)
+ {
+ ListCell *l;
+ int i = 0;
+
+ foreach(l, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(l);
+
+ /* The target should contain at least one grouping column. */
+ Assert(target->sortgrouprefs != NULL);
+
+ if (IsA(texpr, GroupedVar))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides aggregates, the target should contain no expressions w/o
+ * sortgroupref. Plain relation being joined to grouped can have
+ * sortgroupref equal to zero for expressions contained neither in
+ * grouping expression nor in aggregate arguments, but if the target
+ * contains such an expression, it shouldn't be used for aggregation
+ * --- see can_aggregate field of GroupedPathInfo.
+ */
+ Assert(sortgroupref > 0);
+
+ cl = get_sortgroupref_clause(sortgroupref, root->parse->groupClause);
+ *grouping_clauses = list_append_unique(*grouping_clauses, cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ *grouping_exprs = list_append_unique(*grouping_exprs, texpr);
+ }
+
+ /* Now collect the aggregates. */
+ while (l != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(l));
+
+ /* Currently, GroupedVarInfo can only represent aggregate. */
+ Assert(gvar->agg_partial != NULL);
+ *agg_exprs = lappend(*agg_exprs, gvar->agg_partial);
+ l = lnext(l);
+ }
+ }
+
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 857,1081 ----
}
/*
+ * Replace each "grouped var" in the source targetlist with the original
+ * expression.
+ *
+ * TODO Think of more suitable name. Although "grouped var" may substitute for
+ * grouping expressions in the future, currently Aggref is the only outcome of
+ * the replacement. undo_grouped_var_substitutions?
+ */
+ List *
+ restore_grouping_expressions(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te, *te_new;
+ Aggref *expr_new = NULL;
+
+ te = castNode(TargetEntry, lfirst(l));
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ expr_new = gvar->agg_partial;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
+ * For each aggregate add GroupedVar to target if "vars" is true, or the
+ * Aggref (marked as partial) if "vars" is false.
+ *
+ * If caller passes the aggregates, he must do so in the form of
+ * GroupedVarInfos so that we don't have to look for gvid. If NULL is passed,
+ * the function retrieves the suitable aggregates itself.
+ *
+ * List of the aggregates added is returned. This is only useful if the
+ * function had to retrieve the aggregates itself (i.e. NIL was passed for
+ * aggregates) -- caller is expected to do extra checks in that case (and to
+ * also free the list).
+ */
+ List *
+ add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel)
+ {
+ ListCell *lc;
+ GroupedVarInfo *gvi;
+
+ if (aggregates == NIL)
+ {
+ /* Caller should pass the aggregates for base relation. */
+ Assert(rel->reloptkind != RELOPT_BASEREL);
+
+ /* Collect all aggregates that this rel can evaluate. */
+ foreach(lc, root->grouped_var_list)
+ {
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * Overlap is not guarantee of correctness alone, but caller needs
+ * to do additional checks, so we're optimistic here.
+ *
+ * If gv_eval_at is NULL, the underlying Aggref should have
+ * aggstar set.
+ */
+ if (bms_overlap(gvi->gv_eval_at, rel->relids) ||
+ gvi->gv_eval_at == NULL)
+ aggregates = lappend(aggregates, gvi);
+ }
+
+ if (aggregates == NIL)
+ return NIL;
+ }
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, aggregates)
+ {
+ GroupedVar *gvar;
+
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_new_column_to_pathtarget(target, (Expr *) gvar);
+ }
+
+ return aggregates;
+ }
+
+ /*
+ * Return ressortgroupref of the target entry that is either equal to the
+ * expression or exists in the same equivalence class.
+ */
+ Index
+ get_expr_sortgroupref(PlannerInfo *root, Expr *expr)
+ {
+ ListCell *lc;
+ Index sortgroupref;
+
+ /*
+ * First, check if the query group clause contains exactly this
+ * expression.
+ */
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = castNode(TargetEntry, lfirst(lc));
+
+ if (equal(expr, te->expr) && te->ressortgroupref > 0)
+ return te->ressortgroupref;
+ }
+
+ /*
+ * If exactly this expression is not there, check if a grouping clause
+ * exists that belongs to the same equivalence class as the expression.
+ */
+ foreach(lc, root->group_pathkeys)
+ {
+ PathKey *pk = castNode(PathKey, lfirst(lc));
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *lm;
+ EquivalenceMember *em;
+ Expr *em_expr = NULL;
+ Query *query = root->parse;
+
+ /*
+ * Single-member EC cannot provide us with additional expression.
+ */
+ if (list_length(ec->ec_members) < 2)
+ continue;
+
+ /* We need equality anywhere in the join tree. */
+ if (ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * TODO Reconsider this restriction. As the grouping expression is
+ * only evaluated at the relation level (and only the result will be
+ * propagated to the final targetlist), volatile function might be
+ * o.k. Need to think what volatile EC exactly means.
+ */
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(lm, ec->ec_members)
+ {
+ em = (EquivalenceMember *) lfirst(lm);
+
+ /* The EC has !ec_below_outer_join. */
+ Assert(!em->em_nullable_relids);
+ if (equal(em->em_expr, expr))
+ {
+ em_expr = (Expr *) em->em_expr;
+ break;
+ }
+ }
+
+ if (em_expr == NULL)
+ /* Go for the next EC. */
+ continue;
+
+ /*
+ * Find the corresponding SortGroupClause, which provides us with
+ * sortgroupref. (It can belong to any EC member.)
+ */
+ sortgroupref = 0;
+ foreach(lm, ec->ec_members)
+ {
+ ListCell *lsg;
+
+ em = (EquivalenceMember *) lfirst(lm);
+ foreach(lsg, query->groupClause)
+ {
+ SortGroupClause *sgc;
+ Expr *expr;
+
+ sgc = (SortGroupClause *) lfirst(lsg);
+ expr = (Expr *) get_sortgroupclause_expr(sgc,
+ query->targetList);
+ if (equal(em->em_expr, expr))
+ {
+ Assert(sgc->tleSortGroupRef > 0);
+ sortgroupref = sgc->tleSortGroupRef;
+ break;
+ }
+ }
+
+ if (sortgroupref > 0)
+ break;
+ }
+
+ /*
+ * Since we searched in group_pathkeys, at least one EM of this EC
+ * should correspond to a SortGroupClause, otherwise the EC could
+ * not exist at all.
+ */
+ Assert(sortgroupref > 0);
+
+ return sortgroupref;
+ }
+
+ /* No EC found in group_pathkeys. */
+ return 0;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index cbde1ff..49f87ac
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7567,7572 ****
--- 7567,7580 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 9001,9010 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 9009,9026 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index a35b93b..78e24ea
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 114,119 ****
--- 114,120 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
*************** estimate_hash_bucketsize(PlannerInfo *ro
*** 3705,3710 ****
--- 3706,3744 ----
return (Selectivity) estfract;
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index f59d719..ba1eac8
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 218,223 ****
--- 218,224 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_GroupedPathInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 258,267 ****
--- 259,270 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index 7a8e2fd..103ed14
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 256,261 ****
--- 256,263 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 401,406 ****
--- 403,410 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 548,553 ****
--- 552,560 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to produce grouped paths. */
+ struct GroupedPathInfo *gpi;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 913,918 ****
--- 920,947 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * GroupedPathInfo
+ *
+ * If RelOptInfo points to this structure, grouped paths can be created for
+ * it.
+ *
+ * "target" will be used as pathtarget of grouped paths produced by this
+ * relation. Grouped path is either a result of aggregation of the relation
+ * that owns this structure or, if the owning relation is a join, a join path
+ * whose one side is a grouped path and the other is a plain (i.e. not
+ * grouped) one. (Two grouped paths cannot be joined in general because
+ * grouping of one side of the join essentially reduces occurrence of groups
+ * of the other side in the input of the final aggregation.)
+ */
+ typedef struct GroupedPathInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* output of grouped paths. */
+ List *pathlist; /* List of grouped paths. */
+ List *partial_pathlist; /* List of partial grouped paths. */
+ } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct PlaceHolderVar
*** 1852,1857 ****
--- 1881,1919 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. The reason is that aggregates having start as the argument can be
+ * evaluated at various places in the join tree (i.e. cannot be assigned to
+ * target list of exactly one relation). Also this concept seems to be less
+ * invasive than adding the grouped vars to reltarget (in which case
+ * attr_needed and attr_widths arrays of RelOptInfo) would also need
+ * additional changes.
+ *
+ * gvexpr is a pointer to gvexpr field of the corresponding instance
+ * GroupedVarInfo. It's there for the sake of exprType(), exprCollation(),
+ * etc.
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if the
+ * GroupedVar is in the target of a parent relation (RELOPT_BASEREL). However
+ * within a child relation's (RELOPT_OTHER_MEMBER_REL) target it points to a
+ * copy which has argument expressions translated, so they no longer reference
+ * the parent.
+ *
+ * XXX Currently we only create GroupedVar for aggregates, but sometime we can
+ * do it for grouping keys as well. That would allow grouping below the
+ * top-level join by keys other than plain Var.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PlaceHolderInfo
*** 2067,2072 ****
--- 2129,2150 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is
+ * the corresponding partial aggregate */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 77bc770..abc2ac1
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern int compare_path_costs(Path *path
*** 25,37 ****
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
--- 25,39 ----
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path,
! bool grouped);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys,
! bool grouped);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
*************** extern NestPath *create_nestloop_path(Pl
*** 124,130 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 126,133 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern MergePath *create_mergejoin_path(
*** 138,144 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 141,148 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern HashPath *create_hashjoin_path(Pl
*** 149,155 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
--- 153,160 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
*************** extern AggPath *create_agg_path(PlannerI
*** 190,195 ****
--- 195,214 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern ParamPathInfo *get_joinrel_paramp
*** 285,289 ****
--- 304,309 ----
List **restrict_clauses);
extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
+ extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 25fe78c..38967da
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void set_dummy_rel_pathlist(RelOp
*** 53,59 ****
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
--- 53,64 ----
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
!
! extern void create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
! Path *subpath, bool precheck, bool partial,
! AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
*************** extern void debug_print_rel(PlannerInfo
*** 67,73 ****
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
--- 72,79 ----
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index 5df68a2..07bc4c0
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** extern int join_collapse_limit;
*** 74,80 ****
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
--- 74,82 ----
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
! extern void add_grouping_info_to_base_rels(PlannerInfo *root);
! extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index ccb93d8..ddea03c
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern Node *get_sortgroupclause_expr(So
*** 41,46 ****
--- 41,49 ----
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
+ extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses,
+ List **grouping_exprs, List **agg_exprs);
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 68,84 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
+ extern List *add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel);
+ extern Index get_expr_sortgroupref(PlannerInfo *root, Expr *expr);
+ /* TODO Move definition from initsplan.c to tlist.c. */
+ extern PathTarget *create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs,
+ List *rel_agg_vars);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 9f9d2dc..e05e6f6
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern double estimate_num_groups(Planne
*** 206,211 ****
--- 206,214 ----
extern Selectivity estimate_hash_bucketsize(PlannerInfo *root, Node *hashkey,
double nbuckets);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-08-17 15:22 Antonin Houska <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 4 replies; 59+ messages in thread
From: Antonin Houska @ 2017-08-17 15:22 UTC (permalink / raw)
To: pgsql-hackers
Antonin Houska <[email protected]> wrote:
> Antonin Houska <[email protected]> wrote:
>
> > This is a new version of the patch I presented in [1].
>
> Rebased.
>
> cat .git/refs/heads/master
> b9a3ef55b253d885081c2d0e9dc45802cab71c7b
This is another version of the patch.
Besides other changes, it enables the aggregation push-down for postgres_fdw,
although only for aggregates whose transient aggregation state is equal to the
output type. For other aggregates (like avg()) the remote nodes will have to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.
shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
scans of base relation's partitions being pushed to shard nodes, the other
pushes down a join and performs aggregation of the join result on the remote
node. Of course, the query can only references one particular partition, until
the "partition-wise join" [1] patch gets committed and merged with this my
patch.
One thing I'm not sure about is whether the patch should remove
GetForeignUpperPaths function from FdwRoutine, which it essentially makes
obsolete. Or should it only be deprecated so far? I understand that
deprecation is important for "ordinary SQL users", but FdwRoutine is an
interface for extension developers, so the policy might be different.
[1] https://commitfest.postgresql.org/14/994/
Any feedback is appreciated.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[application/x-gzip] agg_pushdown_v3.tgz (78.7K, ../../29613.1502983342@localhost/2-agg_pushdown_v3.tgz)
download | 7 patch file(s) in archive:
agg_pushdown_v3/01_types.diff
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 7204169..c26be6a
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyAggref(const Aggref *from)
*** 1355,1360 ****
--- 1355,1361 ----
COPY_SCALAR_FIELD(aggcollid);
COPY_SCALAR_FIELD(inputcollid);
COPY_SCALAR_FIELD(aggtranstype);
+ COPY_SCALAR_FIELD(aggcombinefn);
COPY_NODE_FIELD(aggargtypes);
COPY_NODE_FIELD(aggdirectargs);
COPY_NODE_FIELD(args);
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2208,2213 ****
--- 2209,2230 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** _copyPlaceHolderInfo(const PlaceHolderIn
*** 2280,2285 ****
--- 2297,2317 ----
return newnode;
}
+ static GroupedVarInfo *
+ _copyGroupedVarInfo(const GroupedVarInfo *from)
+ {
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_SCALAR_FIELD(gvid);
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(gv_width);
+
+ return newnode;
+ }
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
*************** copyObjectImpl(const void *from)
*** 4988,4993 ****
--- 5020,5028 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
*************** copyObjectImpl(const void *from)
*** 5000,5005 ****
--- 5035,5043 ----
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index 8d92c03..9d3c8af
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 874,879 ****
--- 874,887 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3149,3154 ****
--- 3157,3165 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index e3eb0c5..78b64ce
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 259,264 ****
--- 259,270 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ type = exprType((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprTypmod(const Node *expr)
*** 492,497 ****
--- 498,508 ----
return ((const SetToDefault *) expr)->typeMod;
case T_PlaceHolderVar:
return exprTypmod((Node *) ((const PlaceHolderVar *) expr)->phexpr);
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->gvexpr);
default:
break;
}
*************** exprCollation(const Node *expr)
*** 903,908 ****
--- 914,925 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2177,2182 ****
--- 2194,2201 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2968,2973 ****
--- 2987,3002 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 5ce3c7c..402d42c
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2214,2219 ****
--- 2214,2220 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outRelOptInfo(StringInfo str, const Rel
*** 2260,2265 ****
--- 2261,2267 ----
WRITE_NODE_FIELD(cheapest_parameterized_paths);
WRITE_BITMAPSET_FIELD(direct_lateral_relids);
WRITE_BITMAPSET_FIELD(lateral_relids);
+ WRITE_NODE_FIELD(gpi);
WRITE_UINT_FIELD(relid);
WRITE_OID_FIELD(reltablespace);
WRITE_ENUM_FIELD(rtekind, RTEKind);
*************** _outParamPathInfo(StringInfo str, const
*** 2436,2441 ****
--- 2438,2456 ----
}
static void
+ _outGroupedPathInfo(StringInfo str, const GroupedPathInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDPATHINFO");
+
+ WRITE_NODE_FIELD(target_agg);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(pathlist);
+ WRITE_NODE_FIELD(partial_pathlist);
+ WRITE_NODE_FIELD(sortgroupclauses);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2479,2484 ****
--- 2494,2510 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** _outPlaceHolderInfo(StringInfo str, cons
*** 2532,2537 ****
--- 2558,2576 ----
}
static void
+ _outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_UINT_FIELD(gvid);
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_INT_FIELD(gv_width);
+ }
+
+ static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
*************** outNode(StringInfo str, const void *obj)
*** 4031,4042 ****
--- 4070,4087 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_GroupedPathInfo:
+ _outGroupedPathInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
*************** outNode(StringInfo str, const void *obj)
*** 4049,4054 ****
--- 4094,4102 ----
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index 86c811d..92c101b
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 522,527 ****
--- 522,543 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(sortgroupref);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** parseNodeString(void)
*** 2457,2462 ****
--- 2473,2480 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
new file mode 100644
index 8487eda..c908b40
*** a/src/backend/parser/parse_func.c
--- b/src/backend/parser/parse_func.c
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 98,103 ****
--- 98,104 ----
Oid vatype;
FuncDetailCode fdresult;
char aggkind = 0;
+ Oid aggcombinefn = InvalidOid;
ParseCallbackState pcbstate;
/*
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 321,326 ****
--- 322,328 ----
elog(ERROR, "cache lookup failed for aggregate %u", funcid);
classForm = (Form_pg_aggregate) GETSTRUCT(tup);
aggkind = classForm->aggkind;
+ aggcombinefn = classForm->aggcombinefn;
catDirectArgs = classForm->aggnumdirectargs;
ReleaseSysCache(tup);
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 666,671 ****
--- 668,674 ----
aggref->aggstar = agg_star;
aggref->aggvariadic = func_variadic;
aggref->aggkind = aggkind;
+ aggref->aggcombinefn = aggcombinefn;
/* agglevelsup will be set by transformAggregateCall */
aggref->aggsplit = AGGSPLIT_SIMPLE; /* planner might change this */
aggref->location = location;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index 27bd4f3..589be36
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 218,223 ****
--- 218,224 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_GroupedPathInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 258,267 ****
--- 259,270 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index 8c536a8..9ba1e28
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct Aggref
*** 296,301 ****
--- 296,302 ----
Oid aggcollid; /* OID of collation of result */
Oid inputcollid; /* OID of collation that function should use */
Oid aggtranstype; /* type Oid of aggregate's transition value */
+ Oid aggcombinefn; /* combine function (see pg_aggregate.h) */
List *aggargtypes; /* type Oids of direct and aggregated args */
List *aggdirectargs; /* direct arguments, if an ordered-set agg */
List *args; /* aggregated arguments and sort expressions */
*************** typedef struct Aggref
*** 306,311 ****
--- 307,313 ----
bool aggvariadic; /* true if variadic arguments have been
* combined into an array last argument */
char aggkind; /* aggregate kind (see pg_aggregate.h) */
+
Index agglevelsup; /* > 0 if agg belongs to outer query */
AggSplit aggsplit; /* expected agg-splitting mode of parent Agg */
int location; /* token location, or -1 if unknown */
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index be20288..3734b74
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 257,262 ****
--- 257,264 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 402,407 ****
--- 404,411 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 549,554 ****
--- 553,561 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to produce grouped paths. */
+ struct GroupedPathInfo *gpi;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 915,920 ****
--- 922,960 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * GroupedPathInfo
+ *
+ * If RelOptInfo points to this structure, grouped paths can be created for
+ * it.
+ *
+ * "target_agg" will be used as pathtarget of grouped paths produced by
+ * "explicit aggregation" of the relation that owns this structure. CAUTION:
+ * GroupedVars that contain Aggref expressions are supposed to follow the
+ * other ones. Iteration target_agg->exprs do rely on this arrangement.
+ *
+ * "target" is used for a grouped join path whose one input relation is a
+ * grouped relation and the other ones are plain (i.e. not grouped).
+ *
+ * "group_exprs" contains grouping expressions which are not plain vars. These
+ * are only added to path target of a path which should generate input for
+ * partial aggregation.
+ *
+ * (Two grouped paths cannot be joined in general because grouping of one side
+ * of the join essentially reduces occurrence of groups of the other side in
+ * the input of the final aggregation.)
+ */
+ typedef struct GroupedPathInfo
+ {
+ NodeTag type;
+
+ PathTarget *target_agg; /* output of explicit aggregation. */
+ PathTarget *target; /* output of generic grouped path. */
+ PathTarget *group_exprs; /* non-Var grouping expressions. */
+ List *pathlist; /* List of grouped paths. */
+ List *partial_pathlist; /* List of partial grouped paths. */
+ List *sortgroupclauses; /* Relation-specific grouping clauses. */
+ } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct PlaceHolderVar
*** 1855,1860 ****
--- 1895,1935 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. The reason is that aggregates having start as the argument can be
+ * evaluated at various places in the join tree (i.e. cannot be assigned to
+ * target list of exactly one relation). Also this concept seems to be less
+ * invasive than adding the grouped vars to reltarget (in which case
+ * attr_needed and attr_widths arrays of RelOptInfo) would also need
+ * additional changes.
+ *
+ * gvexpr is a pointer to gvexpr field of the corresponding instance
+ * GroupedVarInfo. It's there for the sake of exprType(), exprCollation(),
+ * etc.
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if the
+ * GroupedVar is in the target of a parent relation (RELOPT_BASEREL). However
+ * within a child relation's (RELOPT_OTHER_MEMBER_REL) target it points to a
+ * copy which has argument expressions translated, so they no longer reference
+ * the parent.
+ *
+ * XXX Currently we only create GroupedVar for aggregates, but sometime we can
+ * do it for grouping keys as well. That would allow grouping below the
+ * top-level join by keys other than plain Var.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index sortgroupref; /* SortGroupClause.tleSortGroupRef if gvexpr
+ * is grouping expression. */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PlaceHolderInfo
*** 2070,2075 ****
--- 2145,2169 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
agg_pushdown_v3/02_preprocess.diff
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
new file mode 100644
index 7997f50..e9dc220
*** a/src/backend/optimizer/path/equivclass.c
--- b/src/backend/optimizer/path/equivclass.c
*************** static bool reconsider_outer_join_clause
*** 64,69 ****
--- 64,82 ----
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
+ typedef struct translate_expr_context
+ {
+ Var **keys; /* Dictionary keys. */
+ Var **values; /* Dictionary values */
+ int nitems; /* Number of dictionary items. */
+ Relids *gv_eval_at_p; /* See GroupedVarInfo. */
+ Relids relids; /* Translate into these relids. */
+ } translate_expr_context;
+
+ static Node *translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context * context);
+ static int var_dictionary_comparator(const void *a, const void *b);
+
/*
* process_equivalence
*************** is_redundant_derived_clause(RestrictInfo
*** 2472,2474 ****
--- 2485,2806 ----
return false;
}
+
+ /*
+ * translate_expression_to_rels
+ *
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is contained in relids.
+ */
+ GroupedVarInfo *
+ translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Relids relids)
+ {
+ List *vars;
+ ListCell *l1;
+ int i,
+ j;
+ int nkeys,
+ nkeys_resolved;
+ Var **keys,
+ **values,
+ **keys_tmp;
+ Var *key,
+ *key_prev;
+ translate_expr_context context;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ /*
+ * Before actually trying to modify the expression tree, find out if all
+ * vars can be translated.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, 0);
+
+ /* No vars to translate? */
+ if (vars == NIL)
+ return NULL;
+
+ /*
+ * Search for individual replacement vars as well as the actual expression
+ * translation will be more efficient if we use a dictionary with the keys
+ * (i.e. the "source vars") unique and sorted.
+ */
+ nkeys = list_length(vars);
+ keys = (Var **) palloc(nkeys * sizeof(Var *));
+ i = 0;
+ foreach(l1, vars)
+ {
+ key = lfirst_node(Var, l1);
+ keys[i++] = key;
+ }
+
+ /*
+ * Sort the keys by varno. varattno decides where varnos are equal.
+ */
+ pg_qsort(keys, nkeys, sizeof(Var *), var_dictionary_comparator);
+
+ /*
+ * Pick unique values and get rid of the vars that need no translation.
+ */
+ keys_tmp = (Var **) palloc(nkeys * sizeof(Var *));
+ key_prev = NULL;
+ j = 0;
+ for (i = 0; i < nkeys; i++)
+ {
+ key = keys[i];
+
+ if ((key_prev == NULL || (key->varno != key_prev->varno &&
+ key->varattno != key_prev->varattno)) &&
+ !bms_is_member(key->varno, relids))
+ keys_tmp[j++] = key;
+
+ key_prev = key;
+ }
+ pfree(keys);
+ keys = keys_tmp;
+ nkeys = j;
+
+ /*
+ * Is there actually nothing to be translated?
+ */
+ if (nkeys == 0)
+ {
+ pfree(keys);
+ return NULL;
+ }
+
+ nkeys_resolved = 0;
+
+ /*
+ * Find the replacement vars.
+ */
+ values = (Var **) palloc0(nkeys * sizeof(Var *));
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ Relids ec_var_relids;
+ Var **ec_vars;
+ int ec_nvars;
+ ListCell *l2;
+
+ /* TODO Re-check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ ec_vars = (Var **) palloc(list_length(ec->ec_members) * sizeof(Var *));
+ ec_nvars = 0;
+ ec_var_relids = NULL;
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ ec_vars[ec_nvars++] = ec_var;
+ ec_var_relids = bms_add_member(ec_var_relids, ec_var->varno);
+ }
+
+ /*
+ * At least two vars are needed so that the EC is usable for
+ * translation.
+ */
+ if (ec_nvars <= 1)
+ {
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+ continue;
+ }
+
+ /*
+ * Now check where this EC can help.
+ */
+ for (i = 0; i < nkeys; i++)
+ {
+ Relids ec_rest;
+ bool relids_ok,
+ key_found;
+ Var *key = keys[i];
+ Var *value = values[i];
+
+ /* Skip this item if it's already resolved. */
+ if (value != NULL)
+ continue;
+
+ /*
+ * Can't translate if the EC does not mention key->varno.
+ */
+ if (!bms_is_member(key->varno, ec_var_relids))
+ continue;
+
+ /*
+ * Besides key, at least one EC member must belong to the relation
+ * we're translating our expression to.
+ */
+ ec_rest = bms_copy(ec_var_relids);
+ ec_rest = bms_del_member(ec_rest, key->varno);
+ relids_ok = bms_overlap(ec_rest, relids);
+ bms_free(ec_rest);
+ if (!relids_ok)
+ continue;
+
+ /*
+ * The preliminary checks passed, so try to find the exact vars.
+ */
+ key_found = false;
+ for (j = 0; j < ec_nvars; j++)
+ {
+ Var *ec_var = ec_vars[j];
+
+ if (!key_found && key->varno == ec_var->varno &&
+ key->varattno == ec_var->varattno)
+ key_found = true;
+
+ /*
+ * If relids contains multiple members, it shouldn't matter to
+ * which one we translate our key. Simply use the first one.
+ *
+ * XXX Shouldn't ec_var be copied?
+ */
+ if (value == NULL && bms_is_member(ec_var->varno, relids))
+ value = ec_var;
+
+ if (key_found && value != NULL)
+ break;
+ }
+
+ if (key_found && value != NULL)
+ {
+ values[i] = value;
+ nkeys_resolved++;
+
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+ }
+
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+
+ /* Don't need to check the remaining ECs? */
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+
+ /* Couldn't compose usable dictionary? */
+ if (nkeys_resolved < nkeys)
+ {
+ pfree(keys);
+ pfree(values);
+ return NULL;
+ }
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_copy(result->gv_eval_at);
+
+ /* The dictionary is ready, so perform the translation. */
+ context.keys = keys;
+ context.values = values;
+ context.nitems = nkeys;
+ context.gv_eval_at_p = &result->gv_eval_at;
+ context.relids = relids;
+ result->gvexpr = (Expr *)
+ translate_expression_to_rels_mutator((Node *) gvi->gvexpr, &context);
+
+ pfree(keys);
+ pfree(values);
+ return result;
+ }
+
+ static Node *
+ translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context * context)
+ {
+ if (node == NULL)
+ return NULL;
+
+ if (IsA(node, Var))
+ {
+ Var *var = castNode(Var, node);
+ Var **key_p;
+ Var *value;
+ int index;
+
+ /*
+ * Simply return the existing variable if already belongs to the
+ * relation we're adjusting the expression to.
+ */
+ if (bms_is_member(var->varno, context->relids))
+ return (Node *) var;
+
+ key_p = bsearch(&var, context->keys, context->nitems, sizeof(Var *),
+ var_dictionary_comparator);
+
+ /* We shouldn't have omitted any var from the dictionary. */
+ Assert(key_p != NULL);
+
+ index = key_p - context->keys;
+ Assert(index >= 0 && index < context->nitems);
+ value = context->values[index];
+
+ /* All values should be present in the dictionary. */
+ Assert(value != NULL);
+
+ /* Update gv_eval_at accordingly. */
+ bms_del_member(*context->gv_eval_at_p, var->varno);
+ *context->gv_eval_at_p = bms_add_member(*context->gv_eval_at_p,
+ value->varno);
+
+ return (Node *) value;
+ }
+
+ return expression_tree_mutator(node, translate_expression_to_rels_mutator,
+ (void *) context);
+ }
+
+ static int
+ var_dictionary_comparator(const void *a, const void *b)
+ {
+ Var **var1_p,
+ **var2_p;
+ Var *var1,
+ *var2;
+
+ var1_p = (Var **) a;
+ var1 = castNode(Var, *var1_p);
+ var2_p = (Var **) b;
+ var2 = castNode(Var, *var2_p);
+
+ if (var1->varno < var2->varno)
+ return -1;
+ else if (var1->varno > var2->varno)
+ return 1;
+
+ if (var1->varattno < var2->varattno)
+ return -1;
+ else if (var1->varattno > var2->varattno)
+ return 1;
+
+ return 0;
+ }
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 987c20a..449064f
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,19 ****
--- 14,20 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
***************
*** 26,31 ****
--- 27,33 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 45,50 ****
--- 47,54 ----
} PostponedQual;
+ static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+ static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** add_vars_to_targetlist(PlannerInfo *root
*** 240,245 ****
--- 244,763 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression and setup GroupedPathInfo for each base
+ * relation that can product grouped paths.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouping_info_to_base_rels(PlannerInfo *root)
+ {
+ int i;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause || root->group_pathkeys == NIL)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * TODO Consider if this is a real limitation.
+ */
+ if (root->parse->hasWindowFuncs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Is no grouping is possible below the top-level join? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+
+ /*
+ * "other rels" will have their targets built later, by translation of
+ * the target of the parent rel - see set_append_rel_size. If we
+ * wanted to prepare the child rels here, we'd need another iteration
+ * of simple_rel_array_size.
+ */
+ if (rel != NULL && rel->reloptkind == RELOPT_BASEREL)
+ prepare_rel_for_grouping(root, rel);
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+ static void
+ create_aggregate_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /*
+ * Aggregation push-down does not work w/o aggcombinefn. This field is
+ * not mandatory, so check if this particular aggregate can handle
+ * partial aggregation.
+ */
+ if (!OidIsValid(aggref->aggcombinefn))
+ {
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ /* TODO Initialize gv_width. */
+ gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * group_pathkeys is the source of grouping expressions, as opposed to the
+ * actual GROUP BY clause. That increases the chance to get the relation
+ * output grouped.
+ */
+ static void
+ create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+ {
+ ListCell *l1;
+
+ /*
+ * For each pathkey (or rather its equivalence class), find the grouping
+ * clause to find out the corresponding sortgroupref. Then create
+ * GroupedVarInfo for each member of that EC.
+ */
+ foreach(l1, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l1);
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *l2;
+ EquivalenceMember *em;
+ Index sortgroupref = 0;
+
+ /* We need equality anywhere in the join tree. */
+ if (ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * TODO Reconsider this restriction. As the grouping expression is
+ * only evaluated at the relation level (and only the result will be
+ * propagated to the final targetlist), volatile function might be
+ * o.k. Need to think what volatile EC exactly means.
+ */
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(l2, ec->ec_members)
+ {
+ ListCell *l3;
+
+ em = lfirst_node(EquivalenceMember, l2);
+
+ if (em->em_nullable_relids)
+ continue;
+
+ /*
+ * If target list contains this expression, it should provide us
+ * with the sortgroupref.
+ */
+ foreach(l3, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, l3);
+
+ if (equal(em->em_expr, te->expr))
+ {
+ /*
+ * XXX Not sure if the same expression exist in the
+ * targetlist multiple times and if some occurrences can
+ * miss the ressortgroupref. Maybe we just need to
+ * Assert() here that ressortgroupref is non-zero.
+ */
+ if (te->ressortgroupref > 0)
+ {
+ sortgroupref = te->ressortgroupref;
+ break;
+ }
+ }
+ }
+
+ /*
+ * If a single EC member matches, no need to check the rest of the
+ * EC.
+ */
+ if (sortgroupref > 0)
+ break;
+ }
+
+ if (sortgroupref == 0)
+ /* Go for the next EC. */
+ continue;
+
+ /*
+ * The EC contains a grouping expression, so each member of that EC
+ * should be usable as grouping key. Iterate the EC once again and
+ * create GroupedVarInfo for each member.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ em = lfirst_node(EquivalenceMember, l2);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(em->em_expr);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the aggregate should be evaluated. */
+ gvi->gv_eval_at = pull_varnos((Node *) em->em_expr);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+ }
+
+ /*
+ * Check if rel->reltarget allows the relation to be grouped and initialize
+ * the grouping target(s).
+ *
+ * target_agg is a target that we'll eventually used to aggregate the output
+ * of relation paths.
+ *
+ * target_mixed can be passed to create a grouping target that --- besides the
+ * grouping expressions --- contains expressions which can appear in the
+ * target of a "mixed grouped join" (i.e. join of a grouped relation to
+ * non-grouped one) but cannot be used as grouping keys. Such expressions are
+ * supposedly needed by join expressions higher in the join tree.
+ *
+ * If we succeed to create the grouping target, also replace rel->reltarget
+ * with a new one that has sortgrouprefs initialized --- this is necessary for
+ * create_agg_plan to match the grouping clauses against the input target
+ * expressions. (Thus the scan / join that provides the AggPath with input
+ * does not have to care whether the source relation does have grouped target
+ * or not.)
+ *
+ * The *group_exprs_extra_p list may receive additional grouping expressions
+ * that the query does not have. These can make the aggregation of base
+ * relation / join less efficient, but it may still be better than not trying
+ * at all.
+ *
+ * TODO Make sure cost / width of both "result" and "plain" are correct.
+ */
+ void
+ initialize_grouped_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target_agg, PathTarget *target_mixed,
+ List **group_exprs_extra_p)
+ {
+ PathTarget *target_plain;
+ ListCell *lc1;
+ List *vars_unresolved = NIL;
+
+ /* The target to replace rel->reltarget. */
+ target_plain = create_empty_pathtarget();
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ GroupedVar *gvar;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var. (As for generic grouping expressions,
+ * add_grouped_vars_to_target will retrieve them from the query
+ * targetlist and add them to target_agg outside this function.)
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ gvar = get_grouping_expression(root, (Expr *) tvar);
+ if (gvar != NULL)
+ {
+ /*
+ * It's o.k. to use the target expression for grouping.
+ *
+ * The actual Var is added to the target. If we used the
+ * containing GroupedVar, references from various clauses (e.g.
+ * join quals) wouldn't work.
+ */
+ add_column_to_pathtarget(target_agg, (Expr *) gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /*
+ * The "mixed target" needs grouping expressions too.
+ */
+ if (target_mixed)
+ add_column_to_pathtarget(target_mixed, (Expr *) gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /*
+ * As for the plain target, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(target_plain, gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the plain-var grouping expressions gathered. So far
+ * only store the var in a list.
+ */
+ vars_unresolved = lappend(vars_unresolved, tvar);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ foreach(lc1, vars_unresolved)
+ {
+ Var *var;
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ var = lfirst_node(Var, lc1);
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Dependent var is almost the same as one that has sortgroupref.
+ */
+ if (check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target_agg->exprs, &deps))
+ {
+
+ Index sortgroupref = 0;
+ int i = 0;
+ ListCell *lc2;
+
+ /*
+ * Find the expression the current item depends on and use its
+ * sortgroupref.
+ */
+ Assert(target_agg->sortgrouprefs != NULL);
+ foreach(lc2, target_agg->exprs)
+ {
+ Var *var_tmp = lfirst_node(Var, lc2);
+
+ if (var_tmp->varno == var->varno &&
+ var_tmp->varlevelsup == var->varlevelsup)
+ {
+ sortgroupref = target_agg->sortgrouprefs[i];
+ break;
+ }
+ i++;
+ }
+ Assert(sortgroupref > 0);
+
+ add_column_to_pathtarget(target_agg, (Expr *) var, sortgroupref);
+
+ if (target_mixed)
+ add_column_to_pathtarget(target_mixed, (Expr *) var,
+ sortgroupref);
+
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important. But once we have it ...
+ */
+ add_column_to_pathtarget(target_plain, (Expr *) var,
+ sortgroupref);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) or aggregate arguments, but we already have it in
+ * the targets, so don't care.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon as GroupedVars are installed, the top level
+ * aggregates / grouping expressions no longer need direct reference
+ * to arg_var anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by upper join. This includes one that is
+ * referenced by a generic grouping expression but couldn't be
+ * recognized as grouping expression on its own at the top of the
+ * loop.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ *
+ * Since root->parse->groupClause is not supposed to contain this
+ * expression, we need construct special SortGroupClause. Its
+ * tleSortGroupRef needs to be unique within target_agg, so
+ * postpone creation of the SortGroupRefs until we're done with
+ * the iteration of rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+
+ /*
+ * "join-only" var like this is the purpose of target_mixed.
+ */
+ if (target_mixed != NULL)
+ add_column_to_pathtarget(target_mixed, (Expr *) var, 0);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced either by aggregate argument
+ * or by generic grouping expression. In either case, neither
+ * target_agg nor target_mixed should contain it, as they only
+ * provide input for the *final* aggregation.
+ */
+ }
+
+ /*
+ * The var is not suitable for grouping, but the plain target ought to
+ * stay complete.
+ */
+ add_column_to_pathtarget(target_plain, (Expr *) var, 0);
+ }
+
+ /*
+ * Apply the adjusted input target as the replacement is complete now.
+ */
+ /* TODO Free the old one. */
+ rel->reltarget = target_plain;
+ }
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index f4e0a6e..df93356
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 177,182 ****
--- 177,190 ----
(*qp_callback) (root, qp_extra);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, Initialize GroupedPathInfo
+ * of base relations capable to do the grouping and setup
+ * root->grouped_var_list.
+ */
+ add_grouping_info_to_base_rels(root);
+
+ /*
* Examine any "placeholder" expressions generated during subquery pullup.
* Make sure that the Vars they need are marked as needed at the relevant
* join level. This must be done before join removal because it might
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 8ad0b4a..61387a2
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
*************** find_param_path_info(RelOptInfo *rel, Re
*** 1366,1368 ****
--- 1366,1697 ----
return NULL;
}
+
+ /*
+ * If the relation can produce grouped paths, create GroupedPathInfo for it
+ * and create target for aggregation of the relation output.
+ *
+ * target_join_only will receive expressions that the grouping target may
+ * contain just to satisfy join conditions when joining this (grouped) rel to
+ * another one.
+ */
+ void
+ prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *gvis;
+ List *aggregates = NIL;
+ List *grp_exprs = NIL;
+ bool found_higher;
+ ListCell *lc;
+ GroupedPathInfo *gpi;
+ PathTarget *target_agg,
+ *target_mixed;
+ List *grp_exprs_extra = NIL;
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some grouping expression or Aggref
+ * argument), we'd just let initialize_grouped_targets create GroupedVar
+ * for the corresponding expression (phexpr). On the other hand, if we
+ * knew that the PHV is evaluated below the current rel, we'd ignore it
+ * because the referencing GroupedVar would take care of propagation of
+ * the value to upper joins. (PHV whose ph_eval_at is above the current
+ * rel make the aggregation push-down impossible in any case because the
+ * partial aggregation would receive wrong input if we ignored the
+ * ph_eval_at.)
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return;
+ }
+
+ /*
+ * target_agg will be used to aggregate the output of the current
+ * relation's paths.
+ */
+ target_agg = create_empty_pathtarget();
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return;
+
+ /*
+ * Join-only expression is not expected in a target of a grouped base
+ * relation because that is essentially an output of aggregation. Thus
+ * rel->reltarget can contain nothing but aggregate arguments and
+ * grouping expressions. They'll all end up in target_agg, so there's
+ * no need for a separate target.
+ */
+ target_mixed = NULL;
+ }
+ else
+ {
+ /*
+ * Join-only expression can come from a plain relation, if rel is a
+ * join of grouped and plain relation.
+ */
+ target_mixed = create_empty_pathtarget();
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * TODO For RELOPT_JOINREL, do not return if all the joins that can set
+ * any entry of the grouped target (do we need to postpone this check
+ * until the grouped target is available, and
+ * initialize_grouped_targets_target take care?) of this rel to NULL are
+ * provably below rel. (It's ok if rel is one of these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * While create_grouping_expr_grouped_var_infos generates equivalence
+ * class implied plain-Var grouping expressions at once, doing so for
+ * generic expressions would be complex and might result in very long list
+ * to search in. So generate the EC-related grouping expressions for each
+ * particular set of relids.
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ /*
+ * Should have been processed by
+ * create_grouping_expr_grouped_var_infos.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ gvi = translate_expression_to_rels(root, gvi, rel->relids);
+ if (gvi != NULL)
+ gvis = lappend(gvis, gvi);
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_higher = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * initialize_grouped_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ *
+ * XXX A plain Var could actually be handled w/o GroupedVar, but
+ * thus initialize_grouped_targets would have to spend extra
+ * effort looking for the EC-related vars, instead of relying on
+ * create_grouping_expr_grouped_var_infos. (Processing of
+ * particular expression would look different, so we could hardly
+ * reuse the same piece of code.)
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Accept the aggregate / grouping expression.
+ *
+ * (GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.)
+ */
+ if (IsA(gvi->gvexpr, Aggref))
+ aggregates = lappend(aggregates, gvi);
+ else
+ grp_exprs = lappend(grp_exprs, gvi);
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * Remember that there is at least one aggregate / grouping
+ * expression that needs more than this rel.
+ */
+ found_higher = true;
+ }
+ }
+ list_free(gvis);
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ *
+ * And even if at least one aggregate was found, give up if some other
+ * aggregate(s) need multiple relations including the current one. The
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * In contrast, grp_exprs is only supposed to contain generic grouping
+ * expression, so it can be NIL so far. If all the grouping keys are just
+ * plain Vars, initialize_grouped_targets will take care of them.
+ */
+ if (aggregates == NIL || found_higher)
+ return;
+
+ /*
+ * Initialize GroupedPathInfo.
+ */
+ Assert(rel->gpi == NULL);
+ gpi = makeNode(GroupedPathInfo);
+ gpi->pathlist = NIL;
+ gpi->partial_pathlist = NIL;
+ rel->gpi = gpi;
+
+ /*
+ * Add plain-var grouping expressions to the target(s).
+ */
+ initialize_grouped_targets(root, rel, target_agg, target_mixed,
+ &grp_exprs_extra);
+
+
+ /*
+ * Add (non-Var) grouping expressions (in the form of GroupedVar) to
+ * target_agg.
+ *
+ * Follow the convention that the grouping expressions should precede the
+ * aggregates.
+ */
+ add_grouped_vars_to_target(root, target_agg, grp_exprs);
+
+ /*
+ * Partial aggregation makes no sense w/o grouping expressions.
+ */
+ if (list_length(target_agg->exprs) == 0)
+ {
+ pfree(rel->gpi);
+ rel->gpi = NULL;
+ return;
+ }
+
+ /*
+ * If the aggregation target should have extra grouping expressions, add
+ * them now. This step includes assignment of tleSortGroupRef's which we
+ * can generate now (the "ordinary" grouping expression are present in the
+ * target by now).
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ int i;
+ Index sortgroupref_max = 0;
+
+ /*
+ * Identify the maximum sortgroupref.
+ */
+ Assert(target_agg->sortgrouprefs != NULL ||
+ list_length(target_agg->exprs) == 0);
+ for (i = 0; i < list_length(target_agg->exprs); i++)
+ {
+ Index sortgroupref = target_agg->sortgrouprefs[i];
+
+ if (sortgroupref > sortgroupref_max)
+ sortgroupref_max = sortgroupref;
+ }
+
+ /*
+ * Finally generate the SortGroupClause's and add the expressions to
+ * the target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * TODO Verify that these fields are sufficient for this special
+ * SortGroupClause.
+ */
+ cl->tleSortGroupRef = ++sortgroupref_max;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ NULL, &cl->eqop, NULL,
+ &cl->hashable);
+ gpi->sortgroupclauses = lappend(gpi->sortgroupclauses, cl);
+ add_column_to_pathtarget(target_agg, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ *
+ * TODO If the target already contains this var for another reason
+ * (e.g. a an input for generic grouping expression), it'll
+ * probably not have sortgrouprefs set. Consider removing that
+ * (and adjust target width accordingly) so that the var is not
+ * duplicated.
+ */
+ add_column_to_pathtarget(rel->reltarget, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the target(s).
+ */
+ add_grouped_vars_to_target(root, target_agg, aggregates);
+
+ /* TODO Check if copy is needed here. */
+ gpi->target_agg = copy_pathtarget(target_agg);
+
+ /*
+ * If the relation can be a join of grouped and non-grouped input
+ * relation, initialize target for it.
+ */
+ if (target_mixed)
+ {
+ add_grouped_vars_to_target(root, target_mixed, grp_exprs);
+ add_grouped_vars_to_target(root, target_mixed, aggregates);
+ gpi->target = target_mixed;
+ }
+
+ /*
+ * Add the non-Var group expressions to a separate target. If
+ * rel->reltarget should be used to generate input for partial
+ * aggregation, the corresponding path should include these expressions.
+ */
+ if (list_length(grp_exprs) > 0)
+ {
+ rel->gpi->group_exprs = create_empty_pathtarget();
+ add_grouped_vars_to_target(root, rel->gpi->group_exprs, grp_exprs);
+ }
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 9345891..abdc5d9
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 783,853 ----
}
/*
+ * For each aggregate add GroupedVar to the grouped target.
+ *
+ * Caller passes the aggregates in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+ void
+ add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+ {
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+ GroupedVar *gvar;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_column_to_pathtarget(target, (Expr *) gvar, gvi->sortgroupref);
+ }
+ }
+
+ /*
+ * Return GroupedVar containing the passed-in expression if one exists, or
+ * NULL if the expression cannot be used as grouping key.
+ *
+ * create_grouping_expr_grouped_var_infos should have been called before this
+ * function gets called the first time.
+ */
+ GroupedVar *
+ get_grouping_expression(PlannerInfo *root, Expr *expr)
+ {
+ ListCell *lc;
+
+ /* Caller should have noticed the empty list much earlier. */
+ Assert(root->grouped_var_list != NIL);
+
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ GroupedVar *result = makeNode(GroupedVar);
+
+ Assert(gvi->sortgroupref > 0);
+ result->gvexpr = gvi->gvexpr;
+ result->gvid = gvi->gvid;
+ result->sortgroupref = gvi->sortgroupref;
+ return result;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return NULL;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
new file mode 100644
index e367221..7090d02
*** a/src/include/optimizer/clauses.h
--- b/src/include/optimizer/clauses.h
*************** extern Node *estimate_expression_value(P
*** 84,88 ****
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
!
#endif /* CLAUSES_H */
--- 84,90 ----
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
! extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
! GroupedVarInfo *gvi,
! Relids relids);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index e372f88..4d71ce3
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern ParamPathInfo *get_appendrel_para
*** 290,294 ****
--- 290,295 ----
Relids required_outer);
extern ParamPathInfo *find_param_path_info(RelOptInfo *rel,
Relids required_outer);
+ extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index f1d16cf..d3dbf51
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** extern void add_base_rels_to_query(Plann
*** 75,80 ****
--- 75,82 ----
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+ extern void add_grouping_info_to_base_rels(PlannerInfo *root);
+ extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 0d3ec92..7752497
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern Node *get_sortgroupclause_expr(So
*** 41,46 ****
--- 41,50 ----
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
+ extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List *target_group_clauses,
+ List **grouping_clauses,
+ List **grouping_exprs, List **agg_exprs);
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 69,83 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+ extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
+ extern void initialize_grouped_targets(PlannerInfo *root,
+ RelOptInfo *rel,
+ PathTarget *target_agg,
+ PathTarget *target_mixed,
+ List **group_exprs_extra_p);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
agg_pushdown_v3/03_add_path.diff
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
new file mode 100644
index 2396bd4..822c350
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
*************** fileGetForeignPaths(PlannerInfo *root,
*** 551,557 ****
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! coptions));
/*
* If data file was sorted, and we knew it somehow, we could insert
--- 551,557 ----
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! coptions), false);
/*
* If data file was sorted, and we knew it somehow, we could insert
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
new file mode 100644
index d77c2a7..c379f2f
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 907,913 ****
NULL, /* no outer rel either */
NULL, /* no extra plan */
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path);
/* Add paths with pathkeys */
add_paths_with_pathkeys_for_rel(root, baserel, NULL);
--- 907,913 ----
NULL, /* no outer rel either */
NULL, /* no extra plan */
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
/* Add paths with pathkeys */
add_paths_with_pathkeys_for_rel(root, baserel, NULL);
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 1079,1085 ****
param_info->ppi_req_outer,
NULL,
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path);
}
}
--- 1079,1085 ----
param_info->ppi_req_outer,
NULL,
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
}
}
*************** add_paths_with_pathkeys_for_rel(PlannerI
*** 4342,4348 ****
useful_pathkeys,
NULL,
epq_path,
! NIL));
}
}
--- 4342,4349 ----
useful_pathkeys,
NULL,
epq_path,
! NIL),
! false);
}
}
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4574,4580 ****
NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath);
/* Consider pathkeys for the join relation */
add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
--- 4575,4581 ----
NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, false);
/* Consider pathkeys for the join relation */
add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4898,4904 ****
NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
! add_path(grouped_rel, (Path *) grouppath);
}
/*
--- 4899,4905 ----
NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
! add_path(grouped_rel, (Path *) grouppath, false);
}
/*
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 2d7e1d8..266ecba
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 696,702 ****
required_outer = rel->lateral_relids;
/* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
/* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
--- 696,702 ----
required_outer = rel->lateral_relids;
/* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0), false);
/* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
*************** create_plain_partial_paths(PlannerInfo *
*** 725,731 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 725,732 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers),
! false);
}
/*
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 811,817 ****
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path);
/* For the moment, at least, there are no other paths to consider */
}
--- 812,818 ----
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path, false);
/* For the moment, at least, there are no other paths to consider */
}
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1396,1402 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels));
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1397,1403 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels), false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1424,1430 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1425,1431 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath, false);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1477,1483 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels));
}
}
--- 1478,1484 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels), false);
}
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1573,1586 ****
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels));
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels));
}
}
--- 1574,1589 ----
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
}
}
*************** set_dummy_rel_pathlist(RelOptInfo *rel)
*** 1713,1719 ****
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
--- 1716,1722 ----
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
*************** set_subquery_pathlist(PlannerInfo *root,
*** 1927,1933 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer));
}
}
--- 1930,1937 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer),
! false);
}
}
*************** set_function_pathlist(PlannerInfo *root,
*** 1996,2002 ****
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer));
}
/*
--- 2000,2006 ----
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer), false);
}
/*
*************** set_values_pathlist(PlannerInfo *root, R
*** 2016,2022 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer));
}
/*
--- 2020,2026 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer), false);
}
/*
*************** set_tablefunc_pathlist(PlannerInfo *root
*** 2037,2043 ****
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer));
}
/*
--- 2041,2047 ----
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer), false);
}
/*
*************** set_cte_pathlist(PlannerInfo *root, RelO
*** 2103,2109 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer));
}
/*
--- 2107,2113 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer), false);
}
/*
*************** set_namedtuplestore_pathlist(PlannerInfo
*** 2130,2136 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
--- 2134,2141 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
*************** set_worktable_pathlist(PlannerInfo *root
*** 2183,2189 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer));
}
/*
--- 2188,2195 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer),
! false);
}
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2215,2221 ****
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path);
/*
* For each useful ordering, we can consider an order-preserving Gather
--- 2221,2227 ----
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path, false);
/*
* For each useful ordering, we can consider an order-preserving Gather
*************** generate_gather_paths(PlannerInfo *root,
*** 2231,2237 ****
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path);
}
}
--- 2237,2243 ----
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, false);
}
}
*************** create_partial_bitmap_paths(PlannerInfo
*** 3048,3054 ****
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3054,3061 ----
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers),
! false);
}
/*
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index f353803..1dea66c
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
*************** create_index_paths(PlannerInfo *root, Re
*** 338,344 ****
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
--- 338,344 ----
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath, false);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
*************** create_index_paths(PlannerInfo *root, Re
*** 415,421 ****
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath);
}
}
}
--- 415,421 ----
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath, false);
}
}
}
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,795 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath);
if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
--- 789,795 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, false);
if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1077,1083 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1129,1135 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 43833ea..d6669c4
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** try_nestloop_path(PlannerInfo *root,
*** 385,391 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
add_path(joinrel, (Path *)
create_nestloop_path(root,
--- 385,391 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
add_path(joinrel, (Path *)
create_nestloop_path(root,
*************** try_nestloop_path(PlannerInfo *root,
*** 397,403 ****
inner_path,
extra->restrictlist,
pathkeys,
! required_outer));
}
else
{
--- 397,403 ----
inner_path,
extra->restrictlist,
pathkeys,
! required_outer), false);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 443,449 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/* Might be good enough to be worth trying, so let's try it. */
--- 443,450 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/* Might be good enough to be worth trying, so let's try it. */
*************** try_partial_nestloop_path(PlannerInfo *r
*** 457,463 ****
inner_path,
extra->restrictlist,
pathkeys,
! NULL));
}
/*
--- 458,464 ----
inner_path,
extra->restrictlist,
pathkeys,
! NULL), false);
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 531,537 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
--- 532,538 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
*************** try_mergejoin_path(PlannerInfo *root,
*** 546,552 ****
required_outer,
mergeclauses,
outersortkeys,
! innersortkeys));
}
else
{
--- 547,553 ----
required_outer,
mergeclauses,
outersortkeys,
! innersortkeys), false);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 605,611 ****
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/* Might be good enough to be worth trying, so let's try it. */
--- 606,613 ----
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/* Might be good enough to be worth trying, so let's try it. */
*************** try_partial_mergejoin_path(PlannerInfo *
*** 622,628 ****
NULL,
mergeclauses,
outersortkeys,
! innersortkeys));
}
/*
--- 624,630 ----
NULL,
mergeclauses,
outersortkeys,
! innersortkeys), false);
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 665,671 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
add_path(joinrel, (Path *)
create_hashjoin_path(root,
--- 667,673 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, false))
{
add_path(joinrel, (Path *)
create_hashjoin_path(root,
*************** try_hashjoin_path(PlannerInfo *root,
*** 677,683 ****
inner_path,
extra->restrictlist,
required_outer,
! hashclauses));
}
else
{
--- 679,685 ----
inner_path,
extra->restrictlist,
required_outer,
! hashclauses), false);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 723,729 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
/* Might be good enough to be worth trying, so let's try it. */
--- 725,731 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL, false))
return;
/* Might be good enough to be worth trying, so let's try it. */
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 737,743 ****
inner_path,
extra->restrictlist,
NULL,
! hashclauses));
}
/*
--- 739,745 ----
inner_path,
extra->restrictlist,
NULL,
! hashclauses), false);
}
/*
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 6ee2350..c4aad20
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** mark_dummy_rel(RelOptInfo *rel)
*** 1217,1223 ****
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
--- 1217,1223 ----
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index a2fe661..91d855c
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** create_tidscan_paths(PlannerInfo *root,
*** 266,270 ****
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 266,270 ----
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer), false);
}
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index bba8a1f..51d13ab
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,229 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual));
}
/*
--- 223,230 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual),
! false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index df93356..c9040ce
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,89 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,90 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 407df9a..ce5ce6c
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** inheritance_planner(PlannerInfo *root)
*** 1422,1428 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)));
}
/*--------------------
--- 1422,1429 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)),
! false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 2043,2049 ****
}
/* And shove it into final_rel */
! add_path(final_rel, path);
}
/*
--- 2044,2050 ----
}
/* And shove it into final_rel */
! add_path(final_rel, path, false);
}
/*
*************** create_grouping_paths(PlannerInfo *root,
*** 3603,3609 ****
(List *) parse->havingQual);
}
! add_path(grouped_rel, path);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
--- 3604,3610 ----
(List *) parse->havingQual);
}
! add_path(grouped_rel, path, false);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
*************** create_grouping_paths(PlannerInfo *root,
*** 3780,3786 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
--- 3781,3788 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3789,3795 ****
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups));
}
}
}
--- 3791,3798 ----
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3820,3826 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
}
}
}
--- 3823,3829 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3872,3878 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
else if (parse->groupClause)
{
--- 3875,3881 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
else if (parse->groupClause)
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3887,3893 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
else
{
--- 3890,3896 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
else
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3936,3942 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3939,3945 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3945,3951 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
/*
* The point of using Gather Merge rather than Gather is that it
--- 3948,3954 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
/*
* The point of using Gather Merge rather than Gather is that it
*************** create_grouping_paths(PlannerInfo *root,
*** 3998,4004 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 4001,4007 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4007,4013 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
}
}
--- 4010,4016 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4052,4058 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
}
--- 4055,4061 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4090,4096 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
}
}
}
--- 4093,4099 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
}
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4292,4298 ****
strat,
new_rollups,
agg_costs,
! dNumGroups));
return;
}
--- 4295,4301 ----
strat,
new_rollups,
agg_costs,
! dNumGroups), false);
return;
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4450,4456 ****
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups));
}
}
--- 4453,4459 ----
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups), false);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4467,4473 ****
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups));
}
/*
--- 4470,4476 ----
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups), false);
}
/*
*************** create_one_window_path(PlannerInfo *root
*** 4652,4658 ****
window_pathkeys);
}
! add_path(window_rel, path);
}
/*
--- 4655,4661 ----
window_pathkeys);
}
! add_path(window_rel, path, false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4758,4764 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
}
--- 4761,4767 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4785,4791 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
/*
--- 4788,4794 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4832,4838 ****
parse->distinctClause,
NIL,
NULL,
! numDistinctRows));
}
/* Give a helpful error if we failed to find any implementation */
--- 4835,4841 ----
parse->distinctClause,
NIL,
NULL,
! numDistinctRows), false);
}
/* Give a helpful error if we failed to find any implementation */
*************** create_ordered_paths(PlannerInfo *root,
*** 4930,4936 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 4933,4939 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
*************** create_ordered_paths(PlannerInfo *root,
*** 4980,4986 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 4983,4989 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
new file mode 100644
index 6d8f893..f756920
*** a/src/backend/optimizer/prep/prepunion.c
--- b/src/backend/optimizer/prep/prepunion.c
*************** plan_set_operations(PlannerInfo *root)
*** 209,215 ****
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
--- 209,215 ----
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path, false);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 26567cb..a27311d
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
*************** set_cheapest(RelOptInfo *parent_rel)
*** 409,416 ****
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
--- 409,417 ----
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
+ List *pathlist;
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 427,432 ****
--- 428,441 ----
/* Pretend parameterized paths have no pathkeys, per comment above */
new_path_pathkeys = new_path->param_info ? NIL : new_path->pathkeys;
+ if (!grouped)
+ pathlist = parent_rel->pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->pathlist;
+ }
+
/*
* Loop to check proposed new path against old paths. Note it is possible
* for more than one old path to be tossed out because new_path dominates
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 436,442 ****
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
--- 445,451 ----
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 582,589 ****
*/
if (remove_old)
{
! parent_rel->pathlist = list_delete_cell(parent_rel->pathlist,
! p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
--- 591,597 ----
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 614,622 ****
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(parent_rel->pathlist, insert_after, new_path);
else
! parent_rel->pathlist = lcons(new_path, parent_rel->pathlist);
}
else
{
--- 622,635 ----
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->pathlist = pathlist;
! else
! parent_rel->gpi->pathlist = pathlist;
}
else
{
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 646,653 ****
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer)
{
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
--- 659,667 ----
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped)
{
+ List *pathlist;
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 656,664 ****
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup : parent_rel->consider_startup;
! foreach(p1, parent_rel->pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 670,687 ----
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup :
! parent_rel->consider_startup;
! if (!grouped)
! pathlist = parent_rel->pathlist;
! else
! {
! Assert(parent_rel->gpi != NULL);
! pathlist = parent_rel->gpi->pathlist;
! }
!
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 749,771 ****
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->partial_pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
--- 772,803 ----
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
+ List *pathlist;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
+
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
*************** add_partial_path(RelOptInfo *parent_rel,
*** 819,830 ****
}
/*
! * Remove current element from partial_pathlist if dominated by new.
*/
if (remove_old)
{
! parent_rel->partial_pathlist =
! list_delete_cell(parent_rel->partial_pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
--- 851,861 ----
}
/*
! * Remove current element from pathlist if dominated by new.
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
*************** add_partial_path(RelOptInfo *parent_rel,
*** 839,846 ****
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the partial_pathlist; we will not add new_path, and we
! * assume new_path cannot dominate any later path.
*/
if (!accept_new)
break;
--- 870,877 ----
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the pathlist; we will not add new_path, and we assume
! * new_path cannot dominate any later path.
*/
if (!accept_new)
break;
*************** add_partial_path(RelOptInfo *parent_rel,
*** 850,859 ****
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(parent_rel->partial_pathlist, insert_after, new_path);
else
! parent_rel->partial_pathlist =
! lcons(new_path, parent_rel->partial_pathlist);
}
else
{
--- 881,894 ----
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->partial_pathlist = pathlist;
! else
! parent_rel->gpi->partial_pathlist = pathlist;
}
else
{
*************** add_partial_path(RelOptInfo *parent_rel,
*** 874,882 ****
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys)
{
ListCell *p1;
/*
* Our goal here is twofold. First, we want to find out whether this path
--- 909,926 ----
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys, bool grouped)
{
ListCell *p1;
+ List *pathlist;
+
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
/*
* Our goal here is twofold. First, we want to find out whether this path
*************** add_partial_path_precheck(RelOptInfo *pa
*** 886,895 ****
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist to be very short, and getting a definitive
! * answer at this stage avoids the need to call add_path_precheck.
*/
! foreach(p1, parent_rel->partial_pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 930,940 ----
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist / grouped_pathlist to be very short, and
! * getting a definitive answer at this stage avoids the need to call
! * add_path_precheck.
*/
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_partial_path_precheck(RelOptInfo *pa
*** 918,924 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL))
return false;
return true;
--- 963,969 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, false))
return false;
return true;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 4d71ce3..ef9f982
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern int compare_path_costs(Path *path
*** 25,37 ****
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
--- 25,39 ----
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path,
! bool grouped);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys,
! bool grouped);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
agg_pushdown_v3/04_main.diff
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index 7496189..e995881
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, PlanState *p
*** 723,728 ****
--- 723,759 ----
break;
}
+ case T_GroupedVar:
+
+ /*
+ * If GroupedVar appears in targetlist of Agg node, it can
+ * represent either Aggref or grouping expression.
+ */
+ if (parent && (IsA(parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * TODO Assert() that all Aggrefs of the AggState are partial:
+ * GroupedVar shouldn't find its way to the query targetlist.
+ */
+ if (IsA(gvar->gvexpr, Aggref))
+ ExecInitExprRec((Expr *) gvar->agg_partial, parent, state,
+ resv, resnull);
+ else
+ ExecInitExprRec((Expr *) gvar->gvexpr, parent, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ */
+ elog(ERROR, "parent of GroupedVar is not Agg node");
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
new file mode 100644
index 6a26773..7fccb86
*** a/src/backend/executor/nodeAgg.c
--- b/src/backend/executor/nodeAgg.c
*************** find_unaggregated_cols_walker(Node *node
*** 1833,1838 ****
--- 1833,1845 ----
/* do not descend into aggregate exprs */
return false;
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ if (IsA(gvar->gvexpr, Aggref))
+ return false;
+ }
return expression_tree_walker(node, find_unaggregated_cols_walker,
(void *) colnos);
}
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index b5cab0c..f89406d
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 265,271 ****
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
--- 265,271 ----
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 266ecba..9f3a7c0
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 486,492 ****
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! generate_gather_paths(root, rel);
/*
* Allow a plugin to editorialize on the set of Paths for this base
--- 486,495 ----
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! {
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
! }
/*
* Allow a plugin to editorialize on the set of Paths for this base
*************** static void
*** 687,692 ****
--- 690,696 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 695,709 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0), false);
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel);
/* Consider TID scans */
create_tidscan_paths(root, rel);
--- 699,727 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! add_path(rel, seq_path, false);
! if (rel->gpi != NULL && rel->gpi->target_agg != NULL &&
! required_outer == NULL)
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel, false);
! if (rel->gpi != NULL)
! {
! /*
! * TODO Instead of calling the whole clause-matching machinery twice
! * (there should be no difference between plain and grouped paths from
! * this point of view), consider returning a separate list of paths
! * usable as grouped ones.
! */
! create_index_paths(root, rel, true);
! }
/* Consider TID scans */
create_tidscan_paths(root, rel);
*************** static void
*** 717,722 ****
--- 735,741 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
parallel_workers = compute_parallel_worker(rel, rel->pages, -1);
*************** create_plain_partial_paths(PlannerInfo *
*** 725,732 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers),
! false);
}
/*
--- 744,874 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
! add_partial_path(rel, path, false);
!
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it.
! */
! if (rel->gpi != NULL)
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the
! * appropriate pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck().
! *
! * If "partial" is true, the resulting path is considered partial in terms of
! * parallel execution.
! *
! * The path we create here shouldn't be parameterized because of supposedly
! * high startup cost of aggregation (whether due to build of hash table for
! * AGG_HASHED strategy or due to explicit sort for AGG_SORTED).
! *
! * XXX IndexPath as an input for AGG_SORTED might seem to be an exception, but
! * aggregation of its output is only beneficial if it's performed by multiple
! * workers, i.e. the resulting path is partial (Besides parallel aggregation,
! * the other use case of aggregation push-down is aggregation performed on
! * remote database, but that has nothing to do with IndexScan). And partial
! * path cannot be parameterized because it's semantically wrong to use it on
! * the inner side of NL join.
! */
! void
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! List *group_clauses = NIL;
! List *group_exprs = NIL;
! List *agg_exprs = NIL;
! Path *agg_path;
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Grouped path should never be parameterized, so we're not supposed to
! * receive parameterized subpath.
! */
! Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
!
! /*
! * Non-var grouping expressions will eventually require projection using
! * Result plan, but that does not work with SRFs.
! */
! Assert(rel->gpi != NULL);
! if (rel->gpi->group_exprs != NULL)
! {
! ListCell *lc;
!
! foreach(lc, rel->gpi->group_exprs->exprs)
! {
! GroupedVar *gvar = lfirst_node(GroupedVar, lc);
!
! if (IsA(gvar->gvexpr, FuncExpr))
! {
! FuncExpr *fexpr = (FuncExpr *) gvar->gvexpr;
!
! if (fexpr->funcretset)
! return;
! }
! }
! }
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL,
! true))
! return;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost, pathkeys,
! true))
! return;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path, true);
! else
! add_partial_path(rel, (Path *) agg_path, true);
! }
}
/*
*************** set_append_rel_size(PlannerInfo *root, R
*** 1069,1074 ****
--- 1211,1270 ----
1, &appinfo);
/*
+ * Also the grouped target needs to be adjusted, if one exists.
+ */
+ if (rel->gpi != NULL)
+ {
+ GroupedPathInfo *gpi;
+
+ Assert(childrel->gpi == NULL);
+
+ /*
+ * prepare_rel_for_grouping does not set "target" for a base
+ * relation and it it clears the whole "gpi" if it can't create at
+ * least "target_agg".
+ */
+ Assert(rel->gpi->target == NULL);
+ Assert(rel->gpi->target_agg != NULL);
+
+ childrel->gpi = gpi = makeNode(GroupedPathInfo);
+ memcpy(gpi, rel->gpi, sizeof(GroupedPathInfo));
+
+ /*
+ * add_grouping_info_to_base_rels was not sure if grouping makes
+ * sense for the parent rel, so create a separate copy of the
+ * target now.
+ */
+ gpi->target_agg =
+ copy_pathtarget(childrel->gpi->target_agg);
+ Assert(gpi->target_agg->exprs != NIL);
+ gpi->target_agg->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->target_agg->exprs,
+ 1, &appinfo);
+
+ /*
+ * Non-var grouping expressions need to be translated as well.
+ */
+ if (gpi->group_exprs != NULL)
+ {
+ gpi->group_exprs = copy_pathtarget(gpi->group_exprs);
+ gpi->group_exprs->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->group_exprs->exprs,
+ 1, &appinfo);
+ }
+
+ /*
+ * If the output of the child rel's paths should be aggregated,
+ * sortgrouprefs are also needed. (initialize_grouped_targets
+ * should have initialized sortgrouprefs of the parent rel.)
+ */
+ childrel->reltarget->sortgrouprefs =
+ rel->reltarget->sortgrouprefs;
+ }
+
+ /*
* We have to make child entries in the EquivalenceClass data
* structures as well. This is needed either if the parent
* participates in some eclass joins (because we will want to consider
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1283,1288 ****
--- 1479,1486 ----
bool subpaths_valid = true;
List *partial_subpaths = NIL;
bool partial_subpaths_valid = true;
+ List *grouped_subpaths = NIL;
+ bool grouped_subpaths_valid = true;
List *all_child_pathkeys = NIL;
List *all_child_outers = NIL;
ListCell *l;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1326,1331 ****
--- 1524,1552 ----
partial_subpaths_valid = false;
/*
+ * For grouped paths, use only the unparameterized subpaths.
+ *
+ * XXX Consider if the parameterized subpaths should be processed
+ * below. It's probably not useful for sequential scans (due to
+ * repeated aggregation), but might be worthwhile for other child
+ * nodes.
+ */
+ if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
+ {
+ Path *path;
+
+ path = (Path *) linitial(childrel->gpi->pathlist);
+ if (path->param_info == NULL)
+ grouped_subpaths = accumulate_append_subpath(grouped_subpaths,
+ path);
+ else
+ grouped_subpaths_valid = false;
+ }
+ else
+ grouped_subpaths_valid = false;
+
+
+ /*
* Collect lists of all the available path orderings and
* parameterizations for all the children. We use these as a
* heuristic to indicate which sort orderings and parameterizations we
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1397,1403 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels), false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1618,1625 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels),
! false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1424,1433 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers, partitioned_rels);
add_partial_path(rel, (Path *) appendpath, false);
}
/*
* Also build unparameterized MergeAppend paths based on the collected
* list of child pathkeys.
--- 1646,1668 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers,
! partitioned_rels);
add_partial_path(rel, (Path *) appendpath, false);
}
+ /* TODO Also partial grouped paths? */
+ if (grouped_subpaths_valid)
+ {
+ Path *path;
+
+ path = (Path *) create_append_path(rel, grouped_subpaths, NULL, 0,
+ partitioned_rels);
+ /* pathtarget will produce the grouped relation.. */
+ path->pathtarget = rel->gpi->target_agg;
+ add_path(rel, path, true);
+ }
+
/*
* Also build unparameterized MergeAppend paths based on the collected
* list of child pathkeys.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1478,1484 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels), false);
}
}
--- 1713,1720 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels),
! false);
}
}
*************** set_subquery_pathlist(PlannerInfo *root,
*** 1930,1937 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer),
! false);
}
}
--- 2166,2172 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer), false);
}
}
*************** set_worktable_pathlist(PlannerInfo *root
*** 2202,2215 ****
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
ListCell *lc;
/* If there are no partial paths, there's nothing to do here. */
! if (rel->partial_pathlist == NIL)
return;
/*
--- 2437,2457 ----
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
+ List *pathlist = NIL;
+ PathTarget *partial_target;
ListCell *lc;
+ if (!grouped)
+ pathlist = rel->partial_pathlist;
+ else if (rel->gpi != NULL)
+ pathlist = rel->gpi->partial_pathlist;
+
/* If there are no partial paths, there's nothing to do here. */
! if (pathlist == NIL)
return;
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2217,2233 ****
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(rel->partial_pathlist);
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path, false);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach(lc, rel->partial_pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
--- 2459,2481 ----
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(pathlist);
!
! if (!grouped)
! partial_target = rel->reltarget;
! else if (rel->gpi != NULL && rel->gpi->target != NULL)
! partial_target = rel->gpi->target;
!
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, partial_target,
NULL, NULL);
! add_path(rel, simple_gather_path, grouped);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach(lc, pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
*************** generate_gather_paths(PlannerInfo *root,
*** 2235,2243 ****
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, false);
}
}
--- 2483,2491 ----
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, partial_target,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, grouped);
}
}
*************** standard_join_search(PlannerInfo *root,
*** 2403,2409 ****
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
--- 2651,2658 ----
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 1dea66c..b34bf14
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** static bool eclass_already_used(Equivale
*** 107,119 ****
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
--- 108,121 ----
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! bool grouped);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
*************** static Const *string_to_const(const char
*** 229,235 ****
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel)
{
List *indexpaths;
List *bitindexpaths;
--- 231,237 ----
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
List *indexpaths;
List *bitindexpaths;
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,283 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
! grouped);
/*
* Identify the join clauses that can match the index. For the moment
*************** get_join_index_paths(PlannerInfo *root,
*** 667,673 ****
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
--- 669,675 ----
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths, false);
/*
* Remember we considered paths for this set of relids. We use lcons not
*************** bms_equal_any(Relids relids, List *relid
*** 736,742 ****
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths)
{
List *indexpaths;
bool skip_nonnative_saop = false;
--- 738,744 ----
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped)
{
List *indexpaths;
bool skip_nonnative_saop = false;
*************** get_index_paths(PlannerInfo *root, RelOp
*** 754,760 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
--- 756,762 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop, grouped);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
*************** get_index_paths(PlannerInfo *root, RelOp
*** 769,775 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL));
}
/*
--- 771,777 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL, grouped));
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,797 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, false);
! if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
--- 791,799 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, grouped);
! if (!grouped && index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
*************** get_index_paths(PlannerInfo *root, RelOp
*** 802,815 ****
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
--- 804,818 ----
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (!grouped && skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 861,867 ****
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop)
{
List *result = NIL;
IndexPath *ipath;
--- 864,870 ----
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop, bool grouped)
{
List *result = NIL;
IndexPath *ipath;
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 881,892 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool can_agg_sorted;
+ List *group_clauses,
+ *group_exprs,
+ *agg_exprs;
+ AggPath *agg_path;
+ double agg_input_rows;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 891,896 ****
--- 900,908 ----
case ST_BITMAPSCAN:
if (!index->amhasgetbitmap)
return NIL;
+
+ if (grouped)
+ return NIL;
break;
case ST_ANYSCAN:
/* either or both are OK */
*************** build_index_paths(PlannerInfo *root, Rel
*** 1032,1037 ****
--- 1044,1053 ----
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
*/
+ can_agg_sorted = true;
+ group_clauses = NIL;
+ group_exprs = NIL;
+ agg_exprs = NIL;
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1064,1088 ----
outer_relids,
loop_count,
false);
! if (!grouped)
! result = lappend(result, ipath);
! else if (rel->gpi != NULL && rel->gpi->target_agg != NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root, (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
--- 1111,1142 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL &&
! rel->gpi != NULL && rel->gpi->target_agg != NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1164,1189 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else if (can_agg_sorted &&
! rel->gpi != NULL && rel->gpi->target_agg != NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! Assert(agg_path != NULL);
! result = lappend(result, agg_path);
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
--- 1207,1235 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL &&
! rel->gpi != NULL && rel->gpi->target_agg != NULL)
! {
! /*
! * TODO Double-check if this is the correct input
! * value.
! */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1244,1250 ****
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL);
result = list_concat(result, indexpaths);
}
--- 1344,1351 ----
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index d6669c4..c8557b6
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***************
*** 22,27 ****
--- 22,28 ----
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
+ #include "optimizer/tlist.h"
/* Hook for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
*************** set_join_pathlist_hook_type set_join_pat
*** 29,34 ****
--- 30,39 ----
#define PATH_PARAM_BY_REL(path, rel) \
((path)->param_info && bms_overlap(PATH_REQ_OUTER(path), (rel)->relids))
+ #define REL_HAS_GROUPED_PATHS(rel) ((rel)->gpi && (rel)->gpi->pathlist)
+ #define REL_HAS_PARTIAL_GROUPED_PATHS(rel) \
+ ((rel)->gpi && (rel)->gpi->partial_pathlist)
+
static void try_partial_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
Path *outer_path,
*************** static void try_partial_mergejoin_path(P
*** 38,47 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 43,63 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
+ static void sort_inner_and_outer_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ bool grouped_outer,
+ bool grouped_inner,
+ bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** static void consider_parallel_nestloop(P
*** 50,63 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 66,82 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped_outer,
! bool do_aggregate);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** static void generate_mergejoin_paths(Pla
*** 77,83 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 96,105 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
/*
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 279,286 ****
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
! hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 301,308 ----
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
! hash_inner_and_outer(root, joinrel, outerrel, innerrel, jointype,
! &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 341,347 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
--- 363,371 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
*************** try_nestloop_path(PlannerInfo *root,
*** 351,356 ****
--- 375,394 ----
Relids outerrelids = outerrel->relids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi || !grouped);
+
+ /*
+ * Aggregation target is necessary to produce grouped join output.
+ */
+ Assert((joinrel->gpi && joinrel->gpi->target_agg) || !grouped ||
+ !do_aggregate);
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_nestloop_path(PlannerInfo *root,
*** 383,403 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer), false);
}
else
{
--- 421,456 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! required_outer, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate && required_outer == NULL)
{
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
! {
! add_path(joinrel, join_path, grouped);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 418,426 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 471,489 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_nestloop_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target_agg) || !grouped ||
+ !do_aggregate);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 443,464 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL), false);
}
/*
--- 506,638 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! NULL, join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_nestloop_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target_agg == NULL)
! return;
!
! if (!partial)
! try_nestloop_path(root, joinrel, outer_path, inner_path, pathkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_nestloop_path(root, joinrel, outer_path, inner_path,
! pathkeys, jointype, extra, true,
! do_aggregate);
! }
!
! static void
! try_nestloop_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! grouped_join,
! do_aggregate);
! else
! try_partial_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! grouped_join,
! do_aggregate);
! }
! else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! partial,
! do_aggregate);
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 477,514 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
! if (is_partial)
! {
! try_partial_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra);
! return;
! }
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 651,689 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
! /* Caller should not request aggregation w/o grouped output. */
! Assert(!do_aggregate || grouped);
!
! /* GroupedPathInfo is necessary for us to produce a grouped set. */
! Assert(joinrel->gpi || !grouped);
!
! /*
! * Aggregation target is necessary to produce grouped join output.
! */
! Assert((joinrel->gpi && joinrel->gpi->target_agg) || !grouped ||
! !do_aggregate);
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (!bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 527,553 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys), false);
}
else
{
--- 702,748 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
{
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
! {
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 571,579 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 766,784 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_mergejoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target_agg) || !grouped ||
+ !do_aggregate);
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 603,630 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys), false);
}
/*
--- 808,969 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_mergejoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target_agg == NULL)
! return;
!
! if (!partial)
! try_mergejoin_path(root, joinrel, outer_path, inner_path, pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_mergejoin_path(root, joinrel, outer_path, inner_path,
! pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! }
!
! /*
! * Generic front-end to create combinations of full/partial and grouped/plain
! * merge joins.
! */
! static void
! try_mergejoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! false, false);
! else
! try_partial_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! false, false);
! }
! else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! do_aggregate);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 639,685 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, false))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! required_outer,
! hashclauses), false);
}
else
{
--- 978,1058 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi || !grouped);
+
+ /*
+ * Aggregation target is necessary to produce grouped join output.
+ */
+ Assert((joinrel->gpi && joinrel->gpi->target_agg) || !grouped ||
+ !do_aggregate);
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (!bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
+ *
+ * TODO Need to consider aggregation here?
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist,
! required_outer, hashclauses,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
{
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, grouped))
! {
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 700,708 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1073,1091 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_hashjoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target_agg) || !grouped ||
+ !do_aggregate);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 725,747 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL, false))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! NULL,
! hashclauses), false);
}
/*
* clause_sides_match_join
* Determine whether a join clause is of the right form to use in this join.
--- 1108,1261 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist, NULL,
! hashclauses, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! NIL, grouped))
! {
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! /*
! * Create a new grouped hash join path by joining a grouped path to plain
! * (non-grouped) one, or by joining 2 plain relations and applying grouping on
! * the result.
! *
! * Joining of 2 grouped paths is not supported. If a grouped relation A was
! * joined to grouped relation B, then the grouping of B reduces the number of
! * times each group of A is appears in the join output. This makes difference
! * for some aggregates, e.g. sum().
! *
! * If do_aggregate is true, neither input rel is grouped so we need to
! * aggregate the join result explicitly.
! *
! * partial argument tells whether the join path should be considered partial.
! */
! static void
! try_grouped_hashjoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate
! )
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target_agg == NULL)
! return;
!
! if (!partial)
! try_hashjoin_path(root, joinrel, outer_path, inner_path, hashclauses,
! jointype, extra, true, do_aggregate);
! else
! try_partial_hashjoin_path(root, joinrel, outer_path, inner_path,
! hashclauses, jointype, extra, true,
! do_aggregate);
! }
!
! /*
! * Generic front-end to create combinations of full/partial and grouped/plain
! * hash joins.
! */
! static void
! try_hashjoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! false, false);
! else
! try_partial_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! false, false);
! }
! else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! partial,
! do_aggregate);
! }
}
+
/*
* clause_sides_match_join
* Determine whether a join clause is of the right form to use in this join.
*************** sort_inner_and_outer(PlannerInfo *root,
*** 792,797 ****
--- 1306,1339 ----
JoinType jointype,
JoinPathExtraData *extra)
{
+ /* Plain (non-grouped) join. */
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, false);
+
+ /* Use all the supported strategies to generate grouped join. */
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, true, false, false);
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, true, false);
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, true);
+ }
+
+ /*
+ * TODO As merge_pathkeys shouldn't differ across calls, use a separate
+ * function to derive them and pass them here in a list.
+ */
+ static void
+ sort_inner_and_outer_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ bool grouped_outer,
+ bool grouped_inner,
+ bool do_aggregate)
+ {
JoinType save_jointype = jointype;
Path *outer_path;
Path *inner_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 813,820 ****
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! outer_path = outerrel->cheapest_total_path;
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 1355,1379 ----
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! if (grouped_outer)
! {
! if (REL_HAS_GROUPED_PATHS(outerrel))
! outer_path = linitial(outerrel->gpi->pathlist);
! else
! return;
! }
! else
! outer_path = outerrel->cheapest_total_path;
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** sort_inner_and_outer(PlannerInfo *root,
*** 832,837 ****
--- 1391,1406 ----
*/
if (jointype == JOIN_UNIQUE_OUTER)
{
+ /*
+ * TODO This is just a temporary limitation. Before lifting it, make
+ * sure that the UniquePath does emit GroupedVars. Also try to avoid
+ * the unique-ification if outer_path comes directly from AggPath
+ * (i.e. it's not grouped path combined with plain one) and the
+ * grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_outer)
+ return;
+
outer_path = (Path *) create_unique_path(root, outerrel,
outer_path, extra->sjinfo);
Assert(outer_path);
*************** sort_inner_and_outer(PlannerInfo *root,
*** 839,844 ****
--- 1408,1417 ----
}
else if (jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary limitation, like above. */
+ if (grouped_inner)
+ return;
+
inner_path = (Path *) create_unique_path(root, innerrel,
inner_path, extra->sjinfo);
Assert(inner_path);
*************** sort_inner_and_outer(PlannerInfo *root,
*** 860,872 ****
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! cheapest_partial_outer = (Path *) linitial(outerrel->partial_pathlist);
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
/*
--- 1433,1482 ----
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! if (grouped_outer)
! {
! if (REL_HAS_PARTIAL_GROUPED_PATHS(outerrel))
! cheapest_partial_outer = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
! else
! return;
! }
! else
! cheapest_partial_outer = (Path *)
! linitial(outerrel->partial_pathlist);
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else
+ {
+ Assert(innerrel->gpi != NULL);
+ inner_pathlist = innerrel->gpi->pathlist;
+ }
+
+ /*
+ * All the grouped paths should be unparameterized, so the
+ * function is overly stringent in the grouped_inner case, but
+ * still useful.
+ */
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 942,974 ****
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra);
}
}
--- 1552,1575 ----
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_mergejoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! true, grouped_outer, grouped_inner,
! do_aggregate);
}
}
*************** sort_inner_and_outer(PlannerInfo *root,
*** 985,990 ****
--- 1586,1599 ----
* some sort key requirements). So, we consider truncations of the
* mergeclause list as well as the full list. (Ideally we'd consider all
* subsets of the mergeclause list, but that seems way too expensive.)
+ *
+ * grouped_outer - is outerpath grouped?
+ * grouped_inner - use grouped paths of innerrel?
+ * do_aggregate - apply (partial) aggregation to the output?
+ *
+ * TODO If subsequent calls often differ only by the 3 arguments above,
+ * consider a workspace structure to share useful info (eg merge clauses)
+ * across calls.
*/
static void
generate_mergejoin_paths(PlannerInfo *root,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 996,1002 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1605,1614 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1047,1063 ****
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1659,1676 ----
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner, do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1113,1128 ****
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
List *newclauses = NIL;
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
--- 1726,1747 ----
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
+ List *inner_pathlist = NIL;
Path *innerpath;
List *newclauses = NIL;
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else if (innerrel->gpi != NULL)
+ inner_pathlist = innerrel->gpi->pathlist;
+
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1145,1165 ****
}
else
newclauses = mergeclauses;
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
--- 1764,1788 ----
}
else
newclauses = mergeclauses;
!
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
!
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1190,1206 ****
else
newclauses = mergeclauses;
}
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1813,1831 ----
else
newclauses = mergeclauses;
}
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1213,1255 ****
}
}
/*
! * match_unsorted_outer
! * Creates possible join paths for processing a single join relation
! * 'joinrel' by employing either iterative substitution or
! * mergejoining on each of its possible outer paths (considering
! * only outer paths that are already ordered well enough for merging).
! *
! * We always generate a nestloop path for each available outer path.
! * In fact we may generate as many as five: one on the cheapest-total-cost
! * inner path, one on the same with materialization, one on the
! * cheapest-startup-cost inner path (if different), one on the
! * cheapest-total inner-indexscan path (if any), and one on the
! * cheapest-startup inner-indexscan path (if different).
! *
! * We also consider mergejoins if mergejoin clauses are available. See
! * detailed comments in generate_mergejoin_paths.
! *
! * 'joinrel' is the join relation
! * 'outerrel' is the outer join relation
! * 'innerrel' is the inner join relation
! * 'jointype' is the type of join to do
! * 'extra' contains additional input values
*/
static void
! match_unsorted_outer(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
bool useallclauses;
! Path *inner_cheapest_total = innerrel->cheapest_total_path;
Path *matpath = NULL;
ListCell *lc1;
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
--- 1838,1869 ----
}
}
+
/*
! * TODO As merge_pathkeys shouldn't differ across calls, use a separate
! * function to derive them and pass them here in a list.
*/
static void
! match_unsorted_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
bool useallclauses;
! Path *inner_cheapest_total;
Path *matpath = NULL;
ListCell *lc1;
+ List *outer_pathlist;
+
+ /* At most one input path may be grouped. */
+ Assert(!(grouped_outer && grouped_inner));
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
*************** match_unsorted_outer(PlannerInfo *root,
*** 1286,1298 ****
--- 1900,1937 ----
break;
}
+ if (grouped_outer)
+ {
+ if (REL_HAS_GROUPED_PATHS(outerrel))
+ outer_pathlist = outerrel->gpi->pathlist;
+ else
+ return;
+ }
+ else
+ outer_pathlist = outerrel->pathlist;
+
+ if (grouped_inner)
+ {
+ if (REL_HAS_GROUPED_PATHS(innerrel))
+ inner_cheapest_total = linitial(innerrel->gpi->pathlist);
+ else
+ return;
+ }
+ else
+ inner_cheapest_total = innerrel->cheapest_total_path;
+
/*
* If inner_cheapest_total is parameterized by the outer rel, ignore it;
* we will consider it below as a member of cheapest_parameterized_paths,
* but the other possibilities considered in this routine aren't usable.
*/
if (PATH_PARAM_BY_REL(inner_cheapest_total, outerrel))
+ {
+ /* Grouped path should not be parameterized at all. */
+ Assert(!grouped_inner);
+
inner_cheapest_total = NULL;
+ }
/*
* If we need to unique-ify the inner path, we will consider only the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1303,1318 ****
/* No way to do this with an inner path parameterized by outer rel */
if (inner_cheapest_total == NULL)
return;
inner_cheapest_total = (Path *)
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK)
{
/*
* Consider materializing the cheapest inner path, unless
* enable_material is off or the path in question materializes its
* output anyway.
*/
if (enable_material && inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
--- 1942,1972 ----
/* No way to do this with an inner path parameterized by outer rel */
if (inner_cheapest_total == NULL)
return;
+
+ /*
+ * TODO This is just a temporary limitation. Before lifting it, make
+ * sure that the UniquePath does emit GroupedVars. Also try to avoid
+ * the unique-ification if outer_path comes directly from AggPath
+ * (i.e. it's not grouped path combined with plain one) and the
+ * grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_inner)
+ return;
+
inner_cheapest_total = (Path *)
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK && !grouped_inner)
{
/*
* Consider materializing the cheapest inner path, unless
* enable_material is off or the path in question materializes its
* output anyway.
+ *
+ * TODO Verify that this is ok even if grouped_inner is true (at least
+ * make sure that mathpath does contain GroupedVars) and remove
+ * grouped_inner from the condition above.
*/
if (enable_material && inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
*************** match_unsorted_outer(PlannerInfo *root,
*** 1320,1326 ****
create_material_path(innerrel, inner_cheapest_total);
}
! foreach(lc1, outerrel->pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
--- 1974,1980 ----
create_material_path(innerrel, inner_cheapest_total);
}
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1340,1345 ****
--- 1994,2004 ----
{
if (outerpath != outerrel->cheapest_total_path)
continue;
+
+ /* TODO Temporary restriction, see above. */
+ if (grouped_outer)
+ continue;
+
outerpath = (Path *) create_unique_path(root, outerrel,
outerpath, extra->sjinfo);
Assert(outerpath);
*************** match_unsorted_outer(PlannerInfo *root,
*** 1355,1371 ****
if (save_jointype == JOIN_UNIQUE_INNER)
{
/*
* Consider nestloop join, but only with the unique-ified cheapest
* inner path
*/
! try_nestloop_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra);
}
else if (nestjoinOK)
{
--- 2014,2036 ----
if (save_jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary restriction, see above. */
+ if (grouped_inner)
+ continue;
+
/*
* Consider nestloop join, but only with the unique-ified cheapest
* inner path
*/
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1377,1404 ****
*/
ListCell *lc2;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
! Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra);
}
! /* Also consider materialized form of the cheapest inner path */
if (matpath != NULL)
! try_nestloop_path(root,
! joinrel,
! outerpath,
! matpath,
! merge_pathkeys,
! jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 2042,2081 ----
*/
ListCell *lc2;
! /*
! * There are no grouped paths in cheapest_parameterized_paths.
! */
! if (!grouped_inner)
{
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
! }
}
! /*
! * Also consider materialized form of the cheapest inner path.
! */
if (matpath != NULL)
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! matpath,
! merge_pathkeys,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1413,1419 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 2090,2097 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1424,1440 ****
* we handle extra_lateral_rels, since partial paths must not be
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
! bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 2102,2124 ----
* we handle extra_lateral_rels, since partial paths must not be
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
+ *
+ * grouped_inner must be false because we're not interested in nest loop
+ * joins with the grouped path on the inner side (i.e. repeated
+ * aggregation).
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
! bms_is_empty(joinrel->lateral_relids) &&
! !grouped_inner)
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra,
! grouped_outer, do_aggregate);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1454,1464 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
/*
* consider_parallel_mergejoin
* Try to build partial paths for a joinrel by joining a partial path
* for the outer relation to a complete path for the inner relation.
--- 2138,2195 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total,
! grouped_outer, do_aggregate);
}
}
/*
+ * match_unsorted_outer
+ * Creates possible join paths for processing a single join relation
+ * 'joinrel' by employing either iterative substitution or
+ * mergejoining on each of its possible outer paths (considering
+ * only outer paths that are already ordered well enough for merging).
+ *
+ * We always generate a nestloop path for each available outer path.
+ * In fact we may generate as many as five: one on the cheapest-total-cost
+ * inner path, one on the same with materialization, one on the
+ * cheapest-startup-cost inner path (if different), one on the
+ * cheapest-total inner-indexscan path (if any), and one on the
+ * cheapest-startup inner-indexscan path (if different).
+ *
+ * We also consider mergejoins if mergejoin clauses are available. See
+ * detailed comments in generate_mergejoin_paths.
+ *
+ * 'joinrel' is the join relation
+ * 'outerrel' is the outer join relation
+ * 'innerrel' is the inner join relation
+ * 'jointype' is the type of join to do
+ * 'extra' contains additional input values
+ * 'grouped' indicates that the at least one relation in the join has been
+ * aggregated.
+ */
+ static void
+ match_unsorted_outer(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra)
+ {
+ /* Plain (non-grouped) join. */
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, false);
+
+ /* Use all the supported strategies to generate grouped join. */
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, true, false, false);
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, true, false);
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, true);
+ }
+
+ /*
* consider_parallel_mergejoin
* Try to build partial paths for a joinrel by joining a partial path
* for the outer relation to a complete path for the inner relation.
*************** match_unsorted_outer(PlannerInfo *root,
*** 1470,1475 ****
--- 2201,2210 ----
* 'extra' contains additional input values
* 'inner_cheapest_total' cheapest total path for innerrel
*/
+ /*
+ * TODO Store merge_pathkeys across calls if these (the calls) only differ in
+ * grouped_outer or do_aggregate.
+ */
static void
consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
*************** consider_parallel_mergejoin(PlannerInfo
*** 1477,1488 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
/* generate merge join path for each partial outer path */
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
--- 2212,2247 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped_outer,
! bool do_aggregate)
{
ListCell *lc1;
+ List *outer_pathlist;
+ bool grouped = grouped_outer || do_aggregate;
+
+ if (!grouped || do_aggregate)
+ {
+ /*
+ * If creating grouped paths by explicit aggregation, the input paths
+ * must be plain.
+ */
+ outer_pathlist = outerrel->partial_pathlist;
+ }
+ else if (outerrel->gpi != NULL)
+ {
+ /*
+ * Only the outer paths are accepted as grouped when we try to combine
+ * grouped and plain ones. Grouped inner path implies repeated
+ * aggregation, which doesn't sound as a good idea.
+ */
+ outer_pathlist = outerrel->gpi->partial_pathlist;
+ }
+ else
+ return;
/* generate merge join path for each partial outer path */
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
*************** consider_parallel_mergejoin(PlannerInfo
*** 1493,1501 ****
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
! extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 2252,2261 ----
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, grouped_outer, false, do_aggregate);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1510,1530 ****
* 'jointype' is the type of join to do
* 'extra' contains additional input values
*/
static void
consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
--- 2270,2317 ----
* 'jointype' is the type of join to do
* 'extra' contains additional input values
*/
+ /*
+ * TODO Store pathkeys across calls if these (the calls) only differ in
+ * grouped_outer or do_aggregate.
+ */
static void
consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer, bool do_aggregate)
{
JoinType save_jointype = jointype;
+ List *outer_pathlist;
ListCell *lc1;
+ bool grouped = grouped_outer || do_aggregate;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! if (!grouped || do_aggregate)
! {
! /*
! * If creating grouped paths by explicit aggregation, the input paths
! * must be plain.
! */
! outer_pathlist = outerrel->partial_pathlist;
! }
! else if (outerrel->gpi != NULL)
! {
! /*
! * Only the outer paths are accepted as grouped when we try to combine
! * grouped and plain ones. Grouped inner path implies repeated
! * aggregation, which doesn't sound as a good idea.
! */
! outer_pathlist = outerrel->gpi->partial_pathlist;
! }
! else
! return;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1555,1561 ****
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
--- 2342,2348 ----
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER && !grouped)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1565,1594 ****
Assert(innerpath);
}
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
/*
! * hash_inner_and_outer
! * Create hashjoin join paths by explicitly hashing both the outer and
! * inner keys of each available hash clause.
! *
! * 'joinrel' is the join relation
! * 'outerrel' is the outer join relation
! * 'innerrel' is the inner join relation
! * 'jointype' is the type of join to do
! * 'extra' contains additional input values
*/
static void
! hash_inner_and_outer(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
--- 2352,2380 ----
Assert(innerpath);
}
! try_nestloop_path_common(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! true, grouped_outer, false,
! do_aggregate);
}
}
}
/*
! * TODO hashclauses (and mabye some other info) should be shared across calls
! * if only some of the following arguments change: partial, grouped_outer,
! * grouped_inner, do_aggregate.
*/
static void
! hash_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1635,1643 ****
* outer paths. There's no need to consider any but the
* cheapest-total-cost inner path, however.
*/
! Path *cheapest_startup_outer = outerrel->cheapest_startup_path;
! Path *cheapest_total_outer = outerrel->cheapest_total_path;
! Path *cheapest_total_inner = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 2421,2456 ----
* outer paths. There's no need to consider any but the
* cheapest-total-cost inner path, however.
*/
! Path *cheapest_startup_outer,
! *cheapest_total_outer,
! *cheapest_total_inner;
!
! if (grouped_outer)
! {
! if (REL_HAS_GROUPED_PATHS(outerrel))
! {
! cheapest_total_outer = linitial(outerrel->gpi->pathlist);
! /* No parameterized grouped paths. */
! cheapest_startup_outer = NULL;
! }
! else
! return;
! }
! else
! {
! cheapest_total_outer = outerrel->cheapest_total_path;
! cheapest_startup_outer = outerrel->cheapest_startup_path;
! }
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! cheapest_total_inner = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! cheapest_total_inner = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1652,1694 ****
/* Unique-ify if need be; we ignore parameterized possibilities */
if (jointype == JOIN_UNIQUE_OUTER)
{
cheapest_total_outer = (Path *)
create_unique_path(root, outerrel,
cheapest_total_outer, extra->sjinfo);
Assert(cheapest_total_outer);
jointype = JOIN_INNER;
! try_hashjoin_path(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
{
cheapest_total_inner = (Path *)
create_unique_path(root, innerrel,
cheapest_total_inner, extra->sjinfo);
Assert(cheapest_total_inner);
jointype = JOIN_INNER;
! try_hashjoin_path(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
}
else
{
--- 2465,2530 ----
/* Unique-ify if need be; we ignore parameterized possibilities */
if (jointype == JOIN_UNIQUE_OUTER)
{
+ /*
+ * TODO This is just a temporary limitation. Before lifting it,
+ * make sure that the UniquePath does emit GroupedVars. Also try
+ * to avoid the unique-ification if the outer path comes directly
+ * from AggPath (i.e. it's not grouped path combined with plain
+ * one) and the grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_outer)
+ return;
+
cheapest_total_outer = (Path *)
create_unique_path(root, outerrel,
cheapest_total_outer, extra->sjinfo);
Assert(cheapest_total_outer);
jointype = JOIN_INNER;
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary restriction, see above. */
+ if (grouped_inner)
+ return;
+
cheapest_total_inner = (Path *)
create_unique_path(root, innerrel,
cheapest_total_inner, extra->sjinfo);
Assert(cheapest_total_inner);
jointype = JOIN_INNER;
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
!
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
}
else
{
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1698,1719 ****
* pairings of cheapest-total paths including parameterized ones.
* There is no use in generating parameterized paths on the basis
* of possibly cheap startup cost, so this is sufficient.
*/
ListCell *lc1;
ListCell *lc2;
if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
Path *outerpath = (Path *) lfirst(lc1);
/*
* We cannot use an outer path that is parameterized by the
--- 2534,2568 ----
* pairings of cheapest-total paths including parameterized ones.
* There is no use in generating parameterized paths on the basis
* of possibly cheap startup cost, so this is sufficient.
+ *
+ * For if either side of the join is grouped, we simply use
+ * rel->gpi->pathlist. (cheapest_startup_outer should be NULL if
+ * grouped_outer.)
*/
ListCell *lc1;
ListCell *lc2;
+ List *outer_pathlist;
if (cheapest_startup_outer != NULL)
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
! outer_pathlist = !grouped_outer ?
! outerrel->cheapest_parameterized_paths :
! outerrel->gpi->pathlist;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
+ List *inner_pathlist;
/*
* We cannot use an outer path that is parameterized by the
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1722,1728 ****
if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
Path *innerpath = (Path *) lfirst(lc2);
--- 2571,2581 ----
if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! inner_pathlist = !grouped_inner ?
! innerrel->cheapest_parameterized_paths :
! innerrel->gpi->pathlist;
!
! foreach(lc2, inner_pathlist)
{
Path *innerpath = (Path *) lfirst(lc2);
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1737,1749 ****
innerpath == cheapest_total_inner)
continue; /* already tried it */
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra);
}
}
}
--- 2590,2605 ----
innerpath == cheapest_total_inner)
continue; /* already tried it */
! try_hashjoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1766,1797 ****
Path *cheapest_partial_outer;
Path *cheapest_safe_inner = NULL;
! cheapest_partial_outer =
! (Path *) linitial(outerrel->partial_pathlist);
/*
* Normally, given that the joinrel is parallel-safe, the cheapest
* total inner path will also be parallel-safe, but if not, we'll
! * have to search for the cheapest safe, unparameterized inner
! * path. If doing JOIN_UNIQUE_INNER, we can't use any alternative
! * inner path.
*/
if (cheapest_total_inner->parallel_safe)
cheapest_safe_inner = cheapest_total_inner;
else if (save_jointype != JOIN_UNIQUE_INNER)
! cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra);
}
}
}
/*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
--- 2622,2717 ----
Path *cheapest_partial_outer;
Path *cheapest_safe_inner = NULL;
! if (grouped_outer)
! {
! if (REL_HAS_PARTIAL_GROUPED_PATHS(outerrel))
! {
! cheapest_partial_outer =
! (Path *) linitial(outerrel->gpi->partial_pathlist);
! }
! else
! return;
! }
! else
! cheapest_partial_outer =
! (Path *) linitial(outerrel->partial_pathlist);
/*
* Normally, given that the joinrel is parallel-safe, the cheapest
* total inner path will also be parallel-safe, but if not, we'll
! * have to search cheapest_parameterized_paths for the cheapest
! * safe, unparameterized inner path. If doing JOIN_UNIQUE_INNER,
! * we can't use any alternative inner path.
*/
if (cheapest_total_inner->parallel_safe)
cheapest_safe_inner = cheapest_total_inner;
else if (save_jointype != JOIN_UNIQUE_INNER)
! {
! ListCell *lc;
! List *inner_pathlist;
!
! inner_pathlist = !grouped_inner ?
! innerrel->cheapest_parameterized_paths :
! innerrel->gpi->pathlist;
!
! foreach(lc, inner_pathlist)
! {
! Path *innerpath = (Path *) lfirst(lc);
!
! if (innerpath->parallel_safe &&
! bms_is_empty(PATH_REQ_OUTER(innerpath)))
! {
! cheapest_safe_inner = innerpath;
! break;
! }
! }
! }
if (cheapest_safe_inner != NULL)
! try_hashjoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra,
! true,
! grouped_outer, grouped_inner,
! do_aggregate);
}
}
}
/*
+ * hash_inner_and_outer
+ * Create hashjoin join paths by explicitly hashing both the outer and
+ * inner keys of each available hash clause.
+ *
+ * 'joinrel' is the join relation
+ * 'outerrel' is the outer join relation
+ * 'innerrel' is the inner join relation
+ * 'jointype' is the type of join to do
+ * 'extra' contains additional input values
+ */
+ static void
+ hash_inner_and_outer(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra)
+ {
+ /* Plain (non-grouped) join. */
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, false);
+
+ /* Use all the supported strategies to generate grouped join. */
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, true, false, false);
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, true, false);
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, true);
+ }
+
+ /*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index 5c934f2..aeb8bc3
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** use_physical_tlist(PlannerInfo *root, Pa
*** 807,812 ****
--- 807,818 ----
return false;
/*
+ * Grouped relation's target list contains GroupedVars.
+ */
+ if (rel->gpi != NULL)
+ return false;
+
+ /*
* Can't do it if any system columns or whole-row Vars are requested.
* (This could possibly be fixed but would take some fragile assumptions
* in setrefs.c, I think.)
*************** create_projection_plan(PlannerInfo *root
*** 1571,1578 ****
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
--- 1577,1585 ----
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (!best_path->force_result &&
! (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist)))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 51d13ab..67579cd
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,230 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual),
! false);
}
/*
--- 223,229 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual), false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index c9040ce..2bf4c8b
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,90 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,89 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals), false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
*************** query_planner(PlannerInfo *root, List *t
*** 115,120 ****
--- 114,120 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index ce5ce6c..cccf84d
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 130,138 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
--- 130,135 ----
*************** static void consider_groupingsets_paths(
*** 147,152 ****
--- 144,161 ----
grouping_sets_data *gd,
const AggClauseCosts *agg_costs,
double dNumGroups);
+ static void sort_agg_partial_grouped_paths(PlannerInfo *root, List *pathlist,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups,
+ RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target);
+ static void hash_agg_partial_grouped_path(PlannerInfo *root, Path *path,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups,
+ RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target);
static RelOptInfo *create_window_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *input_target,
*************** inheritance_planner(PlannerInfo *root)
*** 1422,1429 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)),
! false);
}
/*--------------------
--- 1431,1437 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)), false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 1832,1838 ****
newpath = (Path *) create_projection_path(root,
current_rel,
subpath,
! scanjoin_target);
lfirst(lc) = newpath;
}
}
--- 1840,1847 ----
newpath = (Path *) create_projection_path(root,
current_rel,
subpath,
! scanjoin_target,
! false);
lfirst(lc) = newpath;
}
}
*************** get_number_of_groups(PlannerInfo *root,
*** 3450,3489 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3459,3464 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3514,3521 ****
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *grouped_rel;
PathTarget *partial_grouping_target = NULL;
! AggClauseCosts agg_partial_costs; /* parallel only */
! AggClauseCosts agg_final_costs; /* parallel only */
Size hashaggtablesize;
double dNumGroups;
double dNumPartialGroups = 0;
--- 3489,3496 ----
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *grouped_rel;
PathTarget *partial_grouping_target = NULL;
! AggClauseCosts agg_final_costs;
! bool agg_final_costs_known = false;
Size hashaggtablesize;
double dNumGroups;
double dNumPartialGroups = 0;
*************** create_grouping_paths(PlannerInfo *root,
*** 3697,3702 ****
--- 3672,3684 ----
}
/*
+ * Collect statistics about aggregates for estimating costs of performing
+ * aggregation in parallel.
+ */
+ MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
+ agg_final_costs_known = false;
+
+ /*
* Before generating paths for grouped_rel, we first generate any possible
* partial paths; that way, later code can easily consider both parallel
* and non-parallel approaches to grouping. Note that the partial paths
*************** create_grouping_paths(PlannerInfo *root,
*** 3707,3712 ****
--- 3689,3697 ----
if (try_parallel_aggregation)
{
Path *cheapest_partial_path = linitial(input_rel->partial_pathlist);
+ AggClauseCosts agg_partial_costs;
+
+ MemSet(&agg_partial_costs, 0, sizeof(AggClauseCosts));
/*
* Build target list for partial aggregate paths. These paths cannot
*************** create_grouping_paths(PlannerInfo *root,
*** 3722,3747 ****
cheapest_partial_path->rows,
gd);
- /*
- * Collect statistics about aggregates for estimating costs of
- * performing aggregation in parallel.
- */
- MemSet(&agg_partial_costs, 0, sizeof(AggClauseCosts));
- MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
if (parse->hasAggs)
{
/* partial phase */
get_agg_clause_costs(root, (Node *) partial_grouping_target->exprs,
AGGSPLIT_INITIAL_SERIAL,
&agg_partial_costs);
-
- /* final phase */
- get_agg_clause_costs(root, (Node *) target->exprs,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
- get_agg_clause_costs(root, parse->havingQual,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
}
if (can_sort)
--- 3707,3718 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3823,3829 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups), false);
}
}
}
--- 3794,3801 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3831,3847 ****
/* Build final grouping paths */
if (can_sort)
{
/*
* Use any available suitably-sorted path as input, and also consider
* sorting the cheapest-total path.
*/
! foreach(lc, input_rel->pathlist)
{
Path *path = (Path *) lfirst(lc);
bool is_sorted;
is_sorted = pathkeys_contained_in(root->group_pathkeys,
path->pathkeys);
if (path == cheapest_path || is_sorted)
{
/* Sort the cheapest-total path if it isn't already sorted */
--- 3803,3837 ----
/* Build final grouping paths */
if (can_sort)
{
+ List *pathlist = input_rel->pathlist;
+ int nplain = list_length(input_rel->pathlist);
+ int i;
+
+ /*
+ * The grouped paths created out of grouped base relations or joins
+ * can be treated almost identically here (the major difference is
+ * that their targetlists contain GroupedVars instead aggregate input
+ * vars, but this is handled by using the appropriate value of
+ * AggSplit), so process them in the same loop.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL)
+ pathlist = list_concat(pathlist, input_rel->gpi->pathlist);
+
/*
* Use any available suitably-sorted path as input, and also consider
* sorting the cheapest-total path.
*/
! i = 0;
! foreach(lc, pathlist)
{
Path *path = (Path *) lfirst(lc);
bool is_sorted;
+ bool is_grouped;
is_sorted = pathkeys_contained_in(root->group_pathkeys,
path->pathkeys);
+ is_grouped = i >= nplain;
+
if (path == cheapest_path || is_sorted)
{
/* Sort the cheapest-total path if it isn't already sorted */
*************** create_grouping_paths(PlannerInfo *root,
*** 3855,3866 ****
--- 3845,3883 ----
/* Now decide what to stick atop it */
if (parse->groupingSets)
{
+ /*
+ * No grouping should have taken place at base relation /
+ * join level, i.e. pathlist should be equal to
+ * input_rel->pathlist.
+ */
+ Assert(!is_grouped);
+
consider_groupingsets_paths(root, grouped_rel,
path, true, can_hash, target,
gd, agg_costs, dNumGroups);
}
else if (parse->hasAggs)
{
+ AggStrategy aggstrategy;
+ AggSplit aggsplit;
+
+ if (!is_grouped)
+ {
+ aggstrategy = parse->groupClause ? AGG_SORTED : AGG_PLAIN;
+ aggsplit = AGGSPLIT_SIMPLE;
+ }
+ else
+ {
+ /*
+ * Like above, no grouping of base relation is not
+ * possible w/o this.
+ */
+ Assert(parse->groupClause);
+
+ aggstrategy = AGG_SORTED;
+ aggsplit = AGGSPLIT_FINAL_DESERIAL;
+ }
+
/*
* We have aggregation, possibly with plain GROUP BY. Make
* an AggPath.
*************** create_grouping_paths(PlannerInfo *root,
*** 3870,3877 ****
grouped_rel,
path,
target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_SIMPLE,
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
--- 3887,3894 ----
grouped_rel,
path,
target,
! aggstrategy,
! aggsplit,
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
*************** create_grouping_paths(PlannerInfo *root,
*** 3898,4019 ****
Assert(false);
}
}
}
/*
! * Now generate a complete GroupAgg Path atop of the cheapest partial
! * path. We can do this using either Gather or Gather Merge.
*/
! if (grouped_rel->partial_pathlist)
{
! Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
! double total_groups = path->rows * path->parallel_workers;
! path = (Path *) create_gather_path(root,
! grouped_rel,
! path,
! partial_grouping_target,
! NULL,
! &total_groups);
! /*
! * Since Gather's output is always unsorted, we'll need to sort,
! * unless there's no GROUP BY clause or a degenerate (constant)
! * one, in which case there will only be a single group.
! */
! if (root->group_pathkeys)
! path = (Path *) create_sort_path(root,
! grouped_rel,
! path,
! root->group_pathkeys,
! -1.0);
! if (parse->hasAggs)
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! path,
! target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! else
! add_path(grouped_rel, (Path *)
! create_group_path(root,
grouped_rel,
! path,
! target,
! parse->groupClause,
! (List *) parse->havingQual,
! dNumGroups), false);
!
! /*
! * The point of using Gather Merge rather than Gather is that it
! * can preserve the ordering of the input path, so there's no
! * reason to try it unless (1) it's possible to produce more than
! * one output row and (2) we want the output path to be ordered.
! */
! if (parse->groupClause != NIL && root->group_pathkeys != NIL)
! {
! foreach(lc, grouped_rel->partial_pathlist)
! {
! Path *subpath = (Path *) lfirst(lc);
! Path *gmpath;
! double total_groups;
!
! /*
! * It's useful to consider paths that are already properly
! * ordered for Gather Merge, because those don't need a
! * sort. It's also useful to consider the cheapest path,
! * because sorting it in parallel and then doing Gather
! * Merge may be better than doing an unordered Gather
! * followed by a sort. But there's no point in
! * considering non-cheapest paths that aren't already
! * sorted correctly.
! */
! if (path != subpath &&
! !pathkeys_contained_in(root->group_pathkeys,
! subpath->pathkeys))
! continue;
!
! total_groups = subpath->rows * subpath->parallel_workers;
!
! gmpath = (Path *)
! create_gather_merge_path(root,
! grouped_rel,
! subpath,
! partial_grouping_target,
! root->group_pathkeys,
! NULL,
! &total_groups);
!
! if (parse->hasAggs)
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! gmpath,
! target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! else
! add_path(grouped_rel, (Path *)
! create_group_path(root,
! grouped_rel,
! gmpath,
! target,
! parse->groupClause,
! (List *) parse->havingQual,
! dNumGroups), false);
! }
! }
! }
}
if (can_hash)
--- 3915,3964 ----
Assert(false);
}
}
+
+ i++;
}
/*
! * Compute agg_final_costs iff the aggregation should take place in 2
! * steps.
*/
! if (parse->hasAggs &&
! (grouped_rel->partial_pathlist ||
! (input_rel->gpi != NULL && input_rel->gpi->partial_pathlist)))
{
! get_agg_clause_costs(root, (Node *) target->exprs,
! AGGSPLIT_FINAL_DESERIAL,
! &agg_final_costs);
! get_agg_clause_costs(root, parse->havingQual,
! AGGSPLIT_FINAL_DESERIAL,
! &agg_final_costs);
! agg_final_costs_known = true;
! }
! /*
! * Gather grouped partial paths and apply AGG_SORTED to them.
! */
! if (grouped_rel->partial_pathlist)
! sort_agg_partial_grouped_paths(root,
! grouped_rel->partial_pathlist,
! &agg_final_costs,
! dNumGroups, grouped_rel,
! partial_grouping_target, target);
! /*
! * The same for the grouped partial paths involving base relation /
! * join grouping.
! */
! if (input_rel->gpi != NULL && input_rel->gpi->target != NULL &&
! input_rel->gpi->partial_pathlist)
! sort_agg_partial_grouped_paths(root,
! input_rel->gpi->partial_pathlist,
! &agg_final_costs, dNumGroups,
grouped_rel,
! input_rel->gpi->target,
! target);
}
if (can_hash)
*************** create_grouping_paths(PlannerInfo *root,
*** 4060,4065 ****
--- 4005,4026 ----
}
/*
+ * Compute agg_final_costs if needed below and if not done above.
+ */
+ if (parse->hasAggs && !agg_final_costs_known &&
+ (grouped_rel->partial_pathlist ||
+ (input_rel->gpi != NULL &&
+ (input_rel->gpi->pathlist || input_rel->gpi->partial_pathlist))))
+ {
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ }
+
+ /*
* Generate a HashAgg Path atop of the cheapest partial path. Once
* again, we'll only do this if it looks as though the hash table
* won't exceed work_mem.
*************** create_grouping_paths(PlannerInfo *root,
*** 4068,4100 ****
{
Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
hashaggtablesize = estimate_hashagg_tablesize(path,
&agg_final_costs,
dNumGroups);
if (hashaggtablesize < work_mem * 1024L)
{
! double total_groups = path->rows * path->parallel_workers;
! path = (Path *) create_gather_path(root,
! grouped_rel,
! path,
! partial_grouping_target,
! NULL,
! &total_groups);
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! path,
! target,
! AGG_HASHED,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! }
}
}
--- 4029,4084 ----
{
Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
+ hash_agg_partial_grouped_path(root, path, &agg_final_costs,
+ dNumGroups, grouped_rel,
+ partial_grouping_target, target);
+ }
+
+ /*
+ * If input_rel has partially aggregated paths, perform the final
+ * aggregation.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->pathlist);
+
hashaggtablesize = estimate_hashagg_tablesize(path,
&agg_final_costs,
dNumGroups);
if (hashaggtablesize < work_mem * 1024L)
{
! /*
! * The top-level grouped_rel needs to receive the path into
! * regular pathlist, as opposed grouped_rel->gpi->pathlist. So
! * pass FALSE for grouped.
! */
! add_path(grouped_rel,
! (Path *) create_agg_path(root, grouped_rel,
! path,
! target,
! AGG_HASHED,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups),
! false);
! }
! }
! /*
! * If input_rel has partially aggregated partial paths, gather them
! * and perform the final aggregation.
! */
! if (input_rel->gpi != NULL && input_rel->gpi->target != NULL &&
! input_rel->gpi->partial_pathlist != NIL)
! {
! Path *path = (Path *) linitial(input_rel->gpi->partial_pathlist);
! hash_agg_partial_grouped_path(root, path, &agg_final_costs, dNumGroups,
! grouped_rel, input_rel->gpi->target,
! target);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4474,4479 ****
--- 4458,4630 ----
}
/*
+ * Subroutine of create_grouping_paths() to apply GatherPath or
+ * GatherMergePath and AGG_SORTED AggPath to cases which only differ in the
+ * GatherPath / GatherMergePath target.
+ */
+ static void
+ sort_agg_partial_grouped_paths(PlannerInfo *root, List *pathlist,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups, RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target)
+ {
+ Path *path = (Path *) linitial(pathlist);
+ double total_groups = path->rows * path->parallel_workers;
+ Query *parse = root->parse;
+ ListCell *lc;
+
+ /*
+ * Generate a complete GroupAgg Path atop of the cheapest partial path. We
+ * can do this using either Gather or Gather Merge.
+ */
+ path = (Path *) create_gather_path(root,
+ grouped_rel,
+ path,
+ gather_target,
+ NULL,
+ &total_groups);
+
+ /*
+ * Since Gather's output is always unsorted, we'll need to sort, unless
+ * there's no GROUP BY clause or a degenerate (constant) one, in which
+ * case there will only be a single group.
+ */
+ if (root->group_pathkeys)
+ path = (Path *) create_sort_path(root,
+ grouped_rel,
+ path,
+ root->group_pathkeys,
+ -1.0);
+
+ if (parse->hasAggs)
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ parse->groupClause ? AGG_SORTED : AGG_PLAIN,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ else
+ add_path(grouped_rel, (Path *)
+ create_group_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ dNumGroups), false);
+
+ /*
+ * The point of using Gather Merge rather than Gather is that it can
+ * preserve the ordering of the input path, so there's no reason to try it
+ * unless (1) it's possible to produce more than one output row and (2) we
+ * want the output path to be ordered.
+ */
+ if (parse->groupClause != NIL && root->group_pathkeys != NIL)
+ {
+ foreach(lc, pathlist)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Path *gmpath;
+ double total_groups;
+
+ /*
+ * It's useful to consider paths that are already properly ordered
+ * for Gather Merge, because those don't need a sort. It's also
+ * useful to consider the cheapest path, because sorting it in
+ * parallel and then doing Gather Merge may be better than doing
+ * an unordered Gather followed by a sort. But there's no point
+ * in considering non-cheapest paths that aren't already sorted
+ * correctly.
+ */
+ if (path != subpath &&
+ !pathkeys_contained_in(root->group_pathkeys,
+ subpath->pathkeys))
+ continue;
+
+ total_groups = subpath->rows * subpath->parallel_workers;
+
+ gmpath = (Path *)
+ create_gather_merge_path(root,
+ grouped_rel,
+ subpath,
+ gather_target,
+ root->group_pathkeys,
+ NULL,
+ &total_groups);
+
+ if (parse->hasAggs)
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ gmpath,
+ group_target,
+ parse->groupClause ? AGG_SORTED : AGG_PLAIN,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ else
+ add_path(grouped_rel, (Path *)
+ create_group_path(root,
+ grouped_rel,
+ gmpath,
+ group_target,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ dNumGroups), false);
+ }
+ }
+ }
+
+ /*
+ * Subroutine of create_grouping_paths() to apply GatherPath and AGG_HASHED
+ * AggPath to cases which only differ in the GatherPath target.
+ */
+ static void
+ hash_agg_partial_grouped_path(PlannerInfo *root, Path *path,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups, RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target)
+ {
+ Size hashaggtablesize = estimate_hashagg_tablesize(path, agg_final_costs,
+ dNumGroups);
+ Query *parse = root->parse;
+ double total_groups;
+
+ if (hashaggtablesize >= work_mem * 1024L)
+ return;
+
+ total_groups = path->rows * path->parallel_workers;
+
+ path = (Path *) create_gather_path(root,
+ grouped_rel,
+ path,
+ gather_target,
+ NULL,
+ &total_groups);
+
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ }
+
+ /*
* create_window_paths
*
* Build a new upperrel containing Paths for window-function evaluation.
*************** create_distinct_paths(PlannerInfo *root,
*** 4761,4767 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
--- 4912,4919 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows),
! false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4788,4794 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
--- 4940,4947 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows),
! false);
}
/*
*************** adjust_paths_for_srfs(PlannerInfo *root,
*** 5901,5907 ****
newpath = (Path *) create_projection_path(root,
rel,
newpath,
! thistarget);
}
}
lfirst(lc) = newpath;
--- 6054,6061 ----
newpath = (Path *) create_projection_path(root,
rel,
newpath,
! thistarget,
! false);
}
}
lfirst(lc) = newpath;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index b0c9e94..718154d
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
tlist_vinfo vars[FLEXIBLE_ARRAY_MEMBER]; /* has num_vars entries */
} indexed_tlist;
*************** set_upper_references(PlannerInfo *root,
*** 1737,1745 ****
--- 1738,1800 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the current
+ * subplan->targetlist.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ else if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /*
+ * Partial aggregation node can have GroupedVar's on the input
+ * if those represent generic (non-Var) grouping expressions.
+ * Unlike above, the restored expressions should stay there.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1949,1954 ****
--- 2004,2010 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
/* Find the Vars and fill in the index array */
*************** build_tlist_index(List *tlist)
*** 1968,1973 ****
--- 2024,2031 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else
itlist->has_non_vars = true;
}
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2245,2250 ****
--- 2303,2333 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2401,2407 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
--- 2484,2491 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index a27311d..ec081f9
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 24,29 ****
--- 24,31 ----
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if get_grouping_expressions ends up in another module. */
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
*************** typedef enum
*** 46,52 ****
#define STD_FUZZ_FACTOR 1.01
static List *translate_sub_tlist(List *tlist, int relid);
!
/*****************************************************************************
* MISC. PATH UTILITIES
--- 48,56 ----
#define STD_FUZZ_FACTOR 1.01
static List *translate_sub_tlist(List *tlist, int relid);
! static Path *add_grouping_expressions_to_subpath(PlannerInfo *root,
! Path *subpath,
! PathTarget *group_exprs);
/*****************************************************************************
* MISC. PATH UTILITIES
*************** add_partial_path_precheck(RelOptInfo *pa
*** 963,969 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, false))
return false;
return true;
--- 967,973 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, grouped))
return false;
return true;
*************** create_unique_path(PlannerInfo *root, Re
*** 1478,1485 ****
MemoryContext oldcontext;
int numCols;
! /* Caller made a mistake if subpath isn't cheapest_total ... */
! Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
Assert(sjinfo->jointype == JOIN_SEMI);
--- 1482,1493 ----
MemoryContext oldcontext;
int numCols;
! /*
! * Caller made a mistake if subpath isn't cheapest_total (or the cheapest
! * grouped).
! */
! Assert(subpath == rel->cheapest_total_path ||
! (rel->gpi != NULL && subpath == linitial(rel->gpi->pathlist)));
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
Assert(sjinfo->jointype == JOIN_SEMI);
*************** calc_non_nestloop_required_outer(Path *o
*** 2100,2105 ****
--- 2108,2114 ----
* 'restrict_clauses' are the RestrictInfo nodes to apply at the join
* 'pathkeys' are the path keys of the new join path
* 'required_outer' is the set of required outer rels
+ * 'target' can be passed to override that of joinrel.
*
* Returns the resulting path node.
*/
*************** create_nestloop_path(PlannerInfo *root,
*** 2113,2119 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
--- 2122,2129 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
*************** create_nestloop_path(PlannerInfo *root,
*** 2146,2152 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2156,2162 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target == NULL ? joinrel->reltarget : target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2204,2216 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2214,2228 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2255,2260 ****
--- 2267,2273 ----
* 'required_outer' is the set of required outer rels
* 'hashclauses' are the RestrictInfo nodes to use as hash clauses
* (this should be a subset of the restrict_clauses list)
+ * 'target' can be passed to override that of joinrel.
*/
HashPath *
create_hashjoin_path(PlannerInfo *root,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2266,2278 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2279,2293 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2319,2330 ****
* 'rel' is the parent relation associated with the result
* 'subpath' is the path representing the source of data
* 'target' is the PathTarget to be computed
*/
ProjectionPath *
create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target)
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget = subpath->pathtarget;
--- 2334,2347 ----
* 'rel' is the parent relation associated with the result
* 'subpath' is the path representing the source of data
* 'target' is the PathTarget to be computed
+ * 'force_result' enforces implementation using Result plan.
*/
ProjectionPath *
create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target,
! bool force_result)
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget = subpath->pathtarget;
*************** create_projection_path(PlannerInfo *root
*** 2343,2348 ****
--- 2360,2366 ----
pathnode->path.pathkeys = subpath->pathkeys;
pathnode->subpath = subpath;
+ pathnode->force_result = force_result;
/*
* We might not need a separate Result node. If the input plan node type
*************** create_projection_path(PlannerInfo *root
*** 2353,2360 ****
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs))
{
/* No separate Result node needed */
pathnode->dummypp = true;
--- 2371,2379 ----
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (!force_result &&
! (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
*************** apply_projection_to_path(PlannerInfo *ro
*** 2424,2430 ****
* separate ProjectionPath.
*/
if (!is_projection_capable_path(path))
! return (Path *) create_projection_path(root, rel, path, target);
/*
* We can just jam the desired tlist into the existing path, being sure to
--- 2443,2450 ----
* separate ProjectionPath.
*/
if (!is_projection_capable_path(path))
! return (Path *) create_projection_path(root, rel, path, target,
! false);
/*
* We can just jam the desired tlist into the existing path, being sure to
*************** apply_projection_to_path(PlannerInfo *ro
*** 2462,2468 ****
create_projection_path(root,
gpath->subpath->parent,
gpath->subpath,
! target);
}
else if (path->parallel_safe &&
!is_parallel_safe(root, (Node *) target->exprs))
--- 2482,2489 ----
create_projection_path(root,
gpath->subpath->parent,
gpath->subpath,
! target,
! false);
}
else if (path->parallel_safe &&
!is_parallel_safe(root, (Node *) target->exprs))
*************** create_agg_path(PlannerInfo *root,
*** 2758,2763 ****
--- 2779,3010 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * first_call indicates whether the function is being called first time for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * group_clauses, group_exprs and agg_exprs are pointers to lists we populate
+ * when called first time for particular index, and that user passes for
+ * subsequent calls.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+ Assert(rel->gpi->target_agg != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /* Add generic grouping expressions to the subpath if there are some. */
+ if (rel->gpi->group_exprs != NULL)
+ subpath = add_grouping_expressions_to_subpath(root, subpath,
+ rel->gpi->group_exprs);
+
+ if (first_call)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ if (first_call)
+ get_grouping_expressions(root, rel->gpi->target_agg,
+ rel->gpi->sortgroupclauses, group_clauses,
+ group_exprs, agg_exprs);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows, NULL);
+
+ /* TODO HAVING qual. */
+ Assert(*group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, rel->gpi->target_agg,
+ AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL, &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Appy partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ *
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+ Assert(rel->gpi->target_agg != NULL);
+
+ /* Add generic grouping expressions to the subpath if there are some. */
+ if (rel->gpi->group_exprs != NULL)
+ subpath = add_grouping_expressions_to_subpath(root, subpath,
+ rel->gpi->group_exprs);
+
+ if (first_call)
+ {
+ /*
+ * Find one grouping clause per grouping column.
+ *
+ * All that create_agg_plan eventually needs of the clause is
+ * tleSortGroupRef, so we don't have to care that the clause
+ * expression might differ from texpr, in case texpr was derived from
+ * EC.
+ */
+ get_grouping_expressions(root, rel->gpi->target_agg,
+ rel->gpi->sortgroupclauses,
+ group_clauses,
+ group_exprs, agg_exprs);
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows,
+ NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(*group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ rel->gpi->target_agg,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
+ * Add (non-Var) grouping expressions contained in group_exprs target to a
+ * subpath. The passed in subpath should not be touched so that it can still
+ * be used as a subpath for non-grouped paths. Therefore we wrap it into
+ * ResultPath, to which we actually add those expressions.
+ */
+ static Path *
+ add_grouping_expressions_to_subpath(PlannerInfo *root, Path *subpath,
+ PathTarget *group_exprs)
+ {
+ PathTarget *target;
+ ListCell *lc;
+ int i;
+
+ /*
+ * The subpath must stay intact because, besides producing input data for
+ * aggregation, it can participate in a plan that does not use aggregation
+ * push-down. For the same reason we pass force_result=true to
+ * create_projection_path below.
+ */
+ target = copy_pathtarget(subpath->pathtarget);
+
+ Assert(group_exprs->sortgrouprefs != NULL);
+ i = 0;
+ foreach(lc, group_exprs->exprs)
+ {
+ Index sortgroupref;
+ GroupedVar *gvar;
+
+ sortgroupref = group_exprs->sortgrouprefs[i++];
+ gvar = lfirst_node(GroupedVar, lc);
+ add_column_to_pathtarget(target, (Expr *) gvar, sortgroupref);
+ }
+
+ return (Path *) create_projection_path(root,
+ subpath->parent,
+ subpath,
+ target,
+ true);
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 61387a2..ecd9c83
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 25,30 ****
--- 25,32 ----
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
+ #include "parser/parse_oper.h"
#include "utils/hsearch.h"
*************** build_simple_rel(PlannerInfo *root, int
*** 120,125 ****
--- 122,128 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->gpi = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** build_join_rel(PlannerInfo *root,
*** 497,502 ****
--- 500,506 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 544,549 ****
--- 548,564 ----
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
+ * Try to build grouped targets --- both for aggregation and for joining
+ * grouped relation to non-grouped one.
+ */
+
+ /*
+ * TODO Consider if placeholders make sense here. If not, also make the
+ * related code below conditional.
+ */
+ prepare_rel_for_grouping(root, joinrel);
+
+ /*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
* now we can finish computing that. This is much like the computation of
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index abdc5d9..6025ea0
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,413 ****
--- 408,502 ----
return result;
}
+ /*
+ * get_sortgrouplist_clauses
+ *
+ * Given a "grouped target" (i.e. target where each non-GroupedVar
+ * element must have sortgroupref set), build a list of the referencing
+ * SortGroupClauses, a list of the corresponding grouping expressions and
+ * a list of aggregate expressions.
+ *
+ * target_group_clauses can contain a list of target-specific
+ * SortGroupClause's, which correspond to grouping expressions possibly
+ * added to the target (i.e. w/o being present in
+ * root->query->groupClause, see initialize_grouped_targets for
+ * details.).
+ */
+ /* Refine the function name. */
+ void
+ get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List *target_group_clauses,
+ List **grouping_clauses, List **grouping_exprs,
+ List **agg_exprs)
+ {
+ ListCell *l;
+ int i = 0;
+
+ /* The target should contain at least one grouping column. */
+ Assert(target->sortgrouprefs != NULL);
+
+ foreach(l, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(l);
+
+ if (IsA(texpr, GroupedVar) &&
+ IsA(((GroupedVar *) texpr)->gvexpr, Aggref))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides aggregates, the target should contain no expressions w/o
+ * sortgroupref. Plain relation being joined to grouped can have
+ * sortgroupref equal to zero for expressions contained neither in
+ * grouping expression nor in aggregate arguments, but if the target
+ * contains such an expression, it shouldn't be used for aggregation
+ * --- see can_aggregate field of GroupedPathInfo.
+ */
+ Assert(sortgroupref > 0);
+
+ cl = get_sortgroupref_clause_noerr(sortgroupref, root->parse->groupClause);
+
+ /*
+ * If query does not have this clause, it must be target-specific.
+ */
+ if (cl == NULL)
+ cl = get_sortgroupref_clause(sortgroupref, target_group_clauses);
+
+ *grouping_clauses = list_append_unique(*grouping_clauses, cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ *grouping_exprs = list_append_unique(*grouping_exprs, texpr);
+ }
+
+ /* Now collect the aggregates. */
+ while (l != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(l));
+
+ /* Currently, GroupedVarInfo can only represent aggregate. */
+ Assert(gvar->agg_partial != NULL);
+ *agg_exprs = lappend(*agg_exprs, gvar->agg_partial);
+ l = lnext(l);
+ }
+ }
+
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 872,928 ----
}
/*
+ * Replace each "grouped var" in the source targetlist with the original
+ * expression.
+ *
+ * TODO Think of more suitable name undo_grouped_var_substitutions? Also note
+ * that the partial aggregate is retrieved, not the original one.
+ */
+ List *
+ restore_grouping_expressions(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te,
+ *te_new;
+ Expr *expr_new = NULL;
+
+ te = lfirst_node(TargetEntry, l);
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Partial aggregate should appear in the targetlist so that
+ * it looks as if convert_combining_aggrefs arranged it.
+ */
+ expr_new = (Expr *) gvar->agg_partial;
+ }
+ else
+ expr_new = gvar->gvexpr;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
* For each aggregate add GroupedVar to the grouped target.
*
* Caller passes the aggregates in the form of GroupedVarInfos so that we
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index 7469ec7..d26bf68
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7631,7636 ****
--- 7631,7653 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+ Expr *expr = gvar->gvexpr;
+
+ if (IsA(expr, Aggref))
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ else if (IsA(expr, Var))
+ (void) get_variable((Var *) expr, 0, false, context);
+ else
+ {
+ Assert(IsA(gvar->gvexpr, OpExpr));
+ get_oper_expr((OpExpr *) expr, context);
+ }
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 9101,9110 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 9118,9135 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index a7a0614..073d0fe
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 114,119 ****
--- 114,120 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
*************** estimate_hash_bucket_stats(PlannerInfo *
*** 3707,3712 ****
--- 3708,3746 ----
ReleaseVariableStats(vardata);
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index 3734b74..b52a71b
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct HashPath
*** 1425,1436 ****
--- 1425,1440 ----
* ProjectionPath node, which is marked dummy to indicate that we intend to
* assign the work to the input plan node. The estimated cost for the
* ProjectionPath node will account for whether a Result will be used or not.
+ *
+ * force_result field tells that the Result node must be used for some reason
+ * even though the subpath could normally handle the projection.
*/
typedef struct ProjectionPath
{
Path path;
Path *subpath; /* path representing input source */
bool dummypp; /* true if no separate Result is needed */
+ bool force_result; /* Is Result node required? */
} ProjectionPath;
/*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index ef9f982..2d32f7f
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern NestPath *create_nestloop_path(Pl
*** 129,135 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 129,136 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern MergePath *create_mergejoin_path(
*** 143,149 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 144,151 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern HashPath *create_hashjoin_path(Pl
*** 154,165 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target);
extern Path *apply_projection_to_path(PlannerInfo *root,
RelOptInfo *rel,
Path *path,
--- 156,169 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target,
! bool force_result);
extern Path *apply_projection_to_path(PlannerInfo *root,
RelOptInfo *rel,
Path *path,
*************** extern AggPath *create_agg_path(PlannerI
*** 195,200 ****
--- 199,218 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 4e06b2e..df81ffd
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void set_dummy_rel_pathlist(RelOp
*** 53,59 ****
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
--- 53,64 ----
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
!
! extern void create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
! Path *subpath, bool precheck, bool partial,
! AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
*************** extern void debug_print_rel(PlannerInfo
*** 67,73 ****
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
--- 72,79 ----
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 7752497..1e7caf0
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern void split_pathtarget_at_srfs(Pla
*** 69,74 ****
--- 69,77 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index dc6069d..78d8109
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern void estimate_hash_bucket_stats(P
*** 210,215 ****
--- 210,219 ----
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
+
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
agg_pushdown_v3/05_fdw.diff
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
new file mode 100644
index 822c350..c371336
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
*************** PG_FUNCTION_INFO_V1(file_fdw_validator);
*** 117,126 ****
*/
static void fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static void fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static ForeignScan *fileGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
--- 117,128 ----
*/
static void fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static void fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static ForeignScan *fileGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
*************** get_file_fdw_attribute_options(Oid relid
*** 487,497 ****
static void
fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
FileFdwPlanState *fdw_private;
/*
* Fetch options. We only need filename (or program) at this point, but
* we might as well get everything and not need to re-fetch it later in
* planning.
--- 489,507 ----
static void
fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
FileFdwPlanState *fdw_private;
/*
+ * XXX Grouping at relation level is possible but this FDW does not
+ * implement it yet.
+ */
+ if (grouped)
+ return;
+
+ /*
* Fetch options. We only need filename (or program) at this point, but
* we might as well get everything and not need to re-fetch it later in
* planning.
*************** fileGetForeignRelSize(PlannerInfo *root,
*** 518,524 ****
static void
fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
FileFdwPlanState *fdw_private = (FileFdwPlanState *) baserel->fdw_private;
Cost startup_cost;
--- 528,535 ----
static void
fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
FileFdwPlanState *fdw_private = (FileFdwPlanState *) baserel->fdw_private;
Cost startup_cost;
*************** fileGetForeignPaths(PlannerInfo *root,
*** 526,531 ****
--- 537,549 ----
List *columns;
List *coptions = NIL;
+ /*
+ * XXX Grouping at relation level is possible but this FDW does not
+ * implement it yet.
+ */
+ if (grouped)
+ return;
+
/* Decide whether to selectively perform binary conversion */
if (check_selective_binary_conversion(baserel,
foreigntableid,
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
new file mode 100644
index 285cf1b..225e78c
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
*************** static void deparseTargetList(StringInfo
*** 133,138 ****
--- 133,139 ----
bool qualify_col,
List **retrieved_attrs);
static void deparseExplicitTargetList(List *tlist, List **retrieved_attrs,
+ bool grouped,
deparse_expr_cxt *context);
static void deparseSubqueryTargetList(deparse_expr_cxt *context);
static void deparseReturningList(StringInfo buf, PlannerInfo *root,
*************** static void printRemoteParam(int paramin
*** 163,169 ****
static void printRemotePlaceholder(Oid paramtype, int32 paramtypmod,
deparse_expr_cxt *context);
static void deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context);
static void deparseLockingClause(deparse_expr_cxt *context);
static void appendOrderByClause(List *pathkeys, deparse_expr_cxt *context);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
--- 164,170 ----
static void printRemotePlaceholder(Oid paramtype, int32 paramtypmod,
deparse_expr_cxt *context);
static void deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context, bool grouped);
static void deparseLockingClause(deparse_expr_cxt *context);
static void appendOrderByClause(List *pathkeys, deparse_expr_cxt *context);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
*************** static void deparseAggref(Aggref *node,
*** 177,185 ****
static void appendGroupByClause(List *tlist, deparse_expr_cxt *context);
static void appendAggOrderBy(List *orderList, List *targetList,
deparse_expr_cxt *context);
! static void appendFunctionName(Oid funcid, deparse_expr_cxt *context);
static Node *deparseSortGroupClause(Index ref, List *tlist,
deparse_expr_cxt *context);
/*
* Helper functions
--- 178,187 ----
static void appendGroupByClause(List *tlist, deparse_expr_cxt *context);
static void appendAggOrderBy(List *orderList, List *targetList,
deparse_expr_cxt *context);
! static void appendFunctionName(Oid funcid, StringInfo buf, char *schemaname);
static Node *deparseSortGroupClause(Index ref, List *tlist,
deparse_expr_cxt *context);
+ static void deparseSortGroupExpr(Expr *expr, deparse_expr_cxt *context);
/*
* Helper functions
*************** foreign_expr_walker(Node *node,
*** 365,370 ****
--- 367,380 ----
}
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ Assert(!IsA(gvar->gvexpr, Aggref));
+ return foreign_expr_walker((Node *) gvar->gvexpr, glob_cxt,
+ outer_cxt);
+ }
case T_Const:
{
Const *c = (Const *) node;
*************** foreign_expr_walker(Node *node,
*** 676,687 ****
Aggref *agg = (Aggref *) node;
ListCell *lc;
! /* Not safe to pushdown when not in grouping context */
! if (!IS_UPPER_REL(glob_cxt->foreignrel))
! return false;
!
! /* Only non-split aggregates are pushable. */
! if (agg->aggsplit != AGGSPLIT_SIMPLE)
return false;
/* As usual, it must be shippable. */
--- 686,700 ----
Aggref *agg = (Aggref *) node;
ListCell *lc;
! /*
! * Only non-split aggregates are pushable.
! *
! * XXX Like above, AGGSPLIT_SIMPLE shouldn't appear here if
! * the aggregation of the whole upper relation gets abandoned.
! * (Check all occurrences of AGGSPLIT_SIMPLE.)
! */
! if (agg->aggsplit != AGGSPLIT_SIMPLE &&
! agg->aggsplit != AGGSPLIT_INITIAL_SERIAL)
return false;
/* As usual, it must be shippable. */
*************** extern void
*** 925,931 ****
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
List *tlist, List *remote_conds, List *pathkeys,
bool is_subquery, List **retrieved_attrs,
! List **params_list)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
--- 938,944 ----
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
List *tlist, List *remote_conds, List *pathkeys,
bool is_subquery, List **retrieved_attrs,
! List **params_list, bool grouping)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
*************** deparseSelectStmtForRel(StringInfo buf,
*** 945,951 ****
context.params_list = params_list;
/* Construct SELECT clause */
! deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context);
/*
* For upper relations, the WHERE clause is built from the remote
--- 958,964 ----
context.params_list = params_list;
/* Construct SELECT clause */
! deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context, grouping);
/*
* For upper relations, the WHERE clause is built from the remote
*************** deparseSelectStmtForRel(StringInfo buf,
*** 965,977 ****
/* Construct FROM and WHERE clauses */
deparseFromExpr(quals, &context);
! if (IS_UPPER_REL(rel))
{
/* Append GROUP BY clause */
appendGroupByClause(tlist, &context);
! /* Append HAVING clause */
! if (remote_conds)
{
appendStringInfoString(buf, " HAVING ");
appendConditions(remote_conds, &context);
--- 978,1006 ----
/* Construct FROM and WHERE clauses */
deparseFromExpr(quals, &context);
! if (IS_UPPER_REL(rel) || grouping)
{
/* Append GROUP BY clause */
appendGroupByClause(tlist, &context);
! /*
! * Append HAVING clause.
! *
! * XXX As the "partition-wise join" patch will probably be committed
! * to PG core earlier than the "aggregate push-down" patch, we don't
! * add HAVING clause to query that represents remote simple relation
! * or join. The point is that the HAVING clause can't be evaluated
! * below the final aggregation node, and the final aggregation is
! * essentially local.
! *
! * To surpass this limitation, the FDW would have to know whether the
! * relation / join is a partition or the whole table. In case it's
! * partition, HAVING is only legal if the query does not use any other
! * partition. If we eventually can add the HAVING clause to scan /
! * join remote queries, we also have to change the API so that WHERE /
! * ON conditions are passed separate from the HAVING expression.
! */
! if (IS_UPPER_REL(rel) && remote_conds)
{
appendStringInfoString(buf, " HAVING ");
appendConditions(remote_conds, &context);
*************** deparseSelectStmtForRel(StringInfo buf,
*** 1001,1007 ****
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
--- 1030,1036 ----
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context, bool grouping)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
*************** deparseSelectSql(List *tlist, bool is_su
*** 1028,1034 ****
* For a join or upper relation the input tlist gives the list of
* columns required to be fetched from the foreign server.
*/
! deparseExplicitTargetList(tlist, retrieved_attrs, context);
}
else
{
--- 1057,1072 ----
* For a join or upper relation the input tlist gives the list of
* columns required to be fetched from the foreign server.
*/
! deparseExplicitTargetList(tlist, retrieved_attrs, grouping, context);
! }
! else if (grouping && IS_SIMPLE_REL(foreignrel) && foreignrel->gpi != NULL)
! {
! /*
! * An explicit targetlist is also passed if aggregation should take
! * place in the remote database.
! */
! deparseExplicitTargetList(fpinfo->grouped_tlist, retrieved_attrs,
! grouping, context);
}
else
{
*************** get_jointype_name(JoinType jointype)
*** 1342,1348 ****
* from 1. It has same number of entries as tlist.
*/
static void
! deparseExplicitTargetList(List *tlist, List **retrieved_attrs,
deparse_expr_cxt *context)
{
ListCell *lc;
--- 1380,1386 ----
* from 1. It has same number of entries as tlist.
*/
static void
! deparseExplicitTargetList(List *tlist, List **retrieved_attrs, bool grouping,
deparse_expr_cxt *context)
{
ListCell *lc;
*************** deparseRangeTblRef(StringInfo buf, Plann
*** 1505,1511 ****
appendStringInfoChar(buf, '(');
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
fpinfo->remote_conds, NIL, true,
! &retrieved_attrs, params_list);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
--- 1543,1549 ----
appendStringInfoChar(buf, '(');
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
fpinfo->remote_conds, NIL, true,
! &retrieved_attrs, params_list, false);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
*************** deparseExpr(Expr *node, deparse_expr_cxt
*** 2126,2131 ****
--- 2164,2172 ----
case T_Var:
deparseVar((Var *) node, context);
break;
+ case T_GroupedVar:
+ deparseExpr((Expr *) ((GroupedVar *) node)->gvexpr, context);
+ break;
case T_Const:
deparseConst((Const *) node, context, 0);
break;
*************** deparseFuncExpr(FuncExpr *node, deparse_
*** 2469,2475 ****
/*
* Normal function: display as proname(args).
*/
! appendFunctionName(node->funcid, context);
appendStringInfoChar(buf, '(');
/* ... and all the arguments */
--- 2510,2516 ----
/*
* Normal function: display as proname(args).
*/
! appendFunctionName(node->funcid, context->buf, NULL);
appendStringInfoChar(buf, '(');
/* ... and all the arguments */
*************** deparseAggref(Aggref *node, deparse_expr
*** 2747,2761 ****
{
StringInfo buf = context->buf;
bool use_variadic;
/* Only basic, non-split aggregation accepted. */
! Assert(node->aggsplit == AGGSPLIT_SIMPLE);
/* Check if need to print VARIADIC (cf. ruleutils.c) */
use_variadic = node->aggvariadic;
/* Find aggregate name from aggfnoid which is a pg_proc entry */
! appendFunctionName(node->aggfnoid, context);
appendStringInfoChar(buf, '(');
/* Add DISTINCT */
--- 2788,2851 ----
{
StringInfo buf = context->buf;
bool use_variadic;
+ ListCell *lc;
+ bool use_core_aggregate;
+ char *schemaname = NULL;
/* Only basic, non-split aggregation accepted. */
! Assert(node->aggsplit == AGGSPLIT_SIMPLE ||
! node->aggsplit == AGGSPLIT_INITIAL_SERIAL);
/* Check if need to print VARIADIC (cf. ruleutils.c) */
use_variadic = node->aggvariadic;
+ /*
+ * The core aggregates (i.e. those in postgres catalog) do or do not
+ * perform finalization on both remote and local side. But sometimes we
+ * need the same aggregate to run w/o finalization on the remote node and
+ * with the finalization locally.
+ *
+ * Since the current Aggref is partial, it essentially returns the
+ * transient type. We need to find the original aggregate.
+ */
+ foreach(lc, context->root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi;
+ Aggref *orig;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ /* GroupedVar currently represents only aggregates. */
+ orig = castNode(Aggref, gvi->gvexpr);
+
+ /*
+ * If there are multiple aggregates of the same aggfnoid in the query,
+ * they should all have the same values of the fields we're going to
+ * check. So just use the first one we find.
+ */
+ if (orig->aggfnoid == node->aggfnoid)
+ {
+ use_core_aggregate = orig->aggtype == orig->aggtranstype;
+ break;
+ }
+ }
+
+ /* Must have found the original aggregate in the list. */
+ Assert(lc != NULL);
+
+ /*
+ * TODO
+ *
+ * 1. Make the schema name configurable as FDW option.
+ *
+ * 2. Implement the partial aggregates. These don't fit into postgres_fdw
+ * because there's no reason to install postgres_fdw on the remote node.
+ * Separate extension (contrib module?) is probably needed.
+ */
+ if (!use_core_aggregate)
+ schemaname = "partial";
+
/* Find aggregate name from aggfnoid which is a pg_proc entry */
! appendFunctionName(node->aggfnoid, context->buf, schemaname);
appendStringInfoChar(buf, '(');
/* Add DISTINCT */
*************** deparseAggref(Aggref *node, deparse_expr
*** 2829,2834 ****
--- 2919,2940 ----
}
appendStringInfoChar(buf, ')');
+
+ /*
+ * If special (non-core) aggregate is needed on the remote node, it can
+ * return data type for which postgres does not have textual format. cast
+ * to bytea seems to be an universal solution in such a case.
+ *
+ * TODO
+ *
+ * Consider checking if aggserialfn exists --- aggtranstype is used now
+ * because the info is easier to access. Also consider a requirement that
+ * the "remote aggregates" must return bytea, unless they return a
+ * primitive type. Thus we wouldn't have to deal with the cast here at
+ * all.
+ */
+ if (schemaname != NULL && node->aggtranstype == INTERNALOID)
+ appendStringInfoString(buf, "::bytea");
}
/*
*************** appendGroupByClause(List *tlist, deparse
*** 2952,2966 ****
*/
Assert(!query->groupingSets);
! foreach(lc, query->groupClause)
{
! SortGroupClause *grp = (SortGroupClause *) lfirst(lc);
! if (!first)
! appendStringInfoString(buf, ", ");
! first = false;
! deparseSortGroupClause(grp->tleSortGroupRef, tlist, context);
}
}
--- 3058,3085 ----
*/
Assert(!query->groupingSets);
! /*
! * Due to aggregation push-down (which includes join aggregation on the
! * remote server) we should expect multiple TLEs containing different
! * expressions but having the same sortgroupref. The typical case is that
! * an equality join clause receives the input value (var) from each side
! * of a join, and each of these vars is needed above the join for a
! * different reason. That's why we don't use query->groupClause here.
! *
! * TODO Consider using get_grouping_expressions().
! */
! foreach(lc, tlist)
{
! TargetEntry *te = lfirst_node(TargetEntry, lc);
! if (te->ressortgroupref > 0)
! {
! if (!first)
! appendStringInfoString(buf, ", ");
! first = false;
! deparseSortGroupExpr(te->expr, context);
! }
}
}
*************** appendOrderByClause(List *pathkeys, depa
*** 3012,3020 ****
* Deparses function name from given function oid.
*/
static void
! appendFunctionName(Oid funcid, deparse_expr_cxt *context)
{
- StringInfo buf = context->buf;
HeapTuple proctup;
Form_pg_proc procform;
const char *proname;
--- 3131,3138 ----
* Deparses function name from given function oid.
*/
static void
! appendFunctionName(Oid funcid, StringInfo buf, char *schemaname)
{
HeapTuple proctup;
Form_pg_proc procform;
const char *proname;
*************** appendFunctionName(Oid funcid, deparse_e
*** 3025,3035 ****
procform = (Form_pg_proc) GETSTRUCT(proctup);
/* Print schema name only if it's not pg_catalog */
! if (procform->pronamespace != PG_CATALOG_NAMESPACE)
{
! const char *schemaname;
!
! schemaname = get_namespace_name(procform->pronamespace);
appendStringInfo(buf, "%s.", quote_identifier(schemaname));
}
--- 3143,3153 ----
procform = (Form_pg_proc) GETSTRUCT(proctup);
/* Print schema name only if it's not pg_catalog */
! if (procform->pronamespace != PG_CATALOG_NAMESPACE ||
! schemaname != NULL)
{
! if (schemaname == NULL)
! schemaname = get_namespace_name(procform->pronamespace);
appendStringInfo(buf, "%s.", quote_identifier(schemaname));
}
*************** appendFunctionName(Oid funcid, deparse_e
*** 3049,3061 ****
static Node *
deparseSortGroupClause(Index ref, List *tlist, deparse_expr_cxt *context)
{
- StringInfo buf = context->buf;
TargetEntry *tle;
Expr *expr;
tle = get_sortgroupref_tle(ref, tlist);
expr = tle->expr;
if (expr && IsA(expr, Const))
{
/*
--- 3167,3188 ----
static Node *
deparseSortGroupClause(Index ref, List *tlist, deparse_expr_cxt *context)
{
TargetEntry *tle;
Expr *expr;
tle = get_sortgroupref_tle(ref, tlist);
expr = tle->expr;
+ deparseSortGroupExpr(expr, context);
+
+ return (Node *) expr;
+ }
+
+ static void
+ deparseSortGroupExpr(Expr *expr, deparse_expr_cxt *context)
+ {
+ StringInfo buf = context->buf;
+
if (expr && IsA(expr, Const))
{
/*
*************** deparseSortGroupClause(Index ref, List *
*** 3074,3081 ****
deparseExpr(expr, context);
appendStringInfoChar(buf, ')');
}
-
- return (Node *) expr;
}
--- 3201,3206 ----
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
new file mode 100644
index c379f2f..88eb2c9
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
*************** PG_FUNCTION_INFO_V1(postgres_fdw_handler
*** 273,282 ****
*/
static void postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static void postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static ForeignScan *postgresGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
--- 273,283 ----
*/
static void postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid, bool grouped);
static void postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static ForeignScan *postgresGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
*************** static void postgresGetForeignJoinPaths(
*** 341,347 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static bool postgresRecheckForeignScan(ForeignScanState *node,
TupleTableSlot *slot);
static void postgresGetForeignUpperPaths(PlannerInfo *root,
--- 342,349 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped);
static bool postgresRecheckForeignScan(ForeignScanState *node,
TupleTableSlot *slot);
static void postgresGetForeignUpperPaths(PlannerInfo *root,
*************** static void postgresGetForeignUpperPaths
*** 352,363 ****
/*
* Helper functions
*/
static void estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *baserel,
List *join_conds,
List *pathkeys,
! double *p_rows, int *p_width,
! Cost *p_startup_cost, Cost *p_total_cost);
static void get_remote_estimate(const char *sql,
PGconn *conn,
double *rows,
--- 354,367 ----
/*
* Helper functions
*/
+ static void init_pgfdw_baserel_info(PlannerInfo *root, RelOptInfo *rel,
+ Oid foreigntableid, bool grouped);
static void estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *baserel,
List *join_conds,
List *pathkeys,
! EstimateInfo * estimates,
! bool grouped);
static void get_remote_estimate(const char *sql,
PGconn *conn,
double *rows,
*************** static HeapTuple make_tuple_from_result_
*** 404,416 ****
static void conversion_error_callback(void *arg);
static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
RelOptInfo *rel);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
static void add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path);
static void add_foreign_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
RelOptInfo *grouped_rel);
--- 408,423 ----
static void conversion_error_callback(void *arg);
static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra, bool grouped);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel);
+ static List *create_remote_agg_tlist(PlannerInfo *root,
+ RelOptInfo *grouped_rel,
+ PathTarget *grouping_target);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
RelOptInfo *rel);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
static void add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path, bool grouped);
static void add_foreign_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
RelOptInfo *grouped_rel);
*************** postgres_fdw_handler(PG_FUNCTION_ARGS)
*** 485,566 ****
static void
postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
PgFdwRelationInfo *fpinfo;
! ListCell *lc;
! RangeTblEntry *rte = planner_rt_fetch(baserel->relid, root);
! const char *namespace;
! const char *relname;
! const char *refname;
/*
* We use PgFdwRelationInfo to pass various information to subsequent
* functions.
*/
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! baserel->fdw_private = (void *) fpinfo;
!
! /* Base foreign tables need to be pushed down always. */
! fpinfo->pushdown_safe = true;
!
! /* Look up foreign-table catalog info. */
! fpinfo->table = GetForeignTable(foreigntableid);
! fpinfo->server = GetForeignServer(fpinfo->table->serverid);
!
! /*
! * Extract user-settable option values. Note that per-table setting of
! * use_remote_estimate overrides per-server setting.
! */
! fpinfo->use_remote_estimate = false;
! fpinfo->fdw_startup_cost = DEFAULT_FDW_STARTUP_COST;
! fpinfo->fdw_tuple_cost = DEFAULT_FDW_TUPLE_COST;
! fpinfo->shippable_extensions = NIL;
! fpinfo->fetch_size = 100;
! apply_server_options(fpinfo);
! apply_table_options(fpinfo);
! /*
! * If the table or the server is configured to use remote estimates,
! * identify which user to do remote access as during planning. This
! * should match what ExecCheckRTEPerms() does. If we fail due to lack of
! * permissions, the query would have failed at runtime anyway.
! */
! if (fpinfo->use_remote_estimate)
{
! Oid userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
! fpinfo->user = GetUserMapping(userid, fpinfo->server->serverid);
}
- else
- fpinfo->user = NULL;
! /*
! * Identify which baserestrictinfo clauses can be sent to the remote
! * server and which can't.
! */
! classifyConditions(root, baserel, baserel->baserestrictinfo,
! &fpinfo->remote_conds, &fpinfo->local_conds);
!
! /*
! * Identify which attributes will need to be retrieved from the remote
! * server. These include all attrs needed for joins or final output, plus
! * all attrs used in the local_conds. (Note: if we end up using a
! * parameterized scan, it's possible that some of the join clauses will be
! * sent to the remote and thus we wouldn't really need to retrieve the
! * columns used in them. Doesn't seem worth detecting that case though.)
! */
! fpinfo->attrs_used = NULL;
! pull_varattnos((Node *) baserel->reltarget->exprs, baserel->relid,
! &fpinfo->attrs_used);
! foreach(lc, fpinfo->local_conds)
! {
! RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
!
! pull_varattnos((Node *) rinfo->clause, baserel->relid,
! &fpinfo->attrs_used);
! }
/*
* Compute the selectivity and cost of the local_conds, so we don't have
--- 492,537 ----
static void
postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
PgFdwRelationInfo *fpinfo;
! EstimateInfo *estimates;
/*
* We use PgFdwRelationInfo to pass various information to subsequent
* functions.
+ *
+ * This function can be called twice: first time for plain scan, second
+ * time for the grouped one. fpinfo should only be initialized once.
*/
! if (baserel->fdw_private == NULL)
! init_pgfdw_baserel_info(root, baserel, foreigntableid, grouped);
! fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
! if (grouped)
{
! /*
! * baserestrictinfo must be evaluated below the aggregation, so all
! * its expressions must be remote.
! */
! if (fpinfo->local_conds != NIL)
! return;
! /*
! * Given that there are no local_conds and remote_conds are evaluated
! * on the foreign server, baserel->gpi->target_agg is the only thing
! * left to check.
! *
! * XXX Do we need to verify that the target contains no
! * PlaceHolderVars?
! */
! if (!foreign_grouping_ok(root, baserel))
! return;
}
! estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
/*
* Compute the selectivity and cost of the local_conds, so we don't have
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 580,587 ****
* when they are set to some sensible costs during one (usually the first)
* of the calls to estimate_path_cost_size().
*/
! fpinfo->rel_startup_cost = -1;
! fpinfo->rel_total_cost = -1;
/*
* If the table or the server is configured to use remote estimates,
--- 551,558 ----
* when they are set to some sensible costs during one (usually the first)
* of the calls to estimate_path_cost_size().
*/
! estimates->rel_startup_cost = -1;
! estimates->rel_total_cost = -1;
/*
* If the table or the server is configured to use remote estimates,
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 597,609 ****
* values in fpinfo so we don't need to do it again to generate the
* basic foreign path.
*/
! estimate_path_cost_size(root, baserel, NIL, NIL,
! &fpinfo->rows, &fpinfo->width,
! &fpinfo->startup_cost, &fpinfo->total_cost);
/* Report estimated baserel size to planner. */
! baserel->rows = fpinfo->rows;
! baserel->reltarget->width = fpinfo->width;
}
else
{
--- 568,583 ----
* values in fpinfo so we don't need to do it again to generate the
* basic foreign path.
*/
! estimate_path_cost_size(root, baserel, NIL, NIL, estimates, grouped);
/* Report estimated baserel size to planner. */
!
! /*
! * TODO If grouped, make sure baserel->gpi exists and store the values
! * there.
! */
! baserel->rows = estimates->rows;
! baserel->reltarget->width = estimates->width;
}
else
{
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 628,661 ****
set_baserel_size_estimates(root, baserel);
/* Fill in basically-bogus cost estimates for use later. */
! estimate_path_cost_size(root, baserel, NIL, NIL,
! &fpinfo->rows, &fpinfo->width,
! &fpinfo->startup_cost, &fpinfo->total_cost);
}
-
- /*
- * Set the name of relation in fpinfo, while we are constructing it here.
- * It will be used to build the string describing the join relation in
- * EXPLAIN output. We can't know whether VERBOSE option is specified or
- * not, so always schema-qualify the foreign table name.
- */
- fpinfo->relation_name = makeStringInfo();
- namespace = get_namespace_name(get_rel_namespace(foreigntableid));
- relname = get_rel_name(foreigntableid);
- refname = rte->eref->aliasname;
- appendStringInfo(fpinfo->relation_name, "%s.%s",
- quote_identifier(namespace),
- quote_identifier(relname));
- if (*refname && strcmp(refname, relname) != 0)
- appendStringInfo(fpinfo->relation_name, " %s",
- quote_identifier(rte->eref->aliasname));
-
- /* No outer and inner relations. */
- fpinfo->make_outerrel_subquery = false;
- fpinfo->make_innerrel_subquery = false;
- fpinfo->lower_subquery_rels = NULL;
- /* Set the relation index. */
- fpinfo->relation_index = baserel->relid;
}
/*
--- 602,609 ----
set_baserel_size_estimates(root, baserel);
/* Fill in basically-bogus cost estimates for use later. */
! estimate_path_cost_size(root, baserel, NIL, NIL, estimates, grouped);
}
}
/*
*************** get_useful_pathkeys_for_relation(Planner
*** 884,895 ****
static void
postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
ForeignPath *path;
List *ppi_list;
ListCell *lc;
/*
* Create simplest ForeignScan path node and add it to baserel. This path
--- 832,875 ----
static void
postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
ForeignPath *path;
+ PathTarget *target = NULL;
List *ppi_list;
ListCell *lc;
+ List *fdw_private;
+ EstimateInfo *estimates;
+
+ if (grouped)
+ {
+ /*
+ * See postgresGetForeignRelSize().
+ */
+ if (fpinfo->local_conds != NIL)
+ return;
+
+ /*
+ * Wasn't it possible to construct remote query?
+ */
+ if (fpinfo->grouped_tlist == NIL)
+ return;
+
+ /*
+ * The target must contain aggregates.
+ */
+ Assert(baserel->gpi != NULL);
+ target = baserel->gpi->target_agg;
+ }
+
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+
+ /*
+ * Add information to the paths whether they are grouped or not.
+ */
+ fdw_private = list_make1(makeInteger(grouped ? 1 : 0));
/*
* Create simplest ForeignScan path node and add it to baserel. This path
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 899,916 ****
* to estimate cost and size of this path.
*/
path = create_foreignscan_path(root, baserel,
! NULL, /* default pathtarget */
! fpinfo->rows,
! fpinfo->startup_cost,
! fpinfo->total_cost,
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
/* Add paths with pathkeys */
! add_paths_with_pathkeys_for_rel(root, baserel, NULL);
/*
* If we're not using remote estimates, stop here. We have no way to
--- 879,896 ----
* to estimate cost and size of this path.
*/
path = create_foreignscan_path(root, baserel,
! target,
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! fdw_private);
! add_path(baserel, (Path *) path, grouped);
/* Add paths with pathkeys */
! add_paths_with_pathkeys_for_rel(root, baserel, NULL, grouped);
/*
* If we're not using remote estimates, stop here. We have no way to
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 921,926 ****
--- 901,914 ----
return;
/*
+ * Grouped parameterized path would lead to repeated aggregation, which is
+ * not too interesting. Since the rest deals only with parameterized
+ * paths, return now.
+ */
+ if (grouped)
+ return;
+
+ /*
* Thumb through all join clauses for the rel to identify which outer
* relations could supply one or more safe-to-send-to-remote join clauses.
* We'll build a parameterized path for each such outer relation.
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 1052,1085 ****
foreach(lc, ppi_list)
{
ParamPathInfo *param_info = (ParamPathInfo *) lfirst(lc);
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
/* Get a cost estimate from the remote */
estimate_path_cost_size(root, baserel,
! param_info->ppi_clauses, NIL,
! &rows, &width,
! &startup_cost, &total_cost);
/*
* ppi_rows currently won't get looked at by anything, but still we
* may as well ensure that it matches our idea of the rowcount.
*/
! param_info->ppi_rows = rows;
/* Make the path */
path = create_foreignscan_path(root, baserel,
NULL, /* default pathtarget */
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
param_info->ppi_req_outer,
NULL,
! NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
}
}
--- 1040,1070 ----
foreach(lc, ppi_list)
{
ParamPathInfo *param_info = (ParamPathInfo *) lfirst(lc);
! EstimateInfo est_param;
/* Get a cost estimate from the remote */
+ memset(&est_param, 0, sizeof(EstimateInfo));
estimate_path_cost_size(root, baserel,
! param_info->ppi_clauses, NIL, &est_param,
! grouped);
/*
* ppi_rows currently won't get looked at by anything, but still we
* may as well ensure that it matches our idea of the rowcount.
*/
! param_info->ppi_rows = est_param.rows;
/* Make the path */
path = create_foreignscan_path(root, baserel,
NULL, /* default pathtarget */
! est_param.rows,
! est_param.startup_cost,
! est_param.total_cost,
NIL, /* no pathkeys */
param_info->ppi_req_outer,
NULL,
! fdw_private);
! add_path(baserel, (Path *) path, grouped);
}
}
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1107,1119 ****
List *retrieved_attrs;
StringInfoData sql;
ListCell *lc;
if (IS_SIMPLE_REL(foreignrel))
{
! /*
! * For base relations, set scan_relid as the relid of the relation.
! */
! scan_relid = foreignrel->relid;
/*
* In a base-relation scan, we must apply the given scan_clauses.
--- 1092,1130 ----
List *retrieved_attrs;
StringInfoData sql;
ListCell *lc;
+ Value *grouped_val;
+ bool grouped;
+
+ Assert(best_path->fdw_private != NIL);
+ grouped_val = (Value *) linitial(best_path->fdw_private);
+ Assert(IsA(grouped_val, Integer));
+ grouped = intVal(grouped_val) == 1;
if (IS_SIMPLE_REL(foreignrel))
{
! if (!grouped)
! {
! /*
! * For base relations, set scan_relid as the relid of the
! * relation.
! */
! scan_relid = foreignrel->relid;
! }
! else
! {
! /*
! * XXX Something seems to be inconsistent here: deparseSelectSql()
! * would create the targetlist from foreignrel->gpi->target_agg,
! * but not appendGroupByClause(). Try to refactor.
! */
! fdw_scan_tlist = fpinfo->grouped_tlist;
!
! /*
! * The scan tuple descriptor should be constructed from
! * fdw_scan_tlist rather than from that of the remote relation.
! */
! scan_relid = 0;
! }
/*
* In a base-relation scan, we must apply the given scan_clauses.
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1191,1197 ****
*/
/* Build the list of columns to be fetched from the foreign server. */
! fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
/*
* Ensure that the outer plan produces a tuple whose descriptor
--- 1202,1211 ----
*/
/* Build the list of columns to be fetched from the foreign server. */
! if (!grouped)
! fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
! else
! fdw_scan_tlist = fpinfo->grouped_tlist;
/*
* Ensure that the outer plan produces a tuple whose descriptor
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1239,1245 ****
initStringInfo(&sql);
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_exprs, best_path->path.pathkeys,
! false, &retrieved_attrs, ¶ms_list);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
--- 1253,1260 ----
initStringInfo(&sql);
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_exprs, best_path->path.pathkeys,
! false, &retrieved_attrs, ¶ms_list,
! grouped);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
*************** postgresExplainDirectModify(ForeignScanS
*** 2480,2485 ****
--- 2495,2620 ----
}
}
+ /*
+ * init_pgfdw_rel_info
+ * Initialize PgFdwRelationInfo and assign it to the base relation. That
+ * includes estimates, as well as separation of local expressions from
+ * remote ones.
+ */
+ static void
+ init_pgfdw_baserel_info(PlannerInfo *root, RelOptInfo *rel,
+ Oid foreigntableid, bool grouped)
+
+ {
+ const char *namespace;
+ const char *relname;
+ const char *refname;
+ PgFdwRelationInfo *fpinfo;
+ ListCell *lc;
+ RangeTblEntry *rte;
+
+ rte = planner_rt_fetch(rel->relid, root);
+ fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
+
+ /*
+ * Identify which baserestrictinfo clauses can be sent to the remote
+ * server and which can't.
+ */
+ classifyConditions(root, rel, rel->baserestrictinfo,
+ &fpinfo->remote_conds, &fpinfo->local_conds);
+
+ /*
+ * Identify which attributes will need to be retrieved from the remote
+ * server. These include all attrs needed for joins or final output, plus
+ * all attrs used in the local_conds. (Note: if we end up using a
+ * parameterized scan, it's possible that some of the join clauses will be
+ * sent to the remote and thus we wouldn't really need to retrieve the
+ * columns used in them. Doesn't seem worth detecting that case though.)
+ */
+ fpinfo->attrs_used = NULL;
+ if (!grouped)
+ {
+ pull_varattnos((Node *) rel->reltarget->exprs, rel->relid,
+ &fpinfo->attrs_used);
+ foreach(lc, fpinfo->local_conds)
+ {
+ RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
+
+ pull_varattnos((Node *) rinfo->clause, rel->relid,
+ &fpinfo->attrs_used);
+ }
+ }
+ else
+ {
+ /*
+ * deparseExplicitTargetList() is used for the grouped relation, thus
+ * no need to retrieve attrs_used.
+ */
+ }
+
+ /* Base foreign tables need to be pushed down always. */
+ fpinfo->pushdown_safe = true;
+
+ /* Look up foreign-table catalog info. */
+ fpinfo->table = GetForeignTable(foreigntableid);
+ fpinfo->server = GetForeignServer(fpinfo->table->serverid);
+
+ /*
+ * Extract user-settable option values. Note that per-table setting of
+ * use_remote_estimate overrides per-server setting.
+ */
+ fpinfo->use_remote_estimate = false;
+ fpinfo->fdw_startup_cost = DEFAULT_FDW_STARTUP_COST;
+ fpinfo->fdw_tuple_cost = DEFAULT_FDW_TUPLE_COST;
+ fpinfo->shippable_extensions = NIL;
+ fpinfo->fetch_size = 100;
+
+ apply_server_options(fpinfo);
+ apply_table_options(fpinfo);
+
+ /*
+ * If the table or the server is configured to use remote estimates,
+ * identify which user to do remote access as during planning. This
+ * should match what ExecCheckRTEPerms() does. If we fail due to lack of
+ * permissions, the query would have failed at runtime anyway.
+ */
+ if (fpinfo->use_remote_estimate)
+ {
+ Oid userid = rte->checkAsUser ? rte->checkAsUser :
+ GetUserId();
+
+ fpinfo->user = GetUserMapping(userid, fpinfo->server->serverid);
+ }
+ else
+ fpinfo->user = NULL;
+
+ /*
+ * Set the name of relation in fpinfo, while we are constructing it here.
+ * It will be used to build the string describing the join relation in
+ * EXPLAIN output. We can't know whether VERBOSE option is specified or
+ * not, so always schema-qualify the foreign table name.
+ */
+ fpinfo->relation_name = makeStringInfo();
+ namespace = get_namespace_name(get_rel_namespace(foreigntableid));
+ relname = get_rel_name(foreigntableid);
+ refname = rte->eref->aliasname;
+ appendStringInfo(fpinfo->relation_name, "%s.%s",
+ quote_identifier(namespace),
+ quote_identifier(relname));
+ if (*refname && strcmp(refname, relname) != 0)
+ appendStringInfo(fpinfo->relation_name, " %s",
+ quote_identifier(rte->eref->aliasname));
+
+ /* No outer and inner relations. */
+ fpinfo->make_outerrel_subquery = false;
+ fpinfo->make_innerrel_subquery = false;
+ fpinfo->lower_subquery_rels = NULL;
+ /* Set the relation index. */
+ fpinfo->relation_index = rel->relid;
+
+ rel->fdw_private = (void *) fpinfo;
+ }
+
/*
* estimate_path_cost_size
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2498,2505 ****
RelOptInfo *foreignrel,
List *param_join_conds,
List *pathkeys,
! double *p_rows, int *p_width,
! Cost *p_startup_cost, Cost *p_total_cost)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
double rows;
--- 2633,2640 ----
RelOptInfo *foreignrel,
List *param_join_conds,
List *pathkeys,
! EstimateInfo * estimates,
! bool grouped)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
double rows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2538,2545 ****
&remote_param_join_conds, &local_param_join_conds);
/* Build the list of columns to be fetched from the foreign server. */
! if (IS_JOIN_REL(foreignrel) || IS_UPPER_REL(foreignrel))
fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
else
fdw_scan_tlist = NIL;
--- 2673,2685 ----
&remote_param_join_conds, &local_param_join_conds);
/* Build the list of columns to be fetched from the foreign server. */
! if ((IS_JOIN_REL(foreignrel) && !grouped) || IS_UPPER_REL(foreignrel))
fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
+ else if (grouped)
+ {
+ Assert(fpinfo->grouped_tlist != NIL);
+ fdw_scan_tlist = fpinfo->grouped_tlist;
+ }
else
fdw_scan_tlist = NIL;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2560,2566 ****
appendStringInfoString(&sql, "EXPLAIN ");
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_conds, pathkeys, false,
! &retrieved_attrs, NULL);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
--- 2700,2706 ----
appendStringInfoString(&sql, "EXPLAIN ");
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_conds, pathkeys, false,
! &retrieved_attrs, NULL, grouped);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2614,2628 ****
* bare scan each time. Instead, use the costs if we have cached them
* already.
*/
! if (fpinfo->rel_startup_cost > 0 && fpinfo->rel_total_cost > 0)
{
! startup_cost = fpinfo->rel_startup_cost;
! run_cost = fpinfo->rel_total_cost - fpinfo->rel_startup_cost;
}
else if (IS_JOIN_REL(foreignrel))
{
! PgFdwRelationInfo *fpinfo_i;
! PgFdwRelationInfo *fpinfo_o;
QualCost join_cost;
QualCost remote_conds_cost;
double nrows;
--- 2754,2770 ----
* bare scan each time. Instead, use the costs if we have cached them
* already.
*/
! if (estimates->rel_startup_cost > 0 && estimates->rel_total_cost > 0)
{
! startup_cost = estimates->rel_startup_cost;
! run_cost = estimates->rel_total_cost - estimates->rel_startup_cost;
}
else if (IS_JOIN_REL(foreignrel))
{
! PgFdwRelationInfo *fpinfo_i,
! *fpinfo_o;
! EstimateInfo *est_i,
! *est_o;
QualCost join_cost;
QualCost remote_conds_cost;
double nrows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2631,2640 ****
Assert(fpinfo->innerrel && fpinfo->outerrel);
fpinfo_i = (PgFdwRelationInfo *) fpinfo->innerrel->fdw_private;
fpinfo_o = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/* Estimate of number of rows in cross product */
! nrows = fpinfo_i->rows * fpinfo_o->rows;
/* Clamp retrieved rows estimate to at most size of cross product */
retrieved_rows = Min(retrieved_rows, nrows);
--- 2773,2784 ----
Assert(fpinfo->innerrel && fpinfo->outerrel);
fpinfo_i = (PgFdwRelationInfo *) fpinfo->innerrel->fdw_private;
+ est_i = !grouped ? &fpinfo_i->est_plain : &fpinfo_i->est_grouped;
fpinfo_o = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
+ est_o = !grouped ? &fpinfo_o->est_plain : &fpinfo_o->est_grouped;
/* Estimate of number of rows in cross product */
! nrows = est_i->rows * est_o->rows;
/* Clamp retrieved rows estimate to at most size of cross product */
retrieved_rows = Min(retrieved_rows, nrows);
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2658,2664 ****
* tables) since we do not know what strategy the foreign server
* is going to use.
*/
! startup_cost = fpinfo_i->rel_startup_cost + fpinfo_o->rel_startup_cost;
startup_cost += join_cost.startup;
startup_cost += remote_conds_cost.startup;
startup_cost += fpinfo->local_conds_cost.startup;
--- 2802,2808 ----
* tables) since we do not know what strategy the foreign server
* is going to use.
*/
! startup_cost = est_i->rel_startup_cost + est_o->rel_startup_cost;
startup_cost += join_cost.startup;
startup_cost += remote_conds_cost.startup;
startup_cost += fpinfo->local_conds_cost.startup;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2678,2694 ****
* 4. Run time cost of applying nonpushable other clauses locally
* on the result fetched from the foreign server.
*/
! run_cost = fpinfo_i->rel_total_cost - fpinfo_i->rel_startup_cost;
! run_cost += fpinfo_o->rel_total_cost - fpinfo_o->rel_startup_cost;
run_cost += nrows * join_cost.per_tuple;
nrows = clamp_row_est(nrows * fpinfo->joinclause_sel);
run_cost += nrows * remote_conds_cost.per_tuple;
run_cost += fpinfo->local_conds_cost.per_tuple * retrieved_rows;
}
! else if (IS_UPPER_REL(foreignrel))
{
PgFdwRelationInfo *ofpinfo;
! PathTarget *ptarget = root->upper_targets[UPPERREL_GROUP_AGG];
AggClauseCosts aggcosts;
double input_rows;
int numGroupCols;
--- 2822,2850 ----
* 4. Run time cost of applying nonpushable other clauses locally
* on the result fetched from the foreign server.
*/
! run_cost = est_i->rel_total_cost - est_i->rel_startup_cost;
! run_cost += est_o->rel_total_cost - est_o->rel_startup_cost;
run_cost += nrows * join_cost.per_tuple;
nrows = clamp_row_est(nrows * fpinfo->joinclause_sel);
run_cost += nrows * remote_conds_cost.per_tuple;
run_cost += fpinfo->local_conds_cost.per_tuple * retrieved_rows;
}
!
! /*
! * TODO Consider a separate branch for
! *
! * (IS_SIMPLE_REL(foreignrel) && * grouped)
! *
! * and use foreignrel->gpi->target_agg in it instead of
! * fpinfo->grouped_tlist.
! */
! else if (IS_UPPER_REL(foreignrel) ||
! (IS_SIMPLE_REL(foreignrel) && grouped)
! )
{
PgFdwRelationInfo *ofpinfo;
! EstimateInfo *est;
! PathTarget *ptarget;
AggClauseCosts aggcosts;
double input_rows;
int numGroupCols;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2706,2716 ****
* considering remote and local conditions for costing.
*/
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/* Get rows and width from input rel */
! input_rows = ofpinfo->rows;
! width = ofpinfo->width;
/* Collect statistics about aggregates for estimating costs. */
MemSet(&aggcosts, 0, sizeof(AggClauseCosts));
--- 2862,2874 ----
* considering remote and local conditions for costing.
*/
! ofpinfo = IS_UPPER_REL(foreignrel) ?
! (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private : fpinfo;
! est = !grouped ? &ofpinfo->est_plain : &ofpinfo->est_grouped;
/* Get rows and width from input rel */
! input_rows = est->rows;
! width = est->width;
/* Collect statistics about aggregates for estimating costs. */
MemSet(&aggcosts, 0, sizeof(AggClauseCosts));
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2735,2740 ****
--- 2893,2906 ----
*/
rows = retrieved_rows = numGroups;
+ if (IS_UPPER_REL(foreignrel))
+ ptarget = root->upper_targets[UPPERREL_GROUP_AGG];
+ else
+ {
+ Assert(foreignrel->gpi != NULL);
+ ptarget = foreignrel->gpi->target_agg;
+ }
+
/*-----
* Startup cost includes:
* 1. Startup cost for underneath input * relation
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2742,2748 ****
* 3. Startup cost for PathTarget eval
*-----
*/
! startup_cost = ofpinfo->rel_startup_cost;
startup_cost += aggcosts.transCost.startup;
startup_cost += aggcosts.transCost.per_tuple * input_rows;
startup_cost += (cpu_operator_cost * numGroupCols) * input_rows;
--- 2908,2914 ----
* 3. Startup cost for PathTarget eval
*-----
*/
! startup_cost = est->rel_startup_cost;
startup_cost += aggcosts.transCost.startup;
startup_cost += aggcosts.transCost.per_tuple * input_rows;
startup_cost += (cpu_operator_cost * numGroupCols) * input_rows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2755,2761 ****
* 3. PathTarget eval cost for each output row
*-----
*/
! run_cost = ofpinfo->rel_total_cost - ofpinfo->rel_startup_cost;
run_cost += aggcosts.finalCost * numGroups;
run_cost += cpu_tuple_cost * numGroups;
run_cost += ptarget->cost.per_tuple * numGroups;
--- 2921,2927 ----
* 3. PathTarget eval cost for each output row
*-----
*/
! run_cost = est->rel_total_cost - est->rel_startup_cost;
run_cost += aggcosts.finalCost * numGroups;
run_cost += cpu_tuple_cost * numGroups;
run_cost += ptarget->cost.per_tuple * numGroups;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2808,2815 ****
*/
if (pathkeys == NIL && param_join_conds == NIL)
{
! fpinfo->rel_startup_cost = startup_cost;
! fpinfo->rel_total_cost = total_cost;
}
/*
--- 2974,2981 ----
*/
if (pathkeys == NIL && param_join_conds == NIL)
{
! estimates->rel_startup_cost = startup_cost;
! estimates->rel_total_cost = total_cost;
}
/*
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2824,2833 ****
total_cost += cpu_tuple_cost * retrieved_rows;
/* Return results. */
! *p_rows = rows;
! *p_width = width;
! *p_startup_cost = startup_cost;
! *p_total_cost = total_cost;
}
/*
--- 2990,2999 ----
total_cost += cpu_tuple_cost * retrieved_rows;
/* Return results. */
! estimates->rows = rows;
! estimates->width = width;
! estimates->startup_cost = startup_cost;
! estimates->total_cost = total_cost;
}
/*
*************** postgresImportForeignSchema(ImportForeig
*** 4056,4074 ****
* Assess whether the join between inner and outer relations can be pushed down
* to the foreign server. As a side effect, save information we obtain in this
* function to PgFdwRelationInfo passed in.
*/
static bool
foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra)
{
! PgFdwRelationInfo *fpinfo;
! PgFdwRelationInfo *fpinfo_o;
! PgFdwRelationInfo *fpinfo_i;
ListCell *lc;
List *joinclauses;
/*
* We support pushing down INNER, LEFT, RIGHT and FULL OUTER joins.
* Constructing queries representing SEMI and ANTI joins is hard, hence
* not considered right now.
--- 4222,4251 ----
* Assess whether the join between inner and outer relations can be pushed down
* to the foreign server. As a side effect, save information we obtain in this
* function to PgFdwRelationInfo passed in.
+ *
+ * Caller expects that the "grouped" argument does not affect the result, so
+ * it's enough to call the function only once per relation. XXX As that
+ * argument is only used in Assert() statement, shouldn't it be removed?
*/
static bool
foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra, bool grouped)
{
! PgFdwRelationInfo *fpinfo,
! *fpinfo_o,
! *fpinfo_i;
ListCell *lc;
List *joinclauses;
/*
+ * TODO
+ *
+ * Put the (join-relevant) initial checks of foreign_grouping_ok into a
+ * function and call it here.
+ */
+
+ /*
* We support pushing down INNER, LEFT, RIGHT and FULL OUTER joins.
* Constructing queries representing SEMI and ANTI joins is hard, hence
* not considered right now.
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4138,4144 ****
--- 4315,4331 ----
if (is_remote_clause)
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
else
+ {
+ /*
+ * We could return in the grouped case because all clauses
+ * must be evaluated below the grouping plan (Thus if the
+ * grouping is remote, those clauses must be remote too.)
+ * However the same PgFdwRelationInfo is used for both grouped
+ * and non-grouped case and we don't want to force caller to
+ * process the non-grouped case first.
+ */
fpinfo->local_conds = lappend(fpinfo->local_conds, rinfo);
+ }
}
}
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4157,4165 ****
PlaceHolderInfo *phinfo = lfirst(lc);
Relids relids = joinrel->relids;
! if (bms_is_subset(phinfo->ph_eval_at, relids) &&
! bms_nonempty_difference(relids, phinfo->ph_eval_at))
! return false;
}
/* Save the join clauses, for later use. */
--- 4344,4369 ----
PlaceHolderInfo *phinfo = lfirst(lc);
Relids relids = joinrel->relids;
! if (bms_is_subset(phinfo->ph_eval_at, relids))
! {
! if (bms_nonempty_difference(relids, phinfo->ph_eval_at))
! {
! #ifdef USE_ASSERT_CHECKING
! if (grouped)
! {
! /*
! * Grouped join shouldn't have PlaceHolderVar even in its
! * own targetlist. It'd mean that the output of partial
! * aggregation can be set to NULL before it gets to the
! * input of the final aggregation.
! */
! Assert(false);
! }
! #endif /* USE_ASSERT_CHECKING */
!
! return false;
! }
! }
}
/* Save the join clauses, for later use. */
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4282,4295 ****
fpinfo->user = NULL;
/*
- * Set cached relation costs to some negative value, so that we can detect
- * when they are set to some sensible costs, during one (usually the
- * first) of the calls to estimate_path_cost_size().
- */
- fpinfo->rel_startup_cost = -1;
- fpinfo->rel_total_cost = -1;
-
- /*
* Set the string describing this join relation to be used in EXPLAIN
* output of corresponding ForeignScan.
*/
--- 4486,4491 ----
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4314,4349 ****
static void
add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path)
{
List *useful_pathkeys_list = NIL; /* List of all pathkeys */
ListCell *lc;
useful_pathkeys_list = get_useful_pathkeys_for_relation(root, rel);
/* Create one path for each set of pathkeys we found above. */
foreach(lc, useful_pathkeys_list)
{
- double rows;
- int width;
- Cost startup_cost;
- Cost total_cost;
List *useful_pathkeys = lfirst(lc);
estimate_path_cost_size(root, rel, NIL, useful_pathkeys,
! &rows, &width, &startup_cost, &total_cost);
add_path(rel, (Path *)
create_foreignscan_path(root, rel,
! NULL,
! rows,
! startup_cost,
! total_cost,
useful_pathkeys,
NULL,
epq_path,
! NIL),
! false);
}
}
--- 4510,4558 ----
static void
add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path, bool grouped)
{
List *useful_pathkeys_list = NIL; /* List of all pathkeys */
ListCell *lc;
+ List *fdw_private;
+ PathTarget *target = NULL;
useful_pathkeys_list = get_useful_pathkeys_for_relation(root, rel);
+ /*
+ * Add information to the paths whether they are grouped or not.
+ */
+ fdw_private = list_make1(makeInteger(grouped ? 1 : 0));
+
+ /*
+ * Grouped foreign scan should emit tuples according to the grouped
+ * target, i.e. the one containing GroupedVars.
+ */
+ if (grouped)
+ {
+ Assert(rel->gpi != NULL);
+ target = rel->gpi->target_agg;
+ }
+
/* Create one path for each set of pathkeys we found above. */
foreach(lc, useful_pathkeys_list)
{
List *useful_pathkeys = lfirst(lc);
+ EstimateInfo estimates;
estimate_path_cost_size(root, rel, NIL, useful_pathkeys,
! &estimates, grouped);
add_path(rel, (Path *)
create_foreignscan_path(root, rel,
! target,
! estimates.rows,
! estimates.startup_cost,
! estimates.total_cost,
useful_pathkeys,
NULL,
epq_path,
! fdw_private), grouped);
}
}
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4461,4495 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
PgFdwRelationInfo *fpinfo;
ForeignPath *joinpath;
- double rows;
- int width;
- Cost startup_cost;
- Cost total_cost;
Path *epq_path; /* Path to create plan to be executed when
* EvalPlanQual gets triggered. */
! /*
! * Skip if this join combination has been considered already.
! */
! if (joinrel->fdw_private)
! return;
! /*
! * Create unfinished PgFdwRelationInfo entry which is used to indicate
! * that the join relation is already considered, so that we won't waste
! * time in judging safety of join pushdown and adding the same paths again
! * if found safe. Once we know that this join can be pushed down, we fill
! * the entry.
! */
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! fpinfo->pushdown_safe = false;
! joinrel->fdw_private = fpinfo;
! /* attrs_used is only for base relations. */
! fpinfo->attrs_used = NULL;
/*
* If there is a possibility that EvalPlanQual will be executed, we need
--- 4670,4720 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
PgFdwRelationInfo *fpinfo;
+ EstimateInfo *estimates;
ForeignPath *joinpath;
Path *epq_path; /* Path to create plan to be executed when
* EvalPlanQual gets triggered. */
+ bool first_time;
+ PathTarget *target = NULL;
+ List *fdw_private;
! if (joinrel->fdw_private == NULL)
! {
! /*
! * First time through.
! *
! * Create unfinished PgFdwRelationInfo entry which is used to indicate
! * that the join relation is already considered, so that we won't
! * waste time in judging safety of join pushdown and adding the same
! * paths again if found safe. Once we know that this join can be
! * pushed down, we fill the entry.
! */
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! fpinfo->pushdown_safe = false;
! joinrel->fdw_private = fpinfo;
! /* attrs_used is only for base relations. */
! fpinfo->attrs_used = NULL;
! first_time = true;
! }
! else
! {
! fpinfo = (PgFdwRelationInfo *) joinrel->fdw_private;
! first_time = false;
!
! if (!fpinfo->pushdown_safe)
! {
! /*
! * The function was already called but some test failed. No reason
! * to see that failure again.
! */
! return;
! }
! }
/*
* If there is a possibility that EvalPlanQual will be executed, we need
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4501,4527 ****
* reconstruct the row for EvalPlanQual(). Find an alternative local path
* calling foreign_join_ok(), since that function updates fpinfo and marks
* it as pushable if the join is found to be pushable.
*/
! if (root->parse->commandType == CMD_DELETE ||
! root->parse->commandType == CMD_UPDATE ||
! root->rowMarks)
{
! epq_path = GetExistingLocalJoinPath(joinrel);
! if (!epq_path)
{
! elog(DEBUG3, "could not push down foreign join because a local path suitable for EPQ checks was not found");
return;
}
}
- else
- epq_path = NULL;
! if (!foreign_join_ok(root, joinrel, jointype, outerrel, innerrel, extra))
! {
! /* Free path required for EPQ if we copied one; we don't need it now */
! if (epq_path)
! pfree(epq_path);
return;
}
/*
--- 4726,4803 ----
* reconstruct the row for EvalPlanQual(). Find an alternative local path
* calling foreign_join_ok(), since that function updates fpinfo and marks
* it as pushable if the join is found to be pushable.
+ *
+ * The following tests should only be performed once --- repeated, the are
+ * supposed to yield the same result. (If anything failed initially,
+ * pushdown_safe should have caused return above.)
*/
! if (first_time)
{
! if (root->parse->commandType == CMD_DELETE ||
! root->parse->commandType == CMD_UPDATE ||
! root->rowMarks)
{
! epq_path = GetExistingLocalJoinPath(joinrel);
! if (!epq_path)
! {
! elog(DEBUG3, "could not push down foreign join because a local path suitable for EPQ checks was not found");
! return;
! }
! }
! else
! epq_path = NULL;
!
! if (!foreign_join_ok(root, joinrel, jointype, outerrel, innerrel, extra,
! grouped))
! {
! /*
! * Free path required for EPQ if we copied one; we don't need it
! * now
! */
! if (epq_path)
! pfree(epq_path);
return;
}
}
! /*
! * Local expressions would be evaluated above the grouping plan, which
! * will be remote.
! */
! if (grouped && fpinfo->local_conds != NIL)
return;
+
+ /*
+ * This function should not be called repeatedly wit the same value of
+ * "grouped", but if it happened, make sure the "upper conditions" do not
+ * get duplicated or used at all if !grouped.
+ */
+ if (fpinfo->remote_conds_upper)
+ {
+ list_free(fpinfo->remote_conds_upper);
+ fpinfo->remote_conds_upper = NIL;
+ }
+ if (fpinfo->local_conds_upper)
+ {
+ list_free(fpinfo->local_conds_upper);
+ fpinfo->local_conds_upper = NIL;
+ }
+
+ if (grouped)
+ {
+ /*
+ * Create targelist to be used for the remote grouping.
+ */
+ Assert(joinrel->gpi != NULL);
+ fpinfo->grouped_tlist = create_remote_agg_tlist(root, joinrel,
+ joinrel->gpi->target_agg);
+
+ /*
+ * Couldn't push any GROUP BY expression or aggregate to the remote
+ * server?
+ */
+ if (fpinfo->grouped_tlist == NIL)
+ return;
}
/*
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4548,4563 ****
0, fpinfo->jointype,
extra->sjinfo);
/* Estimate costs for bare join relation */
! estimate_path_cost_size(root, joinrel, NIL, NIL, &rows,
! &width, &startup_cost, &total_cost);
/* Now update this information in the joinrel */
! joinrel->rows = rows;
! joinrel->reltarget->width = width;
! fpinfo->rows = rows;
! fpinfo->width = width;
! fpinfo->startup_cost = startup_cost;
! fpinfo->total_cost = total_cost;
/*
* Create a new join path and add it to the joinrel which represents a
--- 4824,4867 ----
0, fpinfo->jointype,
extra->sjinfo);
+ /*
+ * Set cached relation costs to some negative value, so that we can detect
+ * when they are set to some sensible costs, during one (usually the
+ * first) of the calls to estimate_path_cost_size().
+ */
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+ estimates->rel_startup_cost = -1;
+ estimates->rel_total_cost = -1;
+
/* Estimate costs for bare join relation */
!
! /*
! * TODO Include remote_conds_upper and local_conds_upper into the
! * estimates.
! */
! estimate_path_cost_size(root, joinrel, NIL, NIL, estimates, grouped);
/* Now update this information in the joinrel */
!
! /*
! * TODO If grouped, make sure baserel->gpi exists and store the values
! * there.
! */
! joinrel->rows = estimates->rows;
! joinrel->reltarget->width = estimates->width;
!
! if (grouped)
! {
! /*
! * The target must contain aggregates.
! */
! Assert(joinrel->gpi != NULL);
! target = joinrel->gpi->target_agg;
! }
!
! /*
! * Add information to the paths whether they are grouped or not.
! */
! fdw_private = list_make1(makeInteger(grouped ? 1 : 0));
/*
* Create a new join path and add it to the joinrel which represents a
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4565,4584 ****
*/
joinpath = create_foreignscan_path(root,
joinrel,
! NULL, /* default pathtarget */
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
epq_path,
! NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, false);
/* Consider pathkeys for the join relation */
! add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
/* XXX Consider parameterized paths for the join relation */
}
--- 4869,4888 ----
*/
joinpath = create_foreignscan_path(root,
joinrel,
! target, /* default pathtarget */
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
epq_path,
! fdw_private); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, grouped);
/* Consider pathkeys for the join relation */
! add_paths_with_pathkeys_for_rel(root, joinrel, epq_path, grouped);
/* XXX Consider parameterized paths for the join relation */
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4594,4618 ****
Query *query = root->parse;
PathTarget *grouping_target;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
! PgFdwRelationInfo *ofpinfo;
! List *aggvars;
! ListCell *lc;
! int i;
! List *tlist = NIL;
/* Grouping Sets are not pushable */
if (query->groupingSets)
return false;
/* Get the fpinfo of the underlying scan relation. */
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/*
* If underneath input relation has any local conditions, those conditions
* are required to be applied before performing aggregation. Hence the
* aggregate cannot be pushed down.
*/
! if (ofpinfo->local_conds)
return false;
/*
--- 4898,4920 ----
Query *query = root->parse;
PathTarget *grouping_target;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
! PgFdwRelationInfo *ofpinfo = NULL;
/* Grouping Sets are not pushable */
if (query->groupingSets)
return false;
/* Get the fpinfo of the underlying scan relation. */
! if (IS_UPPER_REL(grouped_rel))
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/*
* If underneath input relation has any local conditions, those conditions
* are required to be applied before performing aggregation. Hence the
* aggregate cannot be pushed down.
*/
! if ((IS_UPPER_REL(grouped_rel) && ofpinfo->local_conds) ||
! fpinfo->local_conds)
return false;
/*
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4623,4637 ****
* different from those in the plan's targetlist. Use a copy of path
* target to record the new sortgrouprefs.
*/
! grouping_target = copy_pathtarget(root->upper_targets[UPPERREL_GROUP_AGG]);
/*
! * Evaluate grouping targets and check whether they are safe to push down
! * to the foreign side. All GROUP BY expressions will be part of the
! * grouping target and thus there is no need to evaluate it separately.
! * While doing so, add required expressions into target list which can
! * then be used to pass to foreign server.
*/
i = 0;
foreach(lc, grouping_target->exprs)
{
--- 4925,4993 ----
* different from those in the plan's targetlist. Use a copy of path
* target to record the new sortgrouprefs.
*/
! if (IS_UPPER_REL(grouped_rel))
! grouping_target = root->upper_targets[UPPERREL_GROUP_AGG];
! else
! {
! /*
! * FDW should not be asked to generate grouped paths if the simple or
! * join relation has no relevant target.
! */
! Assert(grouped_rel->gpi != NULL);
! Assert(grouped_rel->gpi->target_agg != NULL);
!
! grouping_target = grouped_rel->gpi->target_agg;
! }
/*
! * Create targelist to be sent to the remote server.
*/
+ fpinfo->grouped_tlist = create_remote_agg_tlist(root, grouped_rel,
+ grouping_target);
+ if (fpinfo->grouped_tlist == NIL)
+ return false;
+
+ /* Safe to pushdown */
+ fpinfo->pushdown_safe = true;
+
+ /*
+ * Set the string describing this grouped relation to be used in EXPLAIN
+ * output of corresponding ForeignScan.
+ */
+ if (ofpinfo != NULL)
+ {
+ fpinfo->relation_name = makeStringInfo();
+ appendStringInfo(fpinfo->relation_name, "Aggregate on (%s)",
+ ofpinfo->relation_name->data);
+ }
+
+ return true;
+ }
+
+ /*
+ * Evaluate grouping targets and check whether they are safe to push down to
+ * the foreign side. All GROUP BY expressions will be part of the grouping
+ * target and thus there is no need to evaluate it separately. While doing
+ * so, add required expressions into target list which can then be used to
+ * pass to foreign server.
+ *
+ * NIL is returned if the remote aggregation appears to be impossible.
+ *
+ * Note that the function can add items to both remote_conds_upper and
+ * local_conds_upper lists of PgFdwRelationInfo.
+ */
+ static List *
+ create_remote_agg_tlist(PlannerInfo *root, RelOptInfo *grouped_rel,
+ PathTarget *grouping_target)
+ {
+ List *tlist = NIL;
+ ListCell *lc;
+ int i;
+ Query *query = root->parse;
+ PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
+
+ grouping_target = copy_pathtarget(grouping_target);
+
i = 0;
foreach(lc, grouping_target->exprs)
{
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4647,4659 ****
* push down aggregation to the foreign server.
*/
if (!is_foreign_expr(root, grouped_rel, expr))
! return false;
/* Pushable, add to tlist */
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
else
{
/* Check entire expression whether it is pushable or not */
if (is_foreign_expr(root, grouped_rel, expr))
{
--- 5003,5036 ----
* push down aggregation to the foreign server.
*/
if (!is_foreign_expr(root, grouped_rel, expr))
! return NIL;
/* Pushable, add to tlist */
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
else
{
+ /*
+ * Aggregate can sometimes be wrapped by GroupedVar.
+ *
+ */
+ if ((IS_SIMPLE_REL(grouped_rel) || IS_JOIN_REL(grouped_rel)) &&
+ IsA(expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, expr);
+
+ /*
+ * GroupedVar is currently used only to handle Aggrefs.
+ *
+ * XXX These may include aggregates that appear in the HAVING
+ * clause. Although the HAVING clause is currently not sent to
+ * the remote server (see deparseSelectStmtForRel), we still
+ * need to send the aggregates the HAVING clause contains.
+ */
+ Assert(IsA(gvar->agg_partial, Aggref));
+ expr = (Expr *) gvar->agg_partial;
+ }
+
/* Check entire expression whether it is pushable or not */
if (is_foreign_expr(root, grouped_rel, expr))
{
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4662,4667 ****
--- 5039,5046 ----
}
else
{
+ List *aggvars;
+
/*
* If we have sortgroupref set, then it means that we have an
* ORDER BY entry pointing to this expression. Since we are
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4670,4681 ****
if (sgref)
grouping_target->sortgrouprefs[i] = 0;
! /* Not matched exactly, pull the var with aggregates then */
aggvars = pull_var_clause((Node *) expr,
PVC_INCLUDE_AGGREGATES);
if (!is_foreign_expr(root, grouped_rel, (Expr *) aggvars))
! return false;
/*
* Add aggregates, if any, into the targetlist. Plain var
--- 5049,5062 ----
if (sgref)
grouping_target->sortgrouprefs[i] = 0;
! /*
! * Not matched exactly, pull the var with aggregates then
! */
aggvars = pull_var_clause((Node *) expr,
PVC_INCLUDE_AGGREGATES);
if (!is_foreign_expr(root, grouped_rel, (Expr *) aggvars))
! return NIL;
/*
* Add aggregates, if any, into the targetlist. Plain var
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4703,4714 ****
/*
* Classify the pushable and non-pushable having clauses and save them in
! * remote_conds and local_conds of the grouped rel's fpinfo.
*/
if (root->hasHavingQual && query->havingQual)
{
- ListCell *lc;
-
foreach(lc, (List *) query->havingQual)
{
Expr *expr = (Expr *) lfirst(lc);
--- 5084,5100 ----
/*
* Classify the pushable and non-pushable having clauses and save them in
! * remote_conds_upper and local_conds_upper of the grouped rel's fpinfo.
! */
!
! /*
! * TODO For non-upper grouped_rel, only pick those expressions for which
! * grouped_rel provides all input vars. (And Assert() that grouped_rel
! * does not emit just some of them --- in such a case grouped_rel
! * shouldn't be subject to partial aggregation at all.)
*/
if (root->hasHavingQual && query->havingQual)
{
foreach(lc, (List *) query->havingQual)
{
Expr *expr = (Expr *) lfirst(lc);
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4728,4736 ****
NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
! fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
else
! fpinfo->local_conds = lappend(fpinfo->local_conds, rinfo);
}
}
--- 5114,5124 ----
NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
! fpinfo->remote_conds_upper =
! lappend(fpinfo->remote_conds_upper, rinfo);
else
! fpinfo->local_conds_upper =
! lappend(fpinfo->local_conds_upper, rinfo);
}
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4738,4749 ****
* If there are any local conditions, pull Vars and aggregates from it and
* check whether they are safe to pushdown or not.
*/
! if (fpinfo->local_conds)
{
List *aggvars = NIL;
- ListCell *lc;
! foreach(lc, fpinfo->local_conds)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
--- 5126,5136 ----
* If there are any local conditions, pull Vars and aggregates from it and
* check whether they are safe to pushdown or not.
*/
! if (fpinfo->local_conds_upper)
{
List *aggvars = NIL;
! foreach(lc, fpinfo->local_conds_upper)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4765,4771 ****
if (IsA(expr, Aggref))
{
if (!is_foreign_expr(root, grouped_rel, expr))
! return false;
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
--- 5152,5158 ----
if (IsA(expr, Aggref))
{
if (!is_foreign_expr(root, grouped_rel, expr))
! return NIL;
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4775,4803 ****
/* Transfer any sortgroupref data to the replacement tlist */
apply_pathtarget_labeling_to_tlist(tlist, grouping_target);
! /* Store generated targetlist */
! fpinfo->grouped_tlist = tlist;
!
! /* Safe to pushdown */
! fpinfo->pushdown_safe = true;
!
! /*
! * Set cached relation costs to some negative value, so that we can detect
! * when they are set to some sensible costs, during one (usually the
! * first) of the calls to estimate_path_cost_size().
! */
! fpinfo->rel_startup_cost = -1;
! fpinfo->rel_total_cost = -1;
!
! /*
! * Set the string describing this grouped relation to be used in EXPLAIN
! * output of corresponding ForeignScan.
! */
! fpinfo->relation_name = makeStringInfo();
! appendStringInfo(fpinfo->relation_name, "Aggregate on (%s)",
! ofpinfo->relation_name->data);
!
! return true;
}
/*
--- 5162,5168 ----
/* Transfer any sortgroupref data to the replacement tlist */
apply_pathtarget_labeling_to_tlist(tlist, grouping_target);
! return tlist;
}
/*
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4846,4857 ****
Query *parse = root->parse;
PgFdwRelationInfo *ifpinfo = input_rel->fdw_private;
PgFdwRelationInfo *fpinfo = grouped_rel->fdw_private;
ForeignPath *grouppath;
PathTarget *grouping_target;
- double rows;
- int width;
- Cost startup_cost;
- Cost total_cost;
/* Nothing to be done, if there is no grouping or aggregation required. */
if (!parse->groupClause && !parse->groupingSets && !parse->hasAggs &&
--- 5211,5219 ----
Query *parse = root->parse;
PgFdwRelationInfo *ifpinfo = input_rel->fdw_private;
PgFdwRelationInfo *fpinfo = grouped_rel->fdw_private;
+ EstimateInfo *estimates;
ForeignPath *grouppath;
PathTarget *grouping_target;
/* Nothing to be done, if there is no grouping or aggregation required. */
if (!parse->groupClause && !parse->groupingSets && !parse->hasAggs &&
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4877,4898 ****
return;
/* Estimate the cost of push down */
! estimate_path_cost_size(root, grouped_rel, NIL, NIL, &rows,
! &width, &startup_cost, &total_cost);
!
! /* Now update this information in the fpinfo */
! fpinfo->rows = rows;
! fpinfo->width = width;
! fpinfo->startup_cost = startup_cost;
! fpinfo->total_cost = total_cost;
/* Create and add foreign path to the grouping relation. */
grouppath = create_foreignscan_path(root,
grouped_rel,
grouping_target,
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
NULL,
--- 5239,5254 ----
return;
/* Estimate the cost of push down */
! estimates = &fpinfo->est_grouped;
! estimate_path_cost_size(root, grouped_rel, NIL, NIL, estimates, true);
/* Create and add foreign path to the grouping relation. */
grouppath = create_foreignscan_path(root,
grouped_rel,
grouping_target,
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
NULL,
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
new file mode 100644
index 788b003..e796236
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 20,25 ****
--- 20,37 ----
#include "libpq-fe.h"
+ typedef struct EstimateInfo
+ {
+ /* Estimated size and cost for a scan or join. */
+ double rows;
+ int width;
+ Cost startup_cost;
+ Cost total_cost;
+ /* Costs excluding costs for transferring data from the foreign server */
+ Cost rel_startup_cost;
+ Cost rel_total_cost;
+ } EstimateInfo;
+
/*
* FDW-specific planner information kept in RelOptInfo.fdw_private for a
* postgres_fdw foreign table. For a baserel, this struct is created by
*************** typedef struct PgFdwRelationInfo
*** 43,48 ****
--- 55,68 ----
List *remote_conds;
List *local_conds;
+ /*
+ * HAVING clauses.
+ *
+ * TODO Use these where appropriate.
+ */
+ List *remote_conds_upper;
+ List *local_conds_upper;
+
/* Actual remote restriction clauses for scan (sans RestrictInfos) */
List *final_remote_exprs;
*************** typedef struct PgFdwRelationInfo
*** 56,69 ****
/* Selectivity of join conditions */
Selectivity joinclause_sel;
! /* Estimated size and cost for a scan or join. */
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
! /* Costs excluding costs for transferring data from the foreign server */
! Cost rel_startup_cost;
! Cost rel_total_cost;
/* Options extracted from catalogs. */
bool use_remote_estimate;
--- 76,84 ----
/* Selectivity of join conditions */
Selectivity joinclause_sel;
! /* Estimates for both plain and grouped relation. */
! EstimateInfo est_plain;
! EstimateInfo est_grouped;
/* Options extracted from catalogs. */
bool use_remote_estimate;
*************** typedef struct PgFdwRelationInfo
*** 92,98 ****
/* joinclauses contains only JOIN/ON conditions for an outer join */
List *joinclauses; /* List of RestrictInfo */
! /* Grouping information */
List *grouped_tlist;
/* Subquery information */
--- 107,115 ----
/* joinclauses contains only JOIN/ON conditions for an outer join */
List *joinclauses; /* List of RestrictInfo */
! /*
! * The targetlist used to construct the remote query.
! */
List *grouped_tlist;
/* Subquery information */
*************** extern unsigned int GetCursorNumber(PGco
*** 121,126 ****
--- 138,144 ----
extern unsigned int GetPrepStmtNumber(PGconn *conn);
extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
+ extern void pgfdw_send_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
*************** extern List *build_tlist_to_deparse(RelO
*** 175,181 ****
extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
List *remote_conds, List *pathkeys, bool is_subquery,
! List **retrieved_attrs, List **params_list);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
--- 193,200 ----
extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
List *remote_conds, List *pathkeys, bool is_subquery,
! List **retrieved_attrs, List **params_list,
! bool grouping);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 9f3a7c0..c8e07bf
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 966,976 ****
static void
set_foreign_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
/* Mark rel with estimated output rows, width, etc */
set_foreign_size_estimates(root, rel);
! /* Let FDW adjust the size estimates, if it can */
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid);
/* ... but do not let it set the rows estimate to zero */
rel->rows = clamp_row_est(rel->rows);
--- 966,991 ----
static void
set_foreign_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
+ Relids required_outer = rel->lateral_relids;
+
/* Mark rel with estimated output rows, width, etc */
set_foreign_size_estimates(root, rel);
! /*
! * Let FDW adjust the size estimates, if it can.
! *
! * For the grouped relation it only makes sense if set_foreign_pathlist is
! * supposed to handle it, see below.
! *
! * Join relation does not need explicit aggregation: the FDW can also form
! * it by aggregating just one of the contained relations and joining the
! * rest (non-aggregated) to it.
! */
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid, false);
! if (rel->gpi != NULL &&
! (rel->gpi->target_agg || (IS_JOIN_REL(rel) && rel->gpi->target)) &&
! required_outer == NULL)
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid, true);
/* ... but do not let it set the rows estimate to zero */
rel->rows = clamp_row_est(rel->rows);
*************** set_foreign_size(PlannerInfo *root, RelO
*** 983,990 ****
static void
set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
/* Call the FDW's GetForeignPaths function to generate path(s) */
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid);
}
/*
--- 998,1015 ----
static void
set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
+ Relids required_outer = rel->lateral_relids;
+
/* Call the FDW's GetForeignPaths function to generate path(s) */
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid, false);
!
! /*
! * Create grouped paths if grouped target exists. Do nothing if outer
! * relations are needed --- that could cause parameterized (and therefore
! * repeated) grouping.
! */
! if (rel->gpi != NULL && rel->gpi->target_agg && required_outer == NULL)
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid, true);
}
/*
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index c8557b6..6c0a060
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 311,319 ****
*/
if (joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
! jointype, &extra);
/*
* 6. Finally, give extensions a chance to manipulate the path list.
--- 311,325 ----
*/
if (joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
+ {
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
! jointype, &extra, false);
! if (joinrel->gpi != NULL && joinrel->gpi->target_agg != NULL)
! joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
! outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 6. Finally, give extensions a chance to manipulate the path list.
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index 718154d..de5c19a
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** set_foreignscan_references(PlannerInfo *
*** 1189,1221 ****
if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0)
{
/*
* Adjust tlist, qual, fdw_exprs, fdw_recheck_quals to reference
! * foreign scan tuple
*/
! indexed_tlist *itlist = build_tlist_index(fscan->fdw_scan_tlist);
!
fscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
! (Node *) fscan->scan.plan.targetlist,
itlist,
INDEX_VAR,
rtoffset);
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
! (Node *) fscan->scan.plan.qual,
itlist,
INDEX_VAR,
rtoffset);
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
! (Node *) fscan->fdw_exprs,
itlist,
INDEX_VAR,
rtoffset);
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
! (Node *) fscan->fdw_recheck_quals,
itlist,
INDEX_VAR,
rtoffset);
--- 1189,1238 ----
if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0)
{
+ List *tlist_tmp;
+ indexed_tlist *itlist = build_tlist_index(fscan->fdw_scan_tlist);
+
/*
* Adjust tlist, qual, fdw_exprs, fdw_recheck_quals to reference
! * foreign scan tuple.
! *
! * Since fdw_scan_tlist contains Aggrefs instead of GroupVars (this is
! * convenient for deparsing), the Aggrefs also need to be restored in
! * the referencing lists.
! *
! * XXX Consider thoroughly where tlist_tmp really needs to be used,
! * i.e. which of the lists can / cannot contain aggregates.
*/
! tlist_tmp = restore_grouping_expressions(root,
! fscan->scan.plan.targetlist);
fscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
! (Node *) tlist_tmp,
itlist,
INDEX_VAR,
rtoffset);
+
+ tlist_tmp = restore_grouping_expressions(root, fscan->scan.plan.qual);
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
! (Node *) tlist_tmp,
itlist,
INDEX_VAR,
rtoffset);
+
+ tlist_tmp = restore_grouping_expressions(root, fscan->fdw_exprs);
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
! (Node *) tlist_tmp,
itlist,
INDEX_VAR,
rtoffset);
+
+ tlist_tmp = restore_grouping_expressions(root,
+ fscan->fdw_recheck_quals);
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
! (Node *) tlist_tmp,
itlist,
INDEX_VAR,
rtoffset);
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
new file mode 100644
index e391f20..05a9923
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
*************** struct ExplainState;
*** 26,36 ****
typedef void (*GetForeignRelSize_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
typedef void (*GetForeignPaths_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
typedef ForeignScan *(*GetForeignPlan_function) (PlannerInfo *root,
RelOptInfo *baserel,
--- 26,38 ----
typedef void (*GetForeignRelSize_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
typedef void (*GetForeignPaths_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
typedef ForeignScan *(*GetForeignPlan_function) (PlannerInfo *root,
RelOptInfo *baserel,
*************** typedef void (*GetForeignJoinPaths_funct
*** 57,63 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
UpperRelationKind stage,
--- 59,66 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped);
typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
UpperRelationKind stage,
agg_pushdown_v3/06_regression.diff
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
new file mode 100644
index e9dc220..86f6fbc
*** a/src/backend/optimizer/path/equivclass.c
--- b/src/backend/optimizer/path/equivclass.c
*************** is_redundant_derived_clause(RestrictInfo
*** 2493,2499 ****
* gvi->gvexpr with vars whose varno is contained in relids.
*/
GroupedVarInfo *
! translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
Relids relids)
{
List *vars;
--- 2493,2499 ----
* gvi->gvexpr with vars whose varno is contained in relids.
*/
GroupedVarInfo *
! translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo * gvi,
Relids relids)
{
List *vars;
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index b52a71b..ee34510
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct GroupedPathInfo
*** 954,960 ****
List *pathlist; /* List of grouped paths. */
List *partial_pathlist; /* List of partial grouped paths. */
List *sortgroupclauses; /* Relation-specific grouping clauses. */
! } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
--- 954,960 ----
List *pathlist; /* List of grouped paths. */
List *partial_pathlist; /* List of partial grouped paths. */
List *sortgroupclauses; /* Relation-specific grouping clauses. */
! } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct GroupedVar
*** 1932,1938 ****
Index sortgroupref; /* SortGroupClause.tleSortGroupRef if gvexpr
* is grouping expression. */
Index gvid; /* GroupedVarInfo */
! } GroupedVar;
/*
* "Special join" info.
--- 1932,1938 ----
Index sortgroupref; /* SortGroupClause.tleSortGroupRef if gvexpr
* is grouping expression. */
Index gvid; /* GroupedVarInfo */
! } GroupedVar;
/*
* "Special join" info.
*************** typedef struct GroupedVarInfo
*** 2165,2171 ****
Relids gv_eval_at; /* lowest level we can evaluate the expression
* at or NULL if it can happen anywhere. */
int32 gv_width; /* estimated width of the expression */
! } GroupedVarInfo;
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
--- 2165,2171 ----
Relids gv_eval_at; /* lowest level we can evaluate the expression
* at or NULL if it can happen anywhere. */
int32 gv_width; /* estimated width of the expression */
! } GroupedVarInfo;
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
new file mode 100644
index 7090d02..040086b
*** a/src/include/optimizer/clauses.h
--- b/src/include/optimizer/clauses.h
*************** extern Node *estimate_expression_value(P
*** 84,90 ****
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
! extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
! GroupedVarInfo *gvi,
! Relids relids);
#endif /* CLAUSES_H */
--- 84,90 ----
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
! extern GroupedVarInfo * translate_expression_to_rels(PlannerInfo *root,
! GroupedVarInfo * gvi,
! Relids relids);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 1e7caf0..9564b62
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern void split_pathtarget_at_srfs(Pla
*** 74,80 ****
extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
! extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
extern void initialize_grouped_targets(PlannerInfo *root,
RelOptInfo *rel,
PathTarget *target_agg,
--- 74,80 ----
extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
! extern GroupedVar * get_grouping_expression(PlannerInfo *root, Expr *expr);
extern void initialize_grouped_targets(PlannerInfo *root,
RelOptInfo *rel,
PathTarget *target_agg,
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index ...2151db1
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
***************
*** 0 ****
--- 1,193 ----
+ BEGIN;
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ CREATE INDEX ON agg_pushdown_child1(parent);
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ ANALYZE;
+ -- encourage use of parallel plan
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET cpu_tuple_cost TO 0.1;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+ -- let parallel workers perform scan of a table and partially aggregate the
+ -- result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ -- let parallel workers scan index on agg_pushdown_child1(parent) column and
+ -- partially aggregate the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ SET enable_seqscan TO on;
+ -- let parallel workers perform nestloop join between agg_pushdown_child1 and
+ -- agg_pushdown_child2 and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (14 rows)
+
+ -- the same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Gather
+ Workers Planned: 2
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Gather
+ Workers Planned: 2
+ -> Parallel Seq Scan on agg_pushdown_child2 c2
+ -> Hash
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on agg_pushdown_parent p
+ (19 rows)
+
+ -- the same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Sort
+ Sort Key: c1.parent, c1.j
+ -> Gather
+ Workers Planned: 2
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Sort
+ Sort Key: c2.parent, c2.k
+ -> Gather
+ Workers Planned: 2
+ -> Parallel Seq Scan on agg_pushdown_child2 c2
+ -> Sort
+ Sort Key: p.i
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Seq Scan on agg_pushdown_parent p
+ (23 rows)
+
+ -- Generic grouping expression.
+ --
+ -- Use this test also to verify that aggregation can be pushed down
+ -- even w/o parallelism.
+ RESET parallel_setup_cost;
+ RESET parallel_tuple_cost;
+ SET cpu_operator_cost TO 0.01;
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Result
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ (17 rows)
+
+ ROLLBACK;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
new file mode 100644
index eefdeea..da9ef37
*** a/src/test/regress/parallel_schedule
--- b/src/test/regress/parallel_schedule
*************** test: rules psql_crosstab amutils
*** 97,102 ****
--- 97,105 ----
# run by itself so it can run parallel workers
test: select_parallel
+ # this one runs parallel workers too
+ test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
new file mode 100644
index 76b0de3..716c389
*** a/src/test/regress/serial_schedule
--- b/src/test/regress/serial_schedule
*************** test: stats_ext
*** 134,139 ****
--- 134,140 ----
test: rules
test: psql_crosstab
test: select_parallel
+ test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index ...cf61436
*** a/src/test/regress/sql/agg_pushdown.sql
--- b/src/test/regress/sql/agg_pushdown.sql
***************
*** 0 ****
--- 1,96 ----
+ BEGIN;
+
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ CREATE INDEX ON agg_pushdown_child1(parent);
+
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ ANALYZE;
+
+ -- encourage use of parallel plan
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET cpu_tuple_cost TO 0.1;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+
+ -- let parallel workers perform scan of a table and partially aggregate the
+ -- result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ -- let parallel workers scan index on agg_pushdown_child1(parent) column and
+ -- partially aggregate the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ SET enable_seqscan TO on;
+
+ -- let parallel workers perform nestloop join between agg_pushdown_child1 and
+ -- agg_pushdown_child2 and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- the same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- the same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- Generic grouping expression.
+ --
+ -- Use this test also to verify that aggregation can be pushed down
+ -- even w/o parallelism.
+ RESET parallel_setup_cost;
+ RESET parallel_tuple_cost;
+ SET cpu_operator_cost TO 0.01;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ ROLLBACK;
agg_pushdown_v3/07_readme.diff
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
new file mode 100644
index fc0fca4..cb027b6
*** a/src/backend/optimizer/README
--- b/src/backend/optimizer/README
*************** be desirable to postpone the Gather stag
*** 1076,1078 ****
--- 1076,1113 ----
plan as possible. Expanding the range of cases in which more work can be
pushed below the Gather (and costing them accurately) is likely to keep us
busy for a long time to come.
+
+ Grouped Paths
+ -------------
+
+ If both query semantics and path type allow, an existing path can be subject
+ to partial aggregation (ie aggregation without calling the aggfinalfn
+ function) and stored as a "grouped path" in a separate pathlist. Both base
+ relation and join paths can be turned into a grouped path this way. If the
+ source path was partial (in terms of parallel processing, see above), then the
+ grouped path is called "partial grouped path".
+
+ If a grouped path appears at the top of the join tree, the transient state
+ values are aggregated and the aggfinalfn function is called for each
+ aggregate. Likewise, if the top-level join produces a partial grouped path, we
+ first apply Gather plan on it and then we finalize the aggregation in the same
+ way.
+
+ Besides being a result of aggregation, (partial) grouped path can be created
+ by joining an existing grouped path to an ordinary (non-grouped) path. This
+ technique can result in duplication of grouping keys, which are essentially
+ unique in the output of aggregation. The final aggregation takes these
+ duplicities into account.
+
+ In contrast, join of two grouped paths is not supported as there doesn't seem
+ to be an interesting use case for it. If we want to support this kind of join
+ in the future, additional attention needs to be paid to the duplication of
+ grouping keys mentioned above. For example, if scan of table A is joined to a
+ grouped scan of table B, then the cardinality of grouping keys of table B
+ present in (non-grouped) table A determines how many times does each
+ aggregation transient state value of B appear on the input of the final
+ aggregation path. However if A is grouped too, the number of those key values
+ on the A side can get lower and thus affect the input of the final aggregation
+ path. When compensating for this effect, we'd have to count input values of
+ each group during the aggregation of A and "multiply" the corresponding
+ transient state values of B accordingly.
[application/x-gzip] shard.tgz (1.5K, ../../29613.1502983342@localhost/3-shard.tgz)
download
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-09-07 09:14 Jeevan Chalke <[email protected]>
parent: Antonin Houska <[email protected]>
3 siblings, 1 reply; 59+ messages in thread
From: Jeevan Chalke @ 2017-09-07 09:14 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
--001a113ee1a09db528055895e493
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Hi Antonin,
To understand the feature you have proposed, I have tried understanding
your patch. Here are few comments so far on it:
1.
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
I did not understand, why you are not pushing aggregate in above cases?
Can you please explain?
2. "make check" in contrib/postgres_fdw crashes.
SELECT COUNT(*) FROM ft1 t1;
! server closed the connection unexpectedly
! This probably means the server terminated abnormally
! before or while processing the request.
! connection to server was lost
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-09-07 11:15 Ashutosh Bapat <[email protected]>
parent: Antonin Houska <[email protected]>
3 siblings, 1 reply; 59+ messages in thread
From: Ashutosh Bapat @ 2017-09-07 11:15 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <[email protected]> wrote:
> Antonin Houska <[email protected]> wrote:
>
>> Antonin Houska <[email protected]> wrote:
>>
>> > This is a new version of the patch I presented in [1].
>>
>> Rebased.
>>
>> cat .git/refs/heads/master
>> b9a3ef55b253d885081c2d0e9dc45802cab71c7b
>
> This is another version of the patch.
>
> Besides other changes, it enables the aggregation push-down for postgres_fdw,
> although only for aggregates whose transient aggregation state is equal to the
> output type. For other aggregates (like avg()) the remote nodes will have to
> return the transient state value in an appropriate form (maybe bytea type),
> which does not depend on PG version.
Having transient aggregation state type same as output type doesn't
mean that we can feed the output of remote aggregation as is to the
finalization stage locally or finalization is not needed at all. I am
not sure why is that test being used to decide whether or not to push
an aggregate (I assume they are partial aggregates) to the foreign
server. postgres_fdw doesn't have a feature to fetch partial aggregate
results from the foreign server. Till we do that, I don't think this
patch can push any partial aggregates down to the foreign server.
>
> shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
> scans of base relation's partitions being pushed to shard nodes, the other
> pushes down a join and performs aggregation of the join result on the remote
> node. Of course, the query can only references one particular partition, until
> the "partition-wise join" [1] patch gets committed and merged with this my
> patch.
Right. Until then joins will not have children.
>
> One thing I'm not sure about is whether the patch should remove
> GetForeignUpperPaths function from FdwRoutine, which it essentially makes
> obsolete. Or should it only be deprecated so far? I understand that
> deprecation is important for "ordinary SQL users", but FdwRoutine is an
> interface for extension developers, so the policy might be different.
I doubt if that's correct. We can do that only when we know that all
kinds aggregates/grouping can be pushed down the join tree, which
looks impossible to me. Apart from that that FdwRoutine handles all
kinds of upper relations like windowing, distinct, ordered which are
not pushed down by this set of patches.
Here's review of the patchset
The patches compile cleanly when applied together.
Testcase "limit" fails in make check.
This is a pretty large patchset and the patches are divided by stages in the
planner rather than functionality. IOW, no single patch provides a full
functionality. Reviewing and probably committing such patches is not easy and
may take quite long. Instead, if the patches are divided by functionality, i.e.
each patch provides some functionality completely, it will be easy to review
and commit such patches with each commit giving something useful. I had
suggested breaking patches on that line at [1]. The patches also refactor
existing code. It's better to move such refactoring in a patch/es by itself.
The patches need more comments, explaining various decisions e.g.
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
- generate_gather_paths(root, joinrel);
+ generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
For base relations, the patch calls generate_gather_paths() with
grouped as true and
false, but for join relations it doesn't do that. There is no comment
explaining why.
OR
in the following code
+ /*
+ * Do partial aggregation at base relation level if the relation is
+ * eligible for it.
+ */
+ if (rel->gpi != NULL)
+ create_grouped_path(root, rel, path, false, true, AGG_HASHED);
Why not to consider AGG_SORTED paths?
The patch maintains an extra rel target, two pathlists, partial pathlist and
non-partial pathlist for grouping in RelOptInfo. These two extra
pathlists require "grouped" argument to be passed to a number of functions.
Instead probably it makes sense to create a RelOptInfo for aggregation/grouping
and set pathlist and partial_pathlist in that RelOptInfo. That will also make a
place for keeping grouped estimates.
For placeholders we have two function add_placeholders_to_base_rels() and
add_placeholders_to_joinrel(). We have add_grouping_info_to_base_rels(), but do
not have corresponding function for joinrels. How do we push aggregates
involving two or more relations to the corresponding joinrels?
Some minor assorted comments
Avoid useless hunk.
@@ -279,8 +301,8 @@ add_paths_to_joinrel(PlannerInfo *root,
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
- hash_inner_and_outer(root, joinrel, outerrel, innerrel,
- jointype, &extra);
+ hash_inner_and_outer(root, joinrel, outerrel, innerrel, jointype,
+ &extra);
In the last "regression" patch, there are some code changes (mostly because of
pg_indent run). May be you want to include those in appropriate code patches.
Some quick observations using two tables t1(a int, b int) and t2(a int, b int),
populated as
insert into t1 select i, i % 5 from generate_series(1, 100) i where i % 2 = 0;
insert into t2 select i, i % 5 from generate_series(1, 100) i where i % 3 = 0;
1. The patch pushes aggregates down join in the following query
explain verbose select avg(t2.a) from t1 inner join t2 on t1.b = t2.b group by
t2.b;
but does not push the aggregates down join in the following query
explain verbose select avg(t2.a) from t1 left join t2 on t1.b = t2.b group by
t2.b;
In fact, it doesn't use the optimization for any OUTER join. I think the reason
for this behaviour is that the patch uses equivalence classes to distribute the
aggregates and grouping to base relations and OUTER equi-joins do not form
equivalence classes. But I think it should be possible to push the aggregates
down the OUTER join by adding one row for NULL values if the grouping is pushed
to the inner side. I don't see much change for outer side. This also means that
we have to choose some means other than equivalence class for propagating the
aggregates.
2. Following query throws error with these patches, but not without the
patches.
explain verbose select sum(t1.a + t2.a) from t1, t2, t2 t3 where t1.a
= t2.a
group by t2.a, t1.a;
ERROR: ORDER/GROUP BY expression not found in targetlist
[1] https://www.postgresql.org/message-id/CAFjFpRejPP4K%3Dg%2B0aaq_US0YrMaZzyM%2BNUCi%3DJgwaxhMUj2Zcg%40...
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-09-07 14:21 Merlin Moncure <[email protected]>
parent: Antonin Houska <[email protected]>
3 siblings, 0 replies; 59+ messages in thread
From: Merlin Moncure @ 2017-09-07 14:21 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Thu, Aug 17, 2017 at 10:22 AM, Antonin Houska <[email protected]> wrote:
> Antonin Houska <[email protected]> wrote:
> output type. For other aggregates (like avg()) the remote nodes will have to
> return the transient state value in an appropriate form (maybe bytea type),
> which does not depend on PG version.
Hm, that seems like an awful lot of work (new version agnostic
serialization format) for very little benefit (version independent
type serialization for remote aggregate pushdown). How about forcing
the version to be the same for the feature to be used?
merlin
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-09-08 12:36 Antonin Houska <[email protected]>
parent: Jeevan Chalke <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: Antonin Houska @ 2017-09-08 12:36 UTC (permalink / raw)
To: pgsql-hackers
Jeevan Chalke <[email protected]> wrote:
> 1.
> + if (aggref->aggvariadic ||
> + aggref->aggdirectargs || aggref->aggorder ||
> + aggref->aggdistinct || aggref->aggfilter)
>
> I did not understand, why you are not pushing aggregate in above cases?
> Can you please explain?
Currently I'm trying to implement infrastructure to propagate result of
partial aggregation from base relation or join to the root of the join tree
and to use these as input for the final aggregation. Some special cases are
disabled because I haven't thought about them enough so far. Some of these
restrictions may be easy to relax, others may be hard. I'll get back to them
at some point.
> 2. "make check" in contrib/postgres_fdw crashes.
>
> SELECT COUNT(*) FROM ft1 t1;
> ! server closed the connection unexpectedly
> ! This probably means the server terminated abnormally
> ! before or while processing the request.
> ! connection to server was lost
>
> From your given setup, if I wrote a query like:
> EXPLAIN VERBOSE SELECT count(*) FROM orders_0;
> it crashes.
>
> Seems like missing few checks.
Please consider this a temporary limitation.
> 3. In case of pushing partial aggregation on the remote side, you use schema
> named "partial", I did not get that change. If I have AVG() aggregate,
> then I end up getting an error saying
> "ERROR: schema "partial" does not exist".
Attached to his email is an extension that I hacked quickly at some point, for
experimental purposes only. It's pretty raw but may be useful if you just want
to play with it.
> 4. I am not sure about the code changes related to pushing partial
> aggregate on the remote side. Basically, you are pushing a whole aggregate
> on the remote side and result of that is treated as partial on the
> basis of aggtype = transtype. But I am not sure that is correct unless
> I miss something here. The type may be same but the result may not.
You're right. I mostly used sum() and count() in my examples but these are
special cases. It should also be checked whether aggfinalfn exists or
not. I'll fix it, thanks.
> 5. There are lot many TODO comments in the patch-set, are you working
> on those?
Yes. I wasn't too eager to complete all the TODOs soon because reviews of the
partch may result in a major rework. And if large parts of the code will have
to be wasted, some / many TODOs will be gone as well.
> Thanks
>
> On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <[email protected]> wrote:
>
> Antonin Houska <[email protected]> wrote:
>
> > Antonin Houska <[email protected]> wrote:
> >
> > > This is a new version of the patch I presented in [1].
> >
> > Rebased.
> >
> > cat .git/refs/heads/master
> > b9a3ef55b253d885081c2d0e9dc45802cab71c7b
>
> This is another version of the patch.
>
> Besides other changes, it enables the aggregation push-down for postgres_fdw,
> although only for aggregates whose transient aggregation state is equal to the
> output type. For other aggregates (like avg()) the remote nodes will have to
> return the transient state value in an appropriate form (maybe bytea type),
> which does not depend on PG version.
>
> shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
> scans of base relation's partitions being pushed to shard nodes, the other
> pushes down a join and performs aggregation of the join result on the remote
> node. Of course, the query can only references one particular partition, until
> the "partition-wise join" [1] patch gets committed and merged with this my
> patch.
>
> One thing I'm not sure about is whether the patch should remove
> GetForeignUpperPaths function from FdwRoutine, which it essentially makes
> obsolete. Or should it only be deprecated so far? I understand that
> deprecation is important for "ordinary SQL users", but FdwRoutine is an
> interface for extension developers, so the policy might be different.
>
> [1] https://commitfest.postgresql.org/14/994/
>
> Any feedback is appreciated.
>
> --
> Antonin Houska
> Cybertec Schönig & Schönig GmbH
> Gröhrmühlgasse 26
> A-2700 Wiener Neustadt
> Web: http://www.postgresql-support.de, http://www.cybertec.at
>
> --
> Sent via pgsql-hackers mailing list ([email protected])
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
> --
> Jeevan Chalke
> Principal Software Engineer, Product Development
> EnterpriseDB Corporation
> The Enterprise PostgreSQL Company
>
>
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[application/x-gzip] partial.tgz (3.2K, ../../22851.1504874163@localhost/2-partial.tgz)
download
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-09-08 13:34 Antonin Houska <[email protected]>
parent: Ashutosh Bapat <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2017-09-08 13:34 UTC (permalink / raw)
To: pgsql-hackers
Ashutosh Bapat <[email protected]> wrote:
> On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <[email protected]> wrote:
> > Antonin Houska <[email protected]> wrote:
> >
> >> Antonin Houska <[email protected]> wrote:
> >>
> >> > This is a new version of the patch I presented in [1].
> >>
> >> Rebased.
> >>
> >> cat .git/refs/heads/master
> >> b9a3ef55b253d885081c2d0e9dc45802cab71c7b
> >
> > This is another version of the patch.
> >
> > Besides other changes, it enables the aggregation push-down for postgres_fdw,
> > although only for aggregates whose transient aggregation state is equal to the
> > output type. For other aggregates (like avg()) the remote nodes will have to
> > return the transient state value in an appropriate form (maybe bytea type),
> > which does not depend on PG version.
>
> Having transient aggregation state type same as output type doesn't
> mean that we can feed the output of remote aggregation as is to the
> finalization stage locally or finalization is not needed at all. I am
> not sure why is that test being used to decide whether or not to push
> an aggregate (I assume they are partial aggregates) to the foreign
> server.
I agree. This seems to be the same problem as reported in [2].
> postgres_fdw doesn't have a feature to fetch partial aggregate
> results from the foreign server. Till we do that, I don't think this
> patch can push any partial aggregates down to the foreign server.
Even if the query contains only aggregates like sum(int4), i.e. aggfinalfn
does not exist and the transient type can be transfered from the remote server
in textual form? (Of course there's a question is if such behaviour is
consistent from user's perspective.)
> >
> > One thing I'm not sure about is whether the patch should remove
> > GetForeignUpperPaths function from FdwRoutine, which it essentially makes
> > obsolete. Or should it only be deprecated so far? I understand that
> > deprecation is important for "ordinary SQL users", but FdwRoutine is an
> > interface for extension developers, so the policy might be different.
>
> I doubt if that's correct. We can do that only when we know that all
> kinds aggregates/grouping can be pushed down the join tree, which
> looks impossible to me. Apart from that that FdwRoutine handles all
> kinds of upper relations like windowing, distinct, ordered which are
> not pushed down by this set of patches.
Good point.
> Here's review of the patchset
> The patches compile cleanly when applied together.
>
> Testcase "limit" fails in make check.
If I remember correctly, this test generates different plan because the
aggregate push-down gets involved. I tend to ignore this error until the patch
has cost estimates refined. Then let's see if the test will pass or if the
expected output should be adjusted.
> This is a pretty large patchset and the patches are divided by stages in the
> planner rather than functionality. IOW, no single patch provides a full
> functionality. Reviewing and probably committing such patches is not easy and
> may take quite long. Instead, if the patches are divided by functionality, i.e.
> each patch provides some functionality completely, it will be easy to review
> and commit such patches with each commit giving something useful. I had
> suggested breaking patches on that line at [1]. The patches also refactor
> existing code. It's better to move such refactoring in a patch/es by itself.
I definitely saw commits whose purpose is preparation for something else. But
you're right that some splitting of the actual funcionality wouldn't
harm. I'll at least separate partial aggregation of base relations and that of
joins. And maybe some more preparation steps.
> The patches need more comments, explaining various decisions
o.k.
> The patch maintains an extra rel target, two pathlists, partial pathlist and
> non-partial pathlist for grouping in RelOptInfo. These two extra
> pathlists require "grouped" argument to be passed to a number of functions.
> Instead probably it makes sense to create a RelOptInfo for aggregation/grouping
> and set pathlist and partial_pathlist in that RelOptInfo. That will also make a
> place for keeping grouped estimates.
If grouped paths require a separate RelOptInfo, why the existing partial paths
do not?
> For placeholders we have two function add_placeholders_to_base_rels() and
> add_placeholders_to_joinrel(). We have add_grouping_info_to_base_rels(), but do
> not have corresponding function for joinrels. How do we push aggregates
> involving two or more relations to the corresponding joinrels?
add_grouping_info_to_base_rels() calls prepare_rel_for_grouping() which
actually adds the "grouping info". For join, prepare_rel_for_grouping() is
called from build_join_rel().
While PlaceHolderVars need to be finalized before planner starts to create
joins, the GroupedPathInfo has to fit particular pair of joined relations.
Perhaps create_grouping_info_... would be better, to indicate that the thing
we're adding does not exist yet. I'll think about it.
> Some minor assorted comments ...
o.k., will fix.
> Some quick observations using two tables t1(a int, b int) and t2(a int, b int),
> populated as
> insert into t1 select i, i % 5 from generate_series(1, 100) i where i % 2 = 0;
> insert into t2 select i, i % 5 from generate_series(1, 100) i where i % 3 = 0;
>
> 1. The patch pushes aggregates down join in the following query
> explain verbose select avg(t2.a) from t1 inner join t2 on t1.b = t2.b group by
> t2.b;
> but does not push the aggregates down join in the following query
> explain verbose select avg(t2.a) from t1 left join t2 on t1.b = t2.b group by
> t2.b;
> In fact, it doesn't use the optimization for any OUTER join. I think the reason
> for this behaviour is that the patch uses equivalence classes to distribute the
> aggregates and grouping to base relations and OUTER equi-joins do not form
> equivalence classes. But I think it should be possible to push the aggregates
> down the OUTER join by adding one row for NULL values if the grouping is pushed
> to the inner side. I don't see much change for outer side. This also means that
> we have to choose some means other than equivalence class for propagating the
> aggregates.
The problem is that aggregate pushed down to the nullable side of an outer
join receives different input values than the original aggregate at the top
level of the query. NULL values generated by the OJ make the difference. I
have no idea how to handle this problem. If the aggregation is performed on
the nullable side of the OJ, it can't predict which of the input values don't
match any value of the other side. Suggestions are appreciated.
> 2. Following query throws error with these patches, but not without the
> patches.
> explain verbose select sum(t1.a + t2.a) from t1, t2, t2 t3 where t1.a
> = t2.a
> group by t2.a, t1.a;
> ERROR: ORDER/GROUP BY expression not found in targetlist
I'll check this. Thanks.
> [1] https://www.postgresql.org/message-id/CAFjFpRejPP4K%3Dg%2B0aaq_US0YrMaZzyM%2BNUCi%3DJgwaxhMUj2Zcg%40...
[2] https://www.postgresql.org/message-id/CAM2%2B6%3DW2J-iaQBgj-sdMERELQLUm5dvOQEWQ2ho%2BQ7KZgnonkQ%40ma...
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-09-11 08:30 Ashutosh Bapat <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: Ashutosh Bapat @ 2017-09-11 08:30 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Fri, Sep 8, 2017 at 7:04 PM, Antonin Houska <[email protected]> wrote:
> Ashutosh Bapat <[email protected]> wrote:
>
>> On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <[email protected]> wrote:
>> > Antonin Houska <[email protected]> wrote:
>> >
>> >> Antonin Houska <[email protected]> wrote:
>> >>
>> >> > This is a new version of the patch I presented in [1].
>> >>
>> >> Rebased.
>> >>
>> >> cat .git/refs/heads/master
>> >> b9a3ef55b253d885081c2d0e9dc45802cab71c7b
>> >
>> > This is another version of the patch.
>> >
>> > Besides other changes, it enables the aggregation push-down for postgres_fdw,
>> > although only for aggregates whose transient aggregation state is equal to the
>> > output type. For other aggregates (like avg()) the remote nodes will have to
>> > return the transient state value in an appropriate form (maybe bytea type),
>> > which does not depend on PG version.
>>
>> Having transient aggregation state type same as output type doesn't
>> mean that we can feed the output of remote aggregation as is to the
>> finalization stage locally or finalization is not needed at all. I am
>> not sure why is that test being used to decide whether or not to push
>> an aggregate (I assume they are partial aggregates) to the foreign
>> server.
>
> I agree. This seems to be the same problem as reported in [2].
>
>> postgres_fdw doesn't have a feature to fetch partial aggregate
>> results from the foreign server. Till we do that, I don't think this
>> patch can push any partial aggregates down to the foreign server.
>
> Even if the query contains only aggregates like sum(int4), i.e. aggfinalfn
> does not exist and the transient type can be transfered from the remote server
> in textual form? (Of course there's a question is if such behaviour is
> consistent from user's perspective.)
Yes, those functions will work.
>
>> >
>> > One thing I'm not sure about is whether the patch should remove
>> > GetForeignUpperPaths function from FdwRoutine, which it essentially makes
>> > obsolete. Or should it only be deprecated so far? I understand that
>> > deprecation is important for "ordinary SQL users", but FdwRoutine is an
>> > interface for extension developers, so the policy might be different.
>>
>> I doubt if that's correct. We can do that only when we know that all
>> kinds aggregates/grouping can be pushed down the join tree, which
>> looks impossible to me. Apart from that that FdwRoutine handles all
>> kinds of upper relations like windowing, distinct, ordered which are
>> not pushed down by this set of patches.
>
> Good point.
>
I think this is where Jeevan Chalke's partition-wise aggregation
helps. It deals with the cases where your patch can not push the
aggregates down to children of join.
>> The patch maintains an extra rel target, two pathlists, partial pathlist and
>> non-partial pathlist for grouping in RelOptInfo. These two extra
>> pathlists require "grouped" argument to be passed to a number of functions.
>> Instead probably it makes sense to create a RelOptInfo for aggregation/grouping
>> and set pathlist and partial_pathlist in that RelOptInfo. That will also make a
>> place for keeping grouped estimates.
>
> If grouped paths require a separate RelOptInfo, why the existing partial paths
> do not?
partial paths produce the same targetlist and the same relation that
non-partial paths do. A RelOptInfo in planner represents a set of rows
and all paths added to that RelOptInfo produce same whole result
(parameterized paths produces the same result collectively). Grouping
paths however are producing a different result and different
targetlist, so may be it's better to have a separate RelOptInfo.
>
>> For placeholders we have two function add_placeholders_to_base_rels() and
>> add_placeholders_to_joinrel(). We have add_grouping_info_to_base_rels(), but do
>> not have corresponding function for joinrels. How do we push aggregates
>> involving two or more relations to the corresponding joinrels?
>
> add_grouping_info_to_base_rels() calls prepare_rel_for_grouping() which
> actually adds the "grouping info". For join, prepare_rel_for_grouping() is
> called from build_join_rel().
Ok.
>
> While PlaceHolderVars need to be finalized before planner starts to create
> joins, the GroupedPathInfo has to fit particular pair of joined relations.
>
> Perhaps create_grouping_info_... would be better, to indicate that the thing
> we're adding does not exist yet. I'll think about it.
>
>> Some minor assorted comments ...
>
> o.k., will fix.
>
>> Some quick observations using two tables t1(a int, b int) and t2(a int, b int),
>> populated as
>> insert into t1 select i, i % 5 from generate_series(1, 100) i where i % 2 = 0;
>> insert into t2 select i, i % 5 from generate_series(1, 100) i where i % 3 = 0;
>>
>> 1. The patch pushes aggregates down join in the following query
>> explain verbose select avg(t2.a) from t1 inner join t2 on t1.b = t2.b group by
>> t2.b;
>> but does not push the aggregates down join in the following query
>> explain verbose select avg(t2.a) from t1 left join t2 on t1.b = t2.b group by
>> t2.b;
>
>> In fact, it doesn't use the optimization for any OUTER join. I think the reason
>> for this behaviour is that the patch uses equivalence classes to distribute the
>> aggregates and grouping to base relations and OUTER equi-joins do not form
>> equivalence classes. But I think it should be possible to push the aggregates
>> down the OUTER join by adding one row for NULL values if the grouping is pushed
>> to the inner side. I don't see much change for outer side. This also means that
>> we have to choose some means other than equivalence class for propagating the
>> aggregates.
>
> The problem is that aggregate pushed down to the nullable side of an outer
> join receives different input values than the original aggregate at the top
> level of the query. NULL values generated by the OJ make the difference. I
> have no idea how to handle this problem. If the aggregation is performed on
> the nullable side of the OJ, it can't predict which of the input values don't
> match any value of the other side. Suggestions are appreciated.
I haven't thought through this fully as well, but I think this can be
fixed by using some kind of all NULL row on the nullable side. If we
are going to use equivalence classes, we won't be able to extend that
mechanism to OUTER joins, so may be you want to rethink about using
equivalence classes as a method of propagating grouping information.
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2017-11-03 15:33 Antonin Houska <[email protected]>
parent: Antonin Houska <[email protected]>
3 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2017-11-03 15:33 UTC (permalink / raw)
To: pgsql-hackers
Antonin Houska <[email protected]> wrote:
> This is another version of the patch.
> shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
> scans of base relation's partitions being pushed to shard nodes, the other
> pushes down a join and performs aggregation of the join result on the remote
> node. Of course, the query can only references one particular partition, until
> the "partition-wise join" [1] patch gets committed and merged with this my
> patch.
Since [1] is already there, the new version of shard.tgz shows what I consider
the typical use case. (I'm aware of the postgres_fdw regression test failures,
I'll try to fix them all in the next version.)
Besides that:
* A concept of "path unique keys" has been introduced. It helps to find out if
the final relation appears to generate a distinct set of grouping keys. If
that happens, the final aggregation is replaced by mere call of aggfinalfn()
function on each transient state value.
* FDW can sort rows by aggregate.
* enable_agg_pushdown GUC was added. The default value is false.
* I fixed errors reported during the previous CF.
* Added a few more regression tests.
I'm not about to add any other features now. Implementation of the missing
parts (see the TODO comments in the code) is the next step. But what I'd
appreciate most is a feedback on the design. Thanks.
> [1] https://commitfest.postgresql.org/14/994/
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[application/x-gzip] agg_pushdown_v4.tgz (102.3K, ../../14577.1509723225@localhost/2-agg_pushdown_v4.tgz)
download | 9 patch file(s) in archive:
agg_pushdown_v4/01_new_types.diff
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index c1a83ca..dfab262
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyAggref(const Aggref *from)
*** 1357,1362 ****
--- 1357,1363 ----
COPY_SCALAR_FIELD(aggcollid);
COPY_SCALAR_FIELD(inputcollid);
COPY_SCALAR_FIELD(aggtranstype);
+ COPY_SCALAR_FIELD(aggcombinefn);
COPY_NODE_FIELD(aggargtypes);
COPY_NODE_FIELD(aggdirectargs);
COPY_NODE_FIELD(args);
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2209,2214 ****
--- 2210,2231 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** _copyPlaceHolderInfo(const PlaceHolderIn
*** 2281,2286 ****
--- 2298,2318 ----
return newnode;
}
+ static GroupedVarInfo *
+ _copyGroupedVarInfo(const GroupedVarInfo *from)
+ {
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_SCALAR_FIELD(gvid);
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(gv_width);
+
+ return newnode;
+ }
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
*************** copyObjectImpl(const void *from)
*** 5002,5007 ****
--- 5034,5042 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
*************** copyObjectImpl(const void *from)
*** 5014,5019 ****
--- 5049,5057 ----
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index 7a70001..516a0c4
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 873,878 ****
--- 873,886 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3159,3164 ****
--- 3167,3175 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index 8e6f27e..e98f983
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 259,264 ****
--- 259,270 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ type = exprType((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprTypmod(const Node *expr)
*** 492,497 ****
--- 498,508 ----
return ((const SetToDefault *) expr)->typeMod;
case T_PlaceHolderVar:
return exprTypmod((Node *) ((const PlaceHolderVar *) expr)->phexpr);
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->gvexpr);
default:
break;
}
*************** exprCollation(const Node *expr)
*** 903,908 ****
--- 914,925 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2176,2181 ****
--- 2193,2200 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2968,2973 ****
--- 2987,3002 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 43d6206..ec83734
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outAggref(StringInfo str, const Aggref
*** 1131,1136 ****
--- 1131,1137 ----
WRITE_OID_FIELD(aggcollid);
WRITE_OID_FIELD(inputcollid);
WRITE_OID_FIELD(aggtranstype);
+ WRITE_OID_FIELD(aggcombinefn);
WRITE_NODE_FIELD(aggargtypes);
WRITE_NODE_FIELD(aggdirectargs);
WRITE_NODE_FIELD(args);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2220,2225 ****
--- 2221,2227 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2227,2232 ****
--- 2229,2235 ----
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
*************** _outRelOptInfo(StringInfo str, const Rel
*** 2266,2271 ****
--- 2269,2275 ----
WRITE_NODE_FIELD(cheapest_parameterized_paths);
WRITE_BITMAPSET_FIELD(direct_lateral_relids);
WRITE_BITMAPSET_FIELD(lateral_relids);
+ WRITE_NODE_FIELD(gpi);
WRITE_UINT_FIELD(relid);
WRITE_OID_FIELD(reltablespace);
WRITE_ENUM_FIELD(rtekind, RTEKind);
*************** _outParamPathInfo(StringInfo str, const
*** 2442,2447 ****
--- 2446,2463 ----
}
static void
+ _outGroupedPathInfo(StringInfo str, const GroupedPathInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDPATHINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(pathlist);
+ WRITE_NODE_FIELD(partial_pathlist);
+ WRITE_NODE_FIELD(sortgroupclauses);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2485,2490 ****
--- 2501,2517 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** _outPlaceHolderInfo(StringInfo str, cons
*** 2538,2543 ****
--- 2565,2583 ----
}
static void
+ _outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_UINT_FIELD(gvid);
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_INT_FIELD(gv_width);
+ }
+
+ static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
*************** outNode(StringInfo str, const void *obj)
*** 4038,4049 ****
--- 4078,4095 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_GroupedPathInfo:
+ _outGroupedPathInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
*************** outNode(StringInfo str, const void *obj)
*** 4056,4061 ****
--- 4102,4110 ----
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index ccb6a1f..fc739b7
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 529,534 ****
--- 529,550 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(sortgroupref);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** _readAggref(void)
*** 584,589 ****
--- 600,606 ----
READ_OID_FIELD(aggcollid);
READ_OID_FIELD(inputcollid);
READ_OID_FIELD(aggtranstype);
+ READ_OID_FIELD(aggcombinefn);
READ_NODE_FIELD(aggargtypes);
READ_NODE_FIELD(aggdirectargs);
READ_NODE_FIELD(args);
*************** parseNodeString(void)
*** 2466,2471 ****
--- 2483,2490 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index f4e0a6e..2e6a60e
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 114,120 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index d58635c..25e0782
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** subquery_planner(PlannerGlobal *glob, Qu
*** 535,540 ****
--- 535,541 ----
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
new file mode 100644
index f3bb73a..e22551a
*** a/src/backend/optimizer/prep/prepjointree.c
--- b/src/backend/optimizer/prep/prepjointree.c
*************** pull_up_simple_subquery(PlannerInfo *roo
*** 911,916 ****
--- 911,917 ----
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ subroot->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
new file mode 100644
index fc0d6bc..071681a
*** a/src/backend/parser/parse_func.c
--- b/src/backend/parser/parse_func.c
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 98,103 ****
--- 98,104 ----
Oid vatype;
FuncDetailCode fdresult;
char aggkind = 0;
+ Oid aggcombinefn = InvalidOid;
ParseCallbackState pcbstate;
/*
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 321,326 ****
--- 322,328 ----
elog(ERROR, "cache lookup failed for aggregate %u", funcid);
classForm = (Form_pg_aggregate) GETSTRUCT(tup);
aggkind = classForm->aggkind;
+ aggcombinefn = classForm->aggcombinefn;
catDirectArgs = classForm->aggnumdirectargs;
ReleaseSysCache(tup);
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 666,671 ****
--- 668,674 ----
aggref->aggstar = agg_star;
aggref->aggvariadic = func_variadic;
aggref->aggkind = aggkind;
+ aggref->aggcombinefn = aggcombinefn;
/* agglevelsup will be set by transformAggregateCall */
aggref->aggsplit = AGGSPLIT_SIMPLE; /* planner might change this */
aggref->location = location;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index ffeeb49..b4afcb0
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 218,223 ****
--- 218,224 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_GroupedPathInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 258,267 ****
--- 259,270 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index c2929ac..5251f98
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct Aggref
*** 296,301 ****
--- 296,302 ----
Oid aggcollid; /* OID of collation of result */
Oid inputcollid; /* OID of collation that function should use */
Oid aggtranstype; /* type Oid of aggregate's transition value */
+ Oid aggcombinefn; /* combine function (see pg_aggregate.h) */
List *aggargtypes; /* type Oids of direct and aggregated args */
List *aggdirectargs; /* direct arguments, if an ordered-set agg */
List *args; /* aggregated arguments and sort expressions */
*************** typedef struct Aggref
*** 306,311 ****
--- 307,313 ----
bool aggvariadic; /* true if variadic arguments have been
* combined into an array last argument */
char aggkind; /* aggregate kind (see pg_aggregate.h) */
+
Index agglevelsup; /* > 0 if agg belongs to outer query */
AggSplit aggsplit; /* expected agg-splitting mode of parent Agg */
int location; /* token location, or -1 if unknown */
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index 663fb2f..a18ee93
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 257,262 ****
--- 257,264 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 283,288 ****
--- 285,296 ----
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
*************** typedef struct PartitionSchemeData *Part
*** 438,443 ****
--- 446,453 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 609,614 ****
--- 619,627 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to produce grouped paths. */
+ struct GroupedPathInfo *gpi;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 1003,1008 ****
--- 1016,1062 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * GroupedPathInfo
+ *
+ * If RelOptInfo points to this structure, grouped paths can be created for
+ * it.
+ *
+ * "target" will be used as pathtarget of grouped paths produced either by
+ * "explicit aggregation" of the relation that owns this structure, or --- if
+ * the relation is a join --- by joining grouped path to a non-grouped
+ * one.
+ *
+ * The target contains plain-Var grouping expressions, generic grouping
+ * expressions wrapped in GroupedVar structure, or Aggrefs which are also
+ * wrapped in GroupedVar. Once GroupedVar is evaluated, its value is passed to
+ * the upper paths w/o being evaluated again. If final aggregation appears to
+ * be necessary above the final join, the contained Aggrefs are supposed to
+ * provide the final aggregation plan with input values, i.e. the aggregate
+ * transient state.
+ *
+ * Note: There's a convention that GroupedVars that contain Aggref expressions
+ * are supposed to follow the other expressions of the target. Iterations of
+ * target->exprs may rely on this arrangement.
+ *
+ * "group_exprs" contains grouping expressions which are not plain vars. These
+ * are only added to path target of a path which should generate input for
+ * partial aggregation.
+ *
+ * (Two grouped paths cannot be joined in general because grouping of one side
+ * of the join essentially reduces occurrence of groups of the other side in
+ * the input of the final aggregation.)
+ */
+ typedef struct GroupedPathInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* target of grouped paths aggregation. */
+ PathTarget *group_exprs; /* non-Var grouping expressions. */
+ List *pathlist; /* List of grouped paths. */
+ List *partial_pathlist; /* List of partial grouped paths. */
+ List *sortgroupclauses; /* Relation-specific grouping clauses. */
+ } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct PlaceHolderVar
*** 1943,1948 ****
--- 1997,2037 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. The reason is that aggregates having start as the argument can be
+ * evaluated at various places in the join tree (i.e. cannot be assigned to
+ * target list of exactly one relation). Also this concept seems to be less
+ * invasive than adding the grouped vars to reltarget (in which case
+ * attr_needed and attr_widths arrays of RelOptInfo) would also need
+ * additional changes.
+ *
+ * gvexpr is a pointer to gvexpr field of the corresponding instance
+ * GroupedVarInfo. It's there for the sake of exprType(), exprCollation(),
+ * etc.
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if the
+ * GroupedVar is in the target of a parent relation (RELOPT_BASEREL). However
+ * within a child relation's (RELOPT_OTHER_MEMBER_REL) target it points to a
+ * copy which has argument expressions translated, so they no longer reference
+ * the parent.
+ *
+ * XXX Currently we only create GroupedVar for aggregates, but sometime we can
+ * do it for grouping keys as well. That would allow grouping below the
+ * top-level join by keys other than plain Var.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index sortgroupref; /* SortGroupClause.tleSortGroupRef if gvexpr
+ * is grouping expression. */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PlaceHolderInfo
*** 2158,2163 ****
--- 2247,2271 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
agg_pushdown_v4/02_preprocess.diff
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
new file mode 100644
index a225414..072b16f
*** a/src/backend/optimizer/path/equivclass.c
--- b/src/backend/optimizer/path/equivclass.c
*************** static bool reconsider_outer_join_clause
*** 65,70 ****
--- 65,83 ----
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
+ typedef struct translate_expr_context
+ {
+ Var **keys; /* Dictionary keys. */
+ Var **values; /* Dictionary values */
+ int nitems; /* Number of dictionary items. */
+ Relids *gv_eval_at_p; /* See GroupedVarInfo. */
+ Relids relids; /* Translate into these relids. */
+ } translate_expr_context;
+
+ static Node *translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context * context);
+ static int var_dictionary_comparator(const void *a, const void *b);
+
/*
* process_equivalence
*************** is_redundant_derived_clause(RestrictInfo
*** 2510,2512 ****
--- 2523,2844 ----
return false;
}
+
+ /*
+ * translate_expression_to_rels
+ *
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is contained in relids.
+ */
+ GroupedVarInfo *
+ translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Relids relids)
+ {
+ List *vars;
+ ListCell *l1;
+ int i,
+ j;
+ int nkeys,
+ nkeys_resolved;
+ Var **keys,
+ **values,
+ **keys_tmp;
+ Var *key,
+ *key_prev;
+ translate_expr_context context;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ /*
+ * Before actually trying to modify the expression tree, find out if all
+ * vars can be translated.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, 0);
+
+ /* No vars to translate? */
+ if (vars == NIL)
+ return NULL;
+
+ /*
+ * Search for individual replacement vars as well as the actual expression
+ * translation will be more efficient if we use a dictionary with the keys
+ * (i.e. the "source vars") unique and sorted.
+ */
+ nkeys = list_length(vars);
+ keys = (Var **) palloc(nkeys * sizeof(Var *));
+ i = 0;
+ foreach(l1, vars)
+ {
+ key = lfirst_node(Var, l1);
+ keys[i++] = key;
+ }
+
+ /*
+ * Sort the keys by varno. varattno decides where varnos are equal.
+ */
+ pg_qsort(keys, nkeys, sizeof(Var *), var_dictionary_comparator);
+
+ /*
+ * Pick unique values and get rid of the vars that need no translation.
+ */
+ keys_tmp = (Var **) palloc(nkeys * sizeof(Var *));
+ key_prev = NULL;
+ j = 0;
+ for (i = 0; i < nkeys; i++)
+ {
+ key = keys[i];
+
+ if ((key_prev == NULL || (key->varno != key_prev->varno &&
+ key->varattno != key_prev->varattno)) &&
+ !bms_is_member(key->varno, relids))
+ keys_tmp[j++] = key;
+
+ key_prev = key;
+ }
+ pfree(keys);
+ keys = keys_tmp;
+ nkeys = j;
+
+ /*
+ * Is there actually nothing to be translated?
+ */
+ if (nkeys == 0)
+ {
+ pfree(keys);
+ return NULL;
+ }
+
+ nkeys_resolved = 0;
+
+ /*
+ * Find the replacement vars.
+ */
+ values = (Var **) palloc0(nkeys * sizeof(Var *));
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ Relids ec_var_relids;
+ Var **ec_vars;
+ int ec_nvars;
+ ListCell *l2;
+
+ /* TODO Re-check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ ec_vars = (Var **) palloc(list_length(ec->ec_members) * sizeof(Var *));
+ ec_nvars = 0;
+ ec_var_relids = NULL;
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ ec_vars[ec_nvars++] = ec_var;
+ ec_var_relids = bms_add_member(ec_var_relids, ec_var->varno);
+ }
+
+ /*
+ * At least two vars are needed so that the EC is usable for
+ * translation.
+ */
+ if (ec_nvars <= 1)
+ {
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+ continue;
+ }
+
+ /*
+ * Now check where this EC can help.
+ */
+ for (i = 0; i < nkeys; i++)
+ {
+ Relids ec_rest;
+ bool relids_ok,
+ key_found;
+ Var *key = keys[i];
+ Var *value = values[i];
+
+ /* Skip this item if it's already resolved. */
+ if (value != NULL)
+ continue;
+
+ /*
+ * Can't translate if the EC does not mention key->varno.
+ */
+ if (!bms_is_member(key->varno, ec_var_relids))
+ continue;
+
+ /*
+ * Besides key, at least one EC member must belong to the relation
+ * we're translating our expression to.
+ */
+ ec_rest = bms_copy(ec_var_relids);
+ ec_rest = bms_del_member(ec_rest, key->varno);
+ relids_ok = bms_overlap(ec_rest, relids);
+ bms_free(ec_rest);
+ if (!relids_ok)
+ continue;
+
+ /*
+ * The preliminary checks passed, so try to find the exact vars.
+ */
+ key_found = false;
+ for (j = 0; j < ec_nvars; j++)
+ {
+ Var *ec_var = ec_vars[j];
+
+ if (!key_found && key->varno == ec_var->varno &&
+ key->varattno == ec_var->varattno)
+ key_found = true;
+
+ /*
+ * If relids contains multiple members, it shouldn't matter to
+ * which one we translate our key. Simply use the first one.
+ *
+ * XXX Shouldn't ec_var be copied?
+ */
+ if (value == NULL && bms_is_member(ec_var->varno, relids))
+ value = ec_var;
+
+ if (key_found && value != NULL)
+ break;
+ }
+
+ if (key_found && value != NULL)
+ {
+ values[i] = value;
+ nkeys_resolved++;
+
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+ }
+
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+
+ /* Don't need to check the remaining ECs? */
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+
+ /* Couldn't compose usable dictionary? */
+ if (nkeys_resolved < nkeys)
+ {
+ pfree(keys);
+ pfree(values);
+ return NULL;
+ }
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_copy(result->gv_eval_at);
+
+ /* The dictionary is ready, so perform the translation. */
+ context.keys = keys;
+ context.values = values;
+ context.nitems = nkeys;
+ context.gv_eval_at_p = &result->gv_eval_at;
+ context.relids = relids;
+ result->gvexpr = (Expr *)
+ translate_expression_to_rels_mutator((Node *) gvi->gvexpr, &context);
+
+ pfree(keys);
+ pfree(values);
+ return result;
+ }
+
+ static Node *
+ translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context * context)
+ {
+ if (node == NULL)
+ return NULL;
+
+ if (IsA(node, Var))
+ {
+ Var *var = castNode(Var, node);
+ Var **key_p;
+ Var *value;
+ int index;
+
+ /*
+ * Simply return the existing variable if already belongs to the
+ * relation we're adjusting the expression to.
+ */
+ if (bms_is_member(var->varno, context->relids))
+ return (Node *) var;
+
+ key_p = bsearch(&var, context->keys, context->nitems, sizeof(Var *),
+ var_dictionary_comparator);
+
+ /* We shouldn't have omitted any var from the dictionary. */
+ Assert(key_p != NULL);
+
+ index = key_p - context->keys;
+ Assert(index >= 0 && index < context->nitems);
+ value = context->values[index];
+
+ /* All values should be present in the dictionary. */
+ Assert(value != NULL);
+
+ /* Update gv_eval_at accordingly. */
+ bms_del_member(*context->gv_eval_at_p, var->varno);
+ *context->gv_eval_at_p = bms_add_member(*context->gv_eval_at_p,
+ value->varno);
+
+ return (Node *) value;
+ }
+
+ return expression_tree_mutator(node, translate_expression_to_rels_mutator,
+ (void *) context);
+ }
+
+ static int
+ var_dictionary_comparator(const void *a, const void *b)
+ {
+ Var **var1_p,
+ **var2_p;
+ Var *var1,
+ *var2;
+
+ var1_p = (Var **) a;
+ var1 = castNode(Var, *var1_p);
+ var2_p = (Var **) b;
+ var2 = castNode(Var, *var2_p);
+
+ if (var1->varno < var2->varno)
+ return -1;
+ else if (var1->varno > var2->varno)
+ return 1;
+
+ if (var1->varattno < var2->varattno)
+ return -1;
+ else if (var1->varattno > var2->varattno)
+ return 1;
+
+ return 0;
+ }
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 974eb58..6e10048
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,21 ****
--- 14,23 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "catalog/pg_class.h"
+ #include "catalog/pg_constraint_fn.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
***************
*** 27,32 ****
--- 29,35 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 46,51 ****
--- 49,56 ----
} PostponedQual;
+ static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+ static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** add_vars_to_targetlist(PlannerInfo *root
*** 241,246 ****
--- 246,784 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression and setup GroupedPathInfo for each base
+ * relation that can product grouped paths.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouping_info_to_base_rels(PlannerInfo *root)
+ {
+ int i;
+ ListCell *lc;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * TODO Consider if this is a real limitation.
+ */
+ if (root->parse->hasWindowFuncs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Is no grouping is possible below the top-level join? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+
+ /*
+ * "other rels" will have their targets built later, by translation of
+ * the target of the parent rel - see set_append_rel_size. If we
+ * wanted to prepare the child rels here, we'd need another iteration
+ * of simple_rel_array_size.
+ */
+ if (rel != NULL && rel->reloptkind == RELOPT_BASEREL)
+ prepare_rel_for_grouping(root, rel);
+ }
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+ static void
+ create_aggregate_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /*
+ * Aggregation push-down does not work w/o aggcombinefn. This field is
+ * not mandatory, so check if this particular aggregate can handle
+ * partial aggregation.
+ */
+ if (!OidIsValid(aggref->aggcombinefn))
+ {
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ /* TODO Initialize gv_width. */
+ gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+ static void
+ create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+ {
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVar exists for each expression usable as grouping key.
+ */
+ foreach(l1, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, l1);
+ Index sortgroupref = te->ressortgroupref;
+
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Try to derive additional grouping expressions from group_pathkeys. This
+ * increases the chance that relation can be grouped even if its columns
+ * are not mentioned in the grouping clause.
+ *
+ * It's important that we have processed the explicit grouping columns
+ * first. If the grouping clause contains multiple expressions belonging
+ * to the same EC, the original (i.e. not derived) one should be preferred
+ * when we build grouping target for a relation. Otherwise we have a
+ * problem when trying to match target entries to grouping clauses during
+ * plan creation, see extract_grouping_cols.
+ */
+ foreach(l1, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l1);
+ EquivalenceClass *ec = pk->pk_eclass;
+ EquivalenceMember *em;
+ Index sortgroupref = 0;
+
+ /* We need equality anywhere in the join tree. */
+ if (ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * TODO Reconsider this restriction. As the grouping expression is
+ * only evaluated at the relation level (and only the result will be
+ * propagated to the final targetlist), volatile function might be
+ * o.k. Need to think what volatile EC exactly means.
+ */
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(l2, ec->ec_members)
+ {
+ ListCell *l3;
+
+ em = lfirst_node(EquivalenceMember, l2);
+
+ /*
+ * We search for grouping expressions now.
+ */
+ if (IsA(em->em_expr, Aggref))
+ continue;
+
+ if (em->em_nullable_relids)
+ continue;
+
+ /*
+ * If target list contains this expression, it should provide us
+ * with the sortgroupref.
+ */
+ foreach(l3, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, l3);
+
+ if (equal(em->em_expr, te->expr))
+ {
+ /*
+ * XXX Not sure if the same expression exist in the
+ * targetlist multiple times and if some occurrences can
+ * miss the ressortgroupref. Maybe we just need to
+ * Assert() here that ressortgroupref is non-zero.
+ */
+ if (te->ressortgroupref > 0)
+ {
+ sortgroupref = te->ressortgroupref;
+ break;
+ }
+ }
+ }
+
+ /*
+ * If a single EC member matches, no need to check the rest of the
+ * EC.
+ */
+ if (sortgroupref > 0)
+ break;
+ }
+
+ if (sortgroupref == 0)
+ /* Go for the next EC. */
+ continue;
+
+ /*
+ * The EC contains a grouping expression, so each member of that EC
+ * should be usable as grouping key. Remember all the expressions of
+ * the EC as well as their (supposedly common) sortgroupref for the
+ * final processing.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ em = lfirst_node(EquivalenceMember, l2);
+ exprs = lappend(exprs, em->em_expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+ }
+
+ /*
+ * Finally construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(expr);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the aggregate should be evaluated. */
+ gvi->gv_eval_at = pull_varnos((Node *) expr);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ /*
+ * Check if rel->reltarget allows the relation to be grouped and initialize
+ * the grouping target(s).
+ *
+ * target_agg is a target that we'll eventually used to aggregate the output
+ * of relation paths.
+ *
+ * If we succeed to create the grouping target, also replace rel->reltarget
+ * with a new one that has sortgrouprefs initialized --- this is necessary for
+ * create_agg_plan to match the grouping clauses against the input target
+ * expressions. (Thus the scan / join that provides the AggPath with input
+ * does not have to care whether the source relation does have grouped target
+ * or not.)
+ *
+ * The *group_exprs_extra_p list may receive additional grouping expressions
+ * that the query does not have. These can make the aggregation of base
+ * relation / join less efficient, but it may still be better than not trying
+ * at all.
+ *
+ * TODO Make sure cost / width of both "result" and "plain" are correct.
+ */
+ void
+ initialize_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target_agg,
+ List **group_exprs_extra_p)
+ {
+ PathTarget *target_plain;
+ ListCell *lc1;
+ List *vars_unresolved = NIL;
+
+ /* The target to replace rel->reltarget. */
+ target_plain = create_empty_pathtarget();
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ GroupedVar *gvar;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var. (As for generic grouping expressions,
+ * add_grouped_vars_to_target will retrieve them from the query
+ * targetlist and add them to target_agg outside this function.)
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ gvar = get_grouping_expression(root, (Expr *) tvar);
+ if (gvar != NULL)
+ {
+ /*
+ * It's o.k. to use the target expression for grouping.
+ *
+ * The actual Var is added to the target. If we used the
+ * containing GroupedVar, references from various clauses (e.g.
+ * join quals) wouldn't work.
+ */
+ add_column_to_pathtarget(target_agg, (Expr *) gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /*
+ * As for the plain target, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(target_plain, gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the plain-var grouping expressions gathered. So far
+ * only store the var in a list.
+ */
+ vars_unresolved = lappend(vars_unresolved, tvar);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ foreach(lc1, vars_unresolved)
+ {
+ Var *var;
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ var = lfirst_node(Var, lc1);
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Dependent var is almost the same as one that has sortgroupref.
+ */
+ if (check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target_agg->exprs, &deps))
+ {
+
+ Index sortgroupref = 0;
+
+ add_column_to_pathtarget(target_agg, (Expr *) var, sortgroupref);
+
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important. But once we have it ...
+ */
+ add_column_to_pathtarget(target_plain, (Expr *) var,
+ sortgroupref);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) or aggregate arguments, but we already have it in
+ * the targets, so don't care.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon as GroupedVars are installed, the top level
+ * aggregates / grouping expressions no longer need direct reference
+ * to arg_var anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by upper join. This includes one that is
+ * referenced by a generic grouping expression but couldn't be
+ * recognized as grouping expression on its own at the top of the
+ * loop.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ *
+ * Since root->parse->groupClause is not supposed to contain this
+ * expression, we need construct special SortGroupClause. Its
+ * tleSortGroupRef needs to be unique within target_agg, so
+ * postpone creation of the SortGroupRefs until we're done with
+ * the iteration of rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced either by aggregate argument
+ * or by generic grouping expression. The aggregation target
+ * should not contain it, as it only provides input for the final
+ * aggregation.
+ */
+ }
+
+ /*
+ * The var is not suitable for grouping, but the plain target ought to
+ * stay complete.
+ */
+ add_column_to_pathtarget(target_plain, (Expr *) var, 0);
+ }
+
+ /*
+ * Apply the adjusted input target as the replacement is complete now.
+ */
+ /* TODO Free the old one. */
+ rel->reltarget = target_plain;
+ }
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 2e6a60e..9a15e5a
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 178,183 ****
--- 178,191 ----
(*qp_callback) (root, qp_extra);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, Initialize GroupedPathInfo
+ * of base relations capable to do the grouping and setup
+ * root->grouped_var_list.
+ */
+ add_grouping_info_to_base_rels(root);
+
+ /*
* Examine any "placeholder" expressions generated during subquery pullup.
* Make sure that the Vars they need are marked as needed at the relevant
* join level. This must be done before join removal because it might
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 39e45d5..167e7a8
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 27,32 ****
--- 27,33 ----
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "parser/parse_oper.h"
#include "utils/hsearch.h"
*************** build_joinrel_partition_info(RelOptInfo
*** 1749,1751 ****
--- 1750,2052 ----
joinrel->nullable_partexprs[cnt] = nullable_partexpr;
}
}
+
+ /*
+ * If the relation can produce grouped paths, create GroupedPathInfo for it
+ * and create target for aggregation of the relation output.
+ */
+ void
+ prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *gvis;
+ List *aggregates = NIL;
+ List *grp_exprs = NIL;
+ bool found_higher_agg;
+ ListCell *lc;
+ GroupedPathInfo *gpi;
+ PathTarget *target;
+ List *grp_exprs_extra = NIL;
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some grouping expression or Aggref
+ * argument), we'd just let initialize_grouped_targets create GroupedVar
+ * for the corresponding expression (phexpr). On the other hand, if we
+ * knew that the PHV is evaluated below the current rel, we'd ignore it
+ * because the referencing GroupedVar would take care of propagation of
+ * the value to upper joins. (PHV whose ph_eval_at is above the current
+ * rel make the aggregation push-down impossible in any case because the
+ * partial aggregation would receive wrong input if we ignored the
+ * ph_eval_at.)
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return;
+ }
+
+ /*
+ * target_agg will be used to aggregate the output of the current
+ * relation's paths.
+ */
+ target = create_empty_pathtarget();
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and initialize_grouped_targets_target
+ * take care?) of this rel to NULL are provably below rel. (It's ok if rel
+ * is one of these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * While create_grouping_expr_grouped_var_infos generates equivalence
+ * class implied plain-Var grouping expressions at once, doing so for
+ * generic expressions would be complex and might result in very long list
+ * to search in. So generate the EC-related grouping expressions for each
+ * particular set of relids.
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ /*
+ * Should have been processed by
+ * create_grouping_expr_grouped_var_infos.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ gvi = translate_expression_to_rels(root, gvi, rel->relids);
+ if (gvi != NULL)
+ gvis = lappend(gvis, gvi);
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_higher_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * initialize_grouped_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ *
+ * XXX A plain Var could actually be handled w/o GroupedVar, but
+ * thus initialize_grouped_targets would have to spend extra
+ * effort looking for the EC-related vars, instead of relying on
+ * create_grouping_expr_grouped_var_infos. (Processing of
+ * particular expression would look different, so we could hardly
+ * reuse the same piece of code.)
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Accept the aggregate / grouping expression.
+ *
+ * (GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.)
+ */
+ if (IsA(gvi->gvexpr, Aggref))
+ aggregates = lappend(aggregates, gvi);
+ else
+ grp_exprs = lappend(grp_exprs, gvi);
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids) &&
+ IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate / grouping
+ * expression that needs more than this rel.
+ */
+ found_higher_agg = true;
+ }
+ }
+ list_free(gvis);
+
+ /*
+ * Give up if some other aggregate(s) need multiple relations including
+ * the current one. The problem is that grouping of the current relation
+ * could make some input variables unavailable for the "higher aggregate",
+ * and it'd also decrease the number of input rows the "higher aggregate"
+ * receives.
+ *
+ * In contrast, grp_exprs is only supposed to contain generic grouping
+ * expression, so it can be NIL so far. If all the grouping keys are just
+ * plain Vars, initialize_grouped_targets will take care of them.
+ */
+ if (found_higher_agg)
+ return;
+
+ /*
+ * Add plain-var grouping expressions to the target.
+ */
+ initialize_grouped_target(root, rel, target, &grp_exprs_extra);
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ *
+ * This test was postponed because initialize_grouped_targets initializes
+ * sortgrouprefs of rel->reltarget. This information is useful to avoid
+ * final aggregation. In particular, even non-grouped relation (e.g.
+ * unique index scan) can generate unique values of grouping keys. See
+ * check_group_key_uniqueness for details.
+ */
+ if (aggregates == NIL)
+ return;
+
+ /*
+ * Add (non-Var) grouping expressions (in the form of GroupedVar) to
+ * target_agg.
+ *
+ * Follow the convention that the grouping expressions should precede the
+ * aggregates.
+ */
+ add_grouped_vars_to_target(root, target, grp_exprs);
+
+ /*
+ * Initialize GroupedPathInfo.
+ */
+ Assert(rel->gpi == NULL);
+ gpi = makeNode(GroupedPathInfo);
+ gpi->pathlist = NIL;
+ gpi->partial_pathlist = NIL;
+ rel->gpi = gpi;
+
+ /*
+ * Partial aggregation makes no sense w/o grouping expressions.
+ */
+ if (list_length(target->exprs) == 0)
+ {
+ pfree(rel->gpi);
+ rel->gpi = NULL;
+ return;
+ }
+
+ /*
+ * If the aggregation target should have extra grouping expressions, add
+ * them now. This step includes assignment of tleSortGroupRef's which we
+ * can generate now (the "ordinary" grouping expression are present in the
+ * target by now).
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * TODO Verify that these fields are sufficient for this special
+ * SortGroupClause.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ NULL, &cl->eqop, NULL,
+ &cl->hashable);
+ gpi->sortgroupclauses = lappend(gpi->sortgroupclauses, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ *
+ * TODO If the target already contains this var for another reason
+ * (e.g. a an input for generic grouping expression), it'll
+ * probably not have sortgrouprefs set. Consider removing that
+ * (and adjust target width accordingly) so that the var is not
+ * duplicated.
+ */
+ add_column_to_pathtarget(rel->reltarget, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the target(s).
+ */
+ add_grouped_vars_to_target(root, target, aggregates);
+
+ /* TODO Check if copy is needed here. */
+ gpi->target = copy_pathtarget(target);
+
+ /*
+ * Add the non-Var group expressions to a separate target. If
+ * rel->reltarget should be used to generate input for partial
+ * aggregation, the corresponding path should include these expressions.
+ */
+ if (list_length(grp_exprs) > 0)
+ {
+ rel->gpi->group_exprs = create_empty_pathtarget();
+ add_grouped_vars_to_target(root, rel->gpi->group_exprs, grp_exprs);
+ }
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 9345891..abdc5d9
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 783,853 ----
}
/*
+ * For each aggregate add GroupedVar to the grouped target.
+ *
+ * Caller passes the aggregates in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+ void
+ add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+ {
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+ GroupedVar *gvar;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_column_to_pathtarget(target, (Expr *) gvar, gvi->sortgroupref);
+ }
+ }
+
+ /*
+ * Return GroupedVar containing the passed-in expression if one exists, or
+ * NULL if the expression cannot be used as grouping key.
+ *
+ * create_grouping_expr_grouped_var_infos should have been called before this
+ * function gets called the first time.
+ */
+ GroupedVar *
+ get_grouping_expression(PlannerInfo *root, Expr *expr)
+ {
+ ListCell *lc;
+
+ /* Caller should have noticed the empty list much earlier. */
+ Assert(root->grouped_var_list != NIL);
+
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ GroupedVar *result = makeNode(GroupedVar);
+
+ Assert(gvi->sortgroupref > 0);
+ result->gvexpr = gvi->gvexpr;
+ result->gvid = gvi->gvid;
+ result->sortgroupref = gvi->sortgroupref;
+ return result;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return NULL;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
new file mode 100644
index e367221..7090d02
*** a/src/include/optimizer/clauses.h
--- b/src/include/optimizer/clauses.h
*************** extern Node *estimate_expression_value(P
*** 84,88 ****
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
!
#endif /* CLAUSES_H */
--- 84,90 ----
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
! extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
! GroupedVarInfo *gvi,
! Relids relids);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index e9ed16a..a5d9b2c
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern RelOptInfo *build_child_join_rel(
*** 296,300 ****
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
!
#endif /* PATHNODE_H */
--- 296,300 ----
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
! extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index f1d16cf..d3dbf51
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** extern void add_base_rels_to_query(Plann
*** 75,80 ****
--- 75,82 ----
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+ extern void add_grouping_info_to_base_rels(PlannerInfo *root);
+ extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 0d3ec92..a725818
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern Node *get_sortgroupclause_expr(So
*** 41,46 ****
--- 41,50 ----
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
+ extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List *target_group_clauses,
+ List **grouping_clauses,
+ List **grouping_exprs, List **agg_exprs);
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 69,82 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+ extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
+ extern void initialize_grouped_target(PlannerInfo *root,
+ RelOptInfo *rel,
+ PathTarget *target_agg,
+ List **group_exprs_extra_p);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
agg_pushdown_v4/03_add_path.diff
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
new file mode 100644
index 94e50e9..9552376
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
*************** fileGetForeignPaths(PlannerInfo *root,
*** 551,557 ****
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! coptions));
/*
* If data file was sorted, and we knew it somehow, we could insert
--- 551,557 ----
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! coptions), false);
/*
* If data file was sorted, and we knew it somehow, we could insert
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
new file mode 100644
index fb65e2e..459c6b9
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 907,913 ****
NULL, /* no outer rel either */
NULL, /* no extra plan */
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path);
/* Add paths with pathkeys */
add_paths_with_pathkeys_for_rel(root, baserel, NULL);
--- 907,913 ----
NULL, /* no outer rel either */
NULL, /* no extra plan */
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
/* Add paths with pathkeys */
add_paths_with_pathkeys_for_rel(root, baserel, NULL);
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 1079,1085 ****
param_info->ppi_req_outer,
NULL,
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path);
}
}
--- 1079,1085 ----
param_info->ppi_req_outer,
NULL,
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
}
}
*************** add_paths_with_pathkeys_for_rel(PlannerI
*** 4343,4349 ****
useful_pathkeys,
NULL,
epq_path,
! NIL));
}
}
--- 4343,4350 ----
useful_pathkeys,
NULL,
epq_path,
! NIL),
! false);
}
}
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4574,4580 ****
NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath);
/* Consider pathkeys for the join relation */
add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
--- 4575,4581 ----
NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, false);
/* Consider pathkeys for the join relation */
add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4898,4904 ****
NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
! add_path(grouped_rel, (Path *) grouppath);
}
/*
--- 4899,4905 ----
NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
! add_path(grouped_rel, (Path *) grouppath, false);
}
/*
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index a6efb4e..e5c9233
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 697,703 ****
required_outer = rel->lateral_relids;
/* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
/* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
--- 697,703 ----
required_outer = rel->lateral_relids;
/* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0), false);
/* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
*************** create_plain_partial_paths(PlannerInfo *
*** 726,732 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 726,733 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers),
! false);
}
/*
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 812,818 ****
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path);
/* For the moment, at least, there are no other paths to consider */
}
--- 813,819 ----
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path, false);
/* For the moment, at least, there are no other paths to consider */
}
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1492,1498 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels));
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1493,1499 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels), false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1520,1526 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1521,1527 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath, false);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1573,1579 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels));
}
}
--- 1574,1580 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels), false);
}
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1669,1682 ****
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels));
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels));
}
}
--- 1670,1685 ----
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
}
}
*************** set_dummy_rel_pathlist(RelOptInfo *rel)
*** 1809,1815 ****
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
--- 1812,1818 ----
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
*************** set_subquery_pathlist(PlannerInfo *root,
*** 2023,2029 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer));
}
}
--- 2026,2033 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer),
! false);
}
}
*************** set_function_pathlist(PlannerInfo *root,
*** 2092,2098 ****
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer));
}
/*
--- 2096,2102 ----
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer), false);
}
/*
*************** set_values_pathlist(PlannerInfo *root, R
*** 2112,2118 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer));
}
/*
--- 2116,2122 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer), false);
}
/*
*************** set_tablefunc_pathlist(PlannerInfo *root
*** 2133,2139 ****
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer));
}
/*
--- 2137,2143 ----
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer), false);
}
/*
*************** set_cte_pathlist(PlannerInfo *root, RelO
*** 2199,2205 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer));
}
/*
--- 2203,2209 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer), false);
}
/*
*************** set_namedtuplestore_pathlist(PlannerInfo
*** 2226,2232 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
--- 2230,2237 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
*************** set_worktable_pathlist(PlannerInfo *root
*** 2279,2285 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer));
}
/*
--- 2284,2291 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer),
! false);
}
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2311,2317 ****
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path);
/*
* For each useful ordering, we can consider an order-preserving Gather
--- 2317,2323 ----
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path, false);
/*
* For each useful ordering, we can consider an order-preserving Gather
*************** generate_gather_paths(PlannerInfo *root,
*** 2327,2333 ****
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path);
}
}
--- 2333,2339 ----
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, false);
}
}
*************** create_partial_bitmap_paths(PlannerInfo
*** 3150,3156 ****
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3156,3163 ----
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers),
! false);
}
/*
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index ce32b8a..096a527
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
***************
*** 91,96 ****
--- 91,97 ----
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
*************** static Selectivity get_foreign_key_join_
*** 160,166 ****
Relids inner_relids,
SpecialJoinInfo *sjinfo,
List **restrictlist);
! static void set_rel_width(PlannerInfo *root, RelOptInfo *rel);
static double relation_byte_size(double tuples, int width);
static double page_size(double tuples, int width);
static double get_parallel_divisor(Path *path);
--- 161,168 ----
Relids inner_relids,
SpecialJoinInfo *sjinfo,
List **restrictlist);
! static void set_rel_width(PlannerInfo *root, RelOptInfo *rel,
! PathTarget *reltarget);
static double relation_byte_size(double tuples, int width);
static double page_size(double tuples, int width);
static double get_parallel_divisor(Path *path);
*************** set_baserel_size_estimates(PlannerInfo *
*** 4056,4062 ****
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel);
}
/*
--- 4058,4071 ----
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel, rel->reltarget);
!
! /*
! * If the relation can produce grouped path, estimate width and costs of
! * the corresponding target.
! */
! if (rel->gpi)
! set_rel_width(root, rel, rel->gpi->target);
}
/*
*************** set_foreign_size_estimates(PlannerInfo *
*** 4807,4813 ****
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel);
}
--- 4816,4822 ----
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel, rel->reltarget);
}
*************** set_foreign_size_estimates(PlannerInfo *
*** 4833,4839 ****
* building join relations or post-scan/join pathtargets.
*/
static void
! set_rel_width(PlannerInfo *root, RelOptInfo *rel)
{
Oid reloid = planner_rt_fetch(rel->relid, root)->relid;
int32 tuple_width = 0;
--- 4842,4848 ----
* building join relations or post-scan/join pathtargets.
*/
static void
! set_rel_width(PlannerInfo *root, RelOptInfo *rel, PathTarget *reltarget)
{
Oid reloid = planner_rt_fetch(rel->relid, root)->relid;
int32 tuple_width = 0;
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 4841,4850 ****
ListCell *lc;
/* Vars are assumed to have cost zero, but other exprs do not */
! rel->reltarget->cost.startup = 0;
! rel->reltarget->cost.per_tuple = 0;
! foreach(lc, rel->reltarget->exprs)
{
Node *node = (Node *) lfirst(lc);
--- 4850,4859 ----
ListCell *lc;
/* Vars are assumed to have cost zero, but other exprs do not */
! reltarget->cost.startup = 0;
! reltarget->cost.per_tuple = 0;
! foreach(lc, reltarget->exprs)
{
Node *node = (Node *) lfirst(lc);
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 4919,4926 ****
tuple_width += phinfo->ph_width;
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
! rel->reltarget->cost.startup += cost.startup;
! rel->reltarget->cost.per_tuple += cost.per_tuple;
}
else
{
--- 4928,4946 ----
tuple_width += phinfo->ph_width;
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
! reltarget->cost.startup += cost.startup;
! reltarget->cost.per_tuple += cost.per_tuple;
! }
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = (GroupedVar *) node;
! GroupedVarInfo *gvinfo = find_grouped_var_info(root, gvar);
! QualCost cost;
!
! tuple_width += gvinfo->gv_width;
! cost_qual_eval_node(&cost, (Node *) gvar->gvexpr, root);
! reltarget->cost.startup += cost.startup;
! reltarget->cost.per_tuple += cost.per_tuple;
}
else
{
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 4937,4944 ****
tuple_width += item_width;
/* Not entirely clear if we need to account for cost, but do so */
cost_qual_eval_node(&cost, node, root);
! rel->reltarget->cost.startup += cost.startup;
! rel->reltarget->cost.per_tuple += cost.per_tuple;
}
}
--- 4957,4964 ----
tuple_width += item_width;
/* Not entirely clear if we need to account for cost, but do so */
cost_qual_eval_node(&cost, node, root);
! reltarget->cost.startup += cost.startup;
! reltarget->cost.per_tuple += cost.per_tuple;
}
}
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 4975,4981 ****
}
Assert(tuple_width >= 0);
! rel->reltarget->width = tuple_width;
}
/*
--- 4995,5001 ----
}
Assert(tuple_width >= 0);
! reltarget->width = tuple_width;
}
/*
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index f353803..1dea66c
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
*************** create_index_paths(PlannerInfo *root, Re
*** 338,344 ****
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
--- 338,344 ----
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath, false);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
*************** create_index_paths(PlannerInfo *root, Re
*** 415,421 ****
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath);
}
}
}
--- 415,421 ----
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath, false);
}
}
}
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,795 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath);
if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
--- 789,795 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, false);
if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1077,1083 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1129,1135 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 310262d..ff784b5
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** try_nestloop_path(PlannerInfo *root,
*** 422,428 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
/*
* If the inner path is parameterized, it is parameterized by the
--- 422,428 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
/*
* If the inner path is parameterized, it is parameterized by the
*************** try_nestloop_path(PlannerInfo *root,
*** 455,461 ****
inner_path,
extra->restrictlist,
pathkeys,
! required_outer));
}
else
{
--- 455,461 ----
inner_path,
extra->restrictlist,
pathkeys,
! required_outer), false);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 513,519 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/*
--- 513,520 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/*
*************** try_partial_nestloop_path(PlannerInfo *r
*** 543,549 ****
inner_path,
extra->restrictlist,
pathkeys,
! NULL));
}
/*
--- 544,550 ----
inner_path,
extra->restrictlist,
pathkeys,
! NULL), false);
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 617,623 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
--- 618,624 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
*************** try_mergejoin_path(PlannerInfo *root,
*** 632,638 ****
required_outer,
mergeclauses,
outersortkeys,
! innersortkeys));
}
else
{
--- 633,639 ----
required_outer,
mergeclauses,
outersortkeys,
! innersortkeys), false);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 691,697 ****
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/* Might be good enough to be worth trying, so let's try it. */
--- 692,699 ----
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/* Might be good enough to be worth trying, so let's try it. */
*************** try_partial_mergejoin_path(PlannerInfo *
*** 708,714 ****
NULL,
mergeclauses,
outersortkeys,
! innersortkeys));
}
/*
--- 710,716 ----
NULL,
mergeclauses,
outersortkeys,
! innersortkeys), false);
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 751,757 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
add_path(joinrel, (Path *)
create_hashjoin_path(root,
--- 753,759 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, false))
{
add_path(joinrel, (Path *)
create_hashjoin_path(root,
*************** try_hashjoin_path(PlannerInfo *root,
*** 763,769 ****
inner_path,
extra->restrictlist,
required_outer,
! hashclauses));
}
else
{
--- 765,771 ----
inner_path,
extra->restrictlist,
required_outer,
! hashclauses), false);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 809,815 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
/* Might be good enough to be worth trying, so let's try it. */
--- 811,817 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL, false))
return;
/* Might be good enough to be worth trying, so let's try it. */
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 823,829 ****
inner_path,
extra->restrictlist,
NULL,
! hashclauses));
}
/*
--- 825,831 ----
inner_path,
extra->restrictlist,
NULL,
! hashclauses), false);
}
/*
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 2b868c5..382c0f0
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** mark_dummy_rel(RelOptInfo *rel)
*** 1232,1238 ****
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
--- 1232,1238 ----
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index a2fe661..91d855c
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** create_tidscan_paths(PlannerInfo *root,
*** 266,270 ****
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 266,270 ----
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer), false);
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 6e10048..4841f9a
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
*************** create_aggregate_grouped_var_infos(Plann
*** 435,440 ****
--- 435,443 ----
else
gvi->gv_eval_at = NULL;
+ gvi->gv_width = get_typavgwidth(exprType((Node *) gvi->gvexpr),
+ exprTypmod((Node *) gvi->gvexpr));
+
root->grouped_var_list = lappend(root->grouped_var_list, gvi);
}
}
*************** create_grouping_expr_grouped_var_infos(P
*** 596,601 ****
--- 599,605 ----
}
}
+
/*
* Check if rel->reltarget allows the relation to be grouped and initialize
* the grouping target(s).
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index bba8a1f..51d13ab
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,229 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual));
}
/*
--- 223,230 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual),
! false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 9a15e5a..c1939a1
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,89 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,90 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 25e0782..6202fab
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** inheritance_planner(PlannerInfo *root)
*** 1511,1517 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)));
}
/*--------------------
--- 1511,1518 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)),
! false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 2132,2138 ****
}
/* And shove it into final_rel */
! add_path(final_rel, path);
}
/*
--- 2133,2139 ----
}
/* And shove it into final_rel */
! add_path(final_rel, path, false);
}
/*
*************** create_grouping_paths(PlannerInfo *root,
*** 3692,3698 ****
(List *) parse->havingQual);
}
! add_path(grouped_rel, path);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
--- 3693,3699 ----
(List *) parse->havingQual);
}
! add_path(grouped_rel, path, false);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
*************** create_grouping_paths(PlannerInfo *root,
*** 3869,3875 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
--- 3870,3877 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3878,3884 ****
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups));
}
}
}
--- 3880,3887 ----
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3909,3915 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
}
}
}
--- 3912,3918 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3961,3967 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
else if (parse->groupClause)
{
--- 3964,3970 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
else if (parse->groupClause)
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3976,3982 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
else
{
--- 3979,3985 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
else
{
*************** create_grouping_paths(PlannerInfo *root,
*** 4025,4031 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 4028,4034 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4034,4040 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
/*
* The point of using Gather Merge rather than Gather is that it
--- 4037,4043 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
/*
* The point of using Gather Merge rather than Gather is that it
*************** create_grouping_paths(PlannerInfo *root,
*** 4087,4093 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 4090,4096 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4096,4102 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
}
}
--- 4099,4105 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4141,4147 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
}
--- 4144,4150 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4179,4185 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
}
}
}
--- 4182,4188 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
}
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4381,4387 ****
strat,
new_rollups,
agg_costs,
! dNumGroups));
return;
}
--- 4384,4390 ----
strat,
new_rollups,
agg_costs,
! dNumGroups), false);
return;
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4539,4545 ****
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups));
}
}
--- 4542,4548 ----
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups), false);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4556,4562 ****
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups));
}
/*
--- 4559,4565 ----
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups), false);
}
/*
*************** create_one_window_path(PlannerInfo *root
*** 4741,4747 ****
window_pathkeys);
}
! add_path(window_rel, path);
}
/*
--- 4744,4750 ----
window_pathkeys);
}
! add_path(window_rel, path, false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4847,4853 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
}
--- 4850,4856 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4874,4880 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
/*
--- 4877,4883 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4921,4927 ****
parse->distinctClause,
NIL,
NULL,
! numDistinctRows));
}
/* Give a helpful error if we failed to find any implementation */
--- 4924,4930 ----
parse->distinctClause,
NIL,
NULL,
! numDistinctRows), false);
}
/* Give a helpful error if we failed to find any implementation */
*************** create_ordered_paths(PlannerInfo *root,
*** 5019,5025 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5022,5028 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
*************** create_ordered_paths(PlannerInfo *root,
*** 5069,5075 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5072,5078 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
new file mode 100644
index 1c84a2c..175429f
*** a/src/backend/optimizer/prep/prepunion.c
--- b/src/backend/optimizer/prep/prepunion.c
*************** plan_set_operations(PlannerInfo *root)
*** 221,227 ****
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
--- 221,227 ----
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path, false);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index ce8a2ca..3e0c313
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
*************** set_cheapest(RelOptInfo *parent_rel)
*** 417,424 ****
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
--- 417,425 ----
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
+ List *pathlist;
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 435,440 ****
--- 436,449 ----
/* Pretend parameterized paths have no pathkeys, per comment above */
new_path_pathkeys = new_path->param_info ? NIL : new_path->pathkeys;
+ if (!grouped)
+ pathlist = parent_rel->pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->pathlist;
+ }
+
/*
* Loop to check proposed new path against old paths. Note it is possible
* for more than one old path to be tossed out because new_path dominates
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 444,450 ****
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
--- 453,459 ----
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 590,597 ****
*/
if (remove_old)
{
! parent_rel->pathlist = list_delete_cell(parent_rel->pathlist,
! p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
--- 599,605 ----
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 622,630 ****
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(parent_rel->pathlist, insert_after, new_path);
else
! parent_rel->pathlist = lcons(new_path, parent_rel->pathlist);
}
else
{
--- 630,643 ----
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->pathlist = pathlist;
! else
! parent_rel->gpi->pathlist = pathlist;
}
else
{
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 654,661 ****
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer)
{
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
--- 667,675 ----
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped)
{
+ List *pathlist;
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 664,672 ****
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup : parent_rel->consider_startup;
! foreach(p1, parent_rel->pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 678,695 ----
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup :
! parent_rel->consider_startup;
! if (!grouped)
! pathlist = parent_rel->pathlist;
! else
! {
! Assert(parent_rel->gpi != NULL);
! pathlist = parent_rel->gpi->pathlist;
! }
!
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 757,779 ****
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->partial_pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
--- 780,811 ----
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
+ List *pathlist;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
+
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
*************** add_partial_path(RelOptInfo *parent_rel,
*** 827,838 ****
}
/*
! * Remove current element from partial_pathlist if dominated by new.
*/
if (remove_old)
{
! parent_rel->partial_pathlist =
! list_delete_cell(parent_rel->partial_pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
--- 859,869 ----
}
/*
! * Remove current element from pathlist if dominated by new.
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
*************** add_partial_path(RelOptInfo *parent_rel,
*** 847,854 ****
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the partial_pathlist; we will not add new_path, and we
! * assume new_path cannot dominate any later path.
*/
if (!accept_new)
break;
--- 878,885 ----
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the pathlist; we will not add new_path, and we assume
! * new_path cannot dominate any later path.
*/
if (!accept_new)
break;
*************** add_partial_path(RelOptInfo *parent_rel,
*** 858,867 ****
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(parent_rel->partial_pathlist, insert_after, new_path);
else
! parent_rel->partial_pathlist =
! lcons(new_path, parent_rel->partial_pathlist);
}
else
{
--- 889,902 ----
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->partial_pathlist = pathlist;
! else
! parent_rel->gpi->partial_pathlist = pathlist;
}
else
{
*************** add_partial_path(RelOptInfo *parent_rel,
*** 882,890 ****
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys)
{
ListCell *p1;
/*
* Our goal here is twofold. First, we want to find out whether this path
--- 917,934 ----
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys, bool grouped)
{
ListCell *p1;
+ List *pathlist;
+
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
/*
* Our goal here is twofold. First, we want to find out whether this path
*************** add_partial_path_precheck(RelOptInfo *pa
*** 894,903 ****
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist to be very short, and getting a definitive
! * answer at this stage avoids the need to call add_path_precheck.
*/
! foreach(p1, parent_rel->partial_pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 938,948 ----
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist / grouped_pathlist to be very short, and
! * getting a definitive answer at this stage avoids the need to call
! * add_path_precheck.
*/
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_partial_path_precheck(RelOptInfo *pa
*** 926,932 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL))
return false;
return true;
--- 971,977 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, false))
return false;
return true;
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
new file mode 100644
index b8d7d3f..4f8de90
*** a/src/backend/optimizer/util/var.c
--- b/src/backend/optimizer/util/var.c
*************** alias_relid_set(PlannerInfo *root, Relid
*** 840,842 ****
--- 840,864 ----
}
return result;
}
+
+ /*
+ * Return GroupedVarInfo for given GroupedVar.
+ *
+ * XXX Consider better location of this routine.
+ */
+ GroupedVarInfo *
+ find_grouped_var_info(PlannerInfo *root, GroupedVar *gvar)
+ {
+ ListCell *l;
+
+ foreach(l, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, l);
+
+ if (gvi->gvid == gvar->gvid)
+ return gvi;
+ }
+
+ elog(ERROR, "GroupedVarInfo not found");
+ return NULL; /* keep compiler quiet */
+ }
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index 5327fc9..ca76b0d
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_bool ConfigureNames
*** 920,926 ****
false,
NULL, NULL, NULL
},
-
{
{"geqo", PGC_USERSET, QUERY_TUNING_GEQO,
gettext_noop("Enables genetic query optimization."),
--- 920,925 ----
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index a5d9b2c..5e7e97a
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern int compare_path_costs(Path *path
*** 25,37 ****
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
--- 25,39 ----
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path,
! bool grouped);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys,
! bool grouped);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
diff --git a/src/include/optimizer/var.h b/src/include/optimizer/var.h
new file mode 100644
index 6186152..22816db
*** a/src/include/optimizer/var.h
--- b/src/include/optimizer/var.h
*************** extern bool contain_vars_of_level(Node *
*** 36,40 ****
--- 36,42 ----
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
extern Node *flatten_join_alias_vars(PlannerInfo *root, Node *node);
+ extern GroupedVarInfo *find_grouped_var_info(PlannerInfo *root,
+ GroupedVar *gvar);
#endif /* VAR_H */
agg_pushdown_v4/04_refactor_planner.diff
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 6202fab..f78e22f
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 130,138 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
--- 130,135 ----
*************** static void consider_groupingsets_paths(
*** 147,152 ****
--- 144,161 ----
grouping_sets_data *gd,
const AggClauseCosts *agg_costs,
double dNumGroups);
+ static void sort_agg_partial_grouped_paths(PlannerInfo *root, List *pathlist,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups,
+ RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target);
+ static void hash_agg_partial_grouped_path(PlannerInfo *root, Path *path,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups,
+ RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target);
static RelOptInfo *create_window_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *input_target,
*************** get_number_of_groups(PlannerInfo *root,
*** 3539,3578 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3548,3553 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3994,4108 ****
* path. We can do this using either Gather or Gather Merge.
*/
if (grouped_rel->partial_pathlist)
! {
! Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
! double total_groups = path->rows * path->parallel_workers;
!
! path = (Path *) create_gather_path(root,
! grouped_rel,
! path,
! partial_grouping_target,
! NULL,
! &total_groups);
!
! /*
! * Since Gather's output is always unsorted, we'll need to sort,
! * unless there's no GROUP BY clause or a degenerate (constant)
! * one, in which case there will only be a single group.
! */
! if (root->group_pathkeys)
! path = (Path *) create_sort_path(root,
! grouped_rel,
! path,
! root->group_pathkeys,
! -1.0);
!
! if (parse->hasAggs)
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! path,
! target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! else
! add_path(grouped_rel, (Path *)
! create_group_path(root,
! grouped_rel,
! path,
! target,
! parse->groupClause,
! (List *) parse->havingQual,
! dNumGroups), false);
!
! /*
! * The point of using Gather Merge rather than Gather is that it
! * can preserve the ordering of the input path, so there's no
! * reason to try it unless (1) it's possible to produce more than
! * one output row and (2) we want the output path to be ordered.
! */
! if (parse->groupClause != NIL && root->group_pathkeys != NIL)
! {
! foreach(lc, grouped_rel->partial_pathlist)
! {
! Path *subpath = (Path *) lfirst(lc);
! Path *gmpath;
! double total_groups;
!
! /*
! * It's useful to consider paths that are already properly
! * ordered for Gather Merge, because those don't need a
! * sort. It's also useful to consider the cheapest path,
! * because sorting it in parallel and then doing Gather
! * Merge may be better than doing an unordered Gather
! * followed by a sort. But there's no point in
! * considering non-cheapest paths that aren't already
! * sorted correctly.
! */
! if (path != subpath &&
! !pathkeys_contained_in(root->group_pathkeys,
! subpath->pathkeys))
! continue;
!
! total_groups = subpath->rows * subpath->parallel_workers;
!
! gmpath = (Path *)
! create_gather_merge_path(root,
! grouped_rel,
! subpath,
! partial_grouping_target,
! root->group_pathkeys,
! NULL,
! &total_groups);
!
! if (parse->hasAggs)
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! gmpath,
! target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! else
! add_path(grouped_rel, (Path *)
! create_group_path(root,
! grouped_rel,
! gmpath,
! target,
! parse->groupClause,
! (List *) parse->havingQual,
! dNumGroups), false);
! }
! }
! }
}
if (can_hash)
--- 3969,3979 ----
* path. We can do this using either Gather or Gather Merge.
*/
if (grouped_rel->partial_pathlist)
! sort_agg_partial_grouped_paths(root,
! grouped_rel->partial_pathlist,
! &agg_final_costs,
! dNumGroups, grouped_rel,
! partial_grouping_target, target);
}
if (can_hash)
*************** create_grouping_paths(PlannerInfo *root,
*** 4157,4189 ****
{
Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
! hashaggtablesize = estimate_hashagg_tablesize(path,
! &agg_final_costs,
! dNumGroups);
!
! if (hashaggtablesize < work_mem * 1024L)
! {
! double total_groups = path->rows * path->parallel_workers;
!
! path = (Path *) create_gather_path(root,
! grouped_rel,
! path,
! partial_grouping_target,
! NULL,
! &total_groups);
!
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! path,
! target,
! AGG_HASHED,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! }
}
}
--- 4028,4036 ----
{
Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
! hash_agg_partial_grouped_path(root, path, &agg_final_costs,
! dNumGroups, grouped_rel,
! partial_grouping_target, target);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4563,4568 ****
--- 4410,4582 ----
}
/*
+ * Subroutine of create_grouping_paths() to apply GatherPath or
+ * GatherMergePath and AGG_SORTED AggPath to cases which only differ in the
+ * GatherPath / GatherMergePath target.
+ */
+ static void
+ sort_agg_partial_grouped_paths(PlannerInfo *root, List *pathlist,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups, RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target)
+ {
+ Path *path = (Path *) linitial(pathlist);
+ double total_groups = path->rows * path->parallel_workers;
+ Query *parse = root->parse;
+ ListCell *lc;
+
+ /*
+ * Generate a complete GroupAgg Path atop of the cheapest partial path. We
+ * can do this using either Gather or Gather Merge.
+ */
+ path = (Path *) create_gather_path(root,
+ grouped_rel,
+ path,
+ gather_target,
+ NULL,
+ &total_groups);
+
+ /*
+ * Since Gather's output is always unsorted, we'll need to sort, unless
+ * there's no GROUP BY clause or a degenerate (constant) one, in which
+ * case there will only be a single group.
+ */
+ if (root->group_pathkeys)
+ path = (Path *) create_sort_path(root,
+ grouped_rel,
+ path,
+ root->group_pathkeys,
+ -1.0);
+
+ if (parse->hasAggs)
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ parse->groupClause ? AGG_SORTED : AGG_PLAIN,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ else
+ add_path(grouped_rel, (Path *)
+ create_group_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ dNumGroups), false);
+
+ /*
+ * The point of using Gather Merge rather than Gather is that it can
+ * preserve the ordering of the input path, so there's no reason to try it
+ * unless (1) it's possible to produce more than one output row and (2) we
+ * want the output path to be ordered.
+ */
+ if (parse->groupClause != NIL && root->group_pathkeys != NIL)
+ {
+ foreach(lc, pathlist)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Path *gmpath;
+ double total_groups;
+
+ /*
+ * It's useful to consider paths that are already properly ordered
+ * for Gather Merge, because those don't need a sort. It's also
+ * useful to consider the cheapest path, because sorting it in
+ * parallel and then doing Gather Merge may be better than doing
+ * an unordered Gather followed by a sort. But there's no point
+ * in considering non-cheapest paths that aren't already sorted
+ * correctly.
+ */
+ if (path != subpath &&
+ !pathkeys_contained_in(root->group_pathkeys,
+ subpath->pathkeys))
+ continue;
+
+ total_groups = subpath->rows * subpath->parallel_workers;
+
+ gmpath = (Path *)
+ create_gather_merge_path(root,
+ grouped_rel,
+ subpath,
+ gather_target,
+ root->group_pathkeys,
+ NULL,
+ &total_groups);
+
+ if (parse->hasAggs)
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ gmpath,
+ group_target,
+ parse->groupClause ? AGG_SORTED : AGG_PLAIN,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ else
+ add_path(grouped_rel, (Path *)
+ create_group_path(root,
+ grouped_rel,
+ gmpath,
+ group_target,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ dNumGroups), false);
+ }
+ }
+ }
+
+ /*
+ * Subroutine of create_grouping_paths() to apply GatherPath and AGG_HASHED
+ * AggPath to cases which only differ in the GatherPath target.
+ */
+ static void
+ hash_agg_partial_grouped_path(PlannerInfo *root, Path *path,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups, RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target)
+ {
+ Size hashaggtablesize = estimate_hashagg_tablesize(path, agg_final_costs,
+ dNumGroups);
+ Query *parse = root->parse;
+ double total_groups;
+
+ if (hashaggtablesize >= work_mem * 1024L)
+ return;
+
+ total_groups = path->rows * path->parallel_workers;
+
+ path = (Path *) create_gather_path(root,
+ grouped_rel,
+ path,
+ gather_target,
+ NULL,
+ &total_groups);
+
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ }
+
+ /*
* create_window_paths
*
* Build a new upperrel containing Paths for window-function evaluation.
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index 7361e9d..2c05028
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 114,119 ****
--- 114,120 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
*************** estimate_hash_bucket_stats(PlannerInfo *
*** 3798,3803 ****
--- 3799,3837 ----
ReleaseVariableStats(vardata);
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 199a631..e2435b6
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern void estimate_hash_bucket_stats(P
*** 210,215 ****
--- 210,219 ----
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
+
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
agg_pushdown_v4/05_base_rel.diff
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index e083961..6be6b70
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, PlanState *p
*** 723,728 ****
--- 723,759 ----
break;
}
+ case T_GroupedVar:
+
+ /*
+ * If GroupedVar appears in targetlist of Agg node, it can
+ * represent either Aggref or grouping expression.
+ */
+ if (parent && (IsA(parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * TODO Assert() that all Aggrefs of the AggState are partial:
+ * GroupedVar shouldn't find its way to the query targetlist.
+ */
+ if (IsA(gvar->gvexpr, Aggref))
+ ExecInitExprRec((Expr *) gvar->agg_partial, parent, state,
+ resv, resnull);
+ else
+ ExecInitExprRec((Expr *) gvar->gvexpr, parent, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ */
+ elog(ERROR, "parent of GroupedVar is not Agg node");
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
new file mode 100644
index 2b11835..26b00d3
*** a/src/backend/executor/nodeAgg.c
--- b/src/backend/executor/nodeAgg.c
*************** find_unaggregated_cols_walker(Node *node
*** 1864,1869 ****
--- 1864,1876 ----
/* do not descend into aggregate exprs */
return false;
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ if (IsA(gvar->gvexpr, Aggref))
+ return false;
+ }
return expression_tree_walker(node, find_unaggregated_cols_walker,
(void *) colnos);
}
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index 3cf268c..e0dc8d9
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 268,274 ****
generate_partition_wise_join_paths(root, joinrel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
--- 268,274 ----
generate_partition_wise_join_paths(root, joinrel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index e5c9233..176c568
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 487,493 ****
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! generate_gather_paths(root, rel);
/*
* Allow a plugin to editorialize on the set of Paths for this base
--- 487,496 ----
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! {
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
! }
/*
* Allow a plugin to editorialize on the set of Paths for this base
*************** static void
*** 688,693 ****
--- 691,697 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 696,710 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0), false);
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel);
/* Consider TID scans */
create_tidscan_paths(root, rel);
--- 700,736 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! add_path(rel, seq_path, false);
!
! if (rel->gpi != NULL && rel->gpi->target != NULL &&
! required_outer == NULL)
! {
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input +
! * set to be sorted.
! */
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! }
!
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel, false);
! if (rel->gpi != NULL)
! {
! /*
! * TODO Instead of calling the whole clause-matching machinery twice
! * (there should be no difference between plain and grouped paths from
! * this point of view), consider returning a separate list of paths
! * usable as grouped ones.
! */
! create_index_paths(root, rel, true);
! }
/* Consider TID scans */
create_tidscan_paths(root, rel);
*************** static void
*** 718,723 ****
--- 744,750 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
parallel_workers = compute_parallel_worker(rel, rel->pages, -1);
*************** create_plain_partial_paths(PlannerInfo *
*** 726,733 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers),
! false);
}
/*
--- 753,880 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
! add_partial_path(rel, path, false);
!
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it. Only AGG_HASHED is suitable here as it does not expect
! * the input set to be sorted.
! */
! if (rel->gpi != NULL)
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the
! * appropriate pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck().
! *
! * If "partial" is true, the resulting path is considered partial in terms of
! * parallel execution.
! *
! * The path we create here shouldn't be parameterized because of supposedly
! * high startup cost of aggregation (whether due to build of hash table for
! * AGG_HASHED strategy or due to explicit sort for AGG_SORTED).
! *
! * XXX IndexPath as an input for AGG_SORTED seems to be an exception ---
! * consider implementing parameterized AGG_SORTED unless the IndexPath is
! * partial.
! */
! void
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! List *group_clauses = NIL;
! List *group_exprs = NIL;
! List *agg_exprs = NIL;
! Path *agg_path;
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Grouped path should never be parameterized, so we're not supposed to
! * receive parameterized subpath.
! */
! Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
!
! /*
! * Non-var grouping expressions will eventually require projection using
! * Result plan, but that does not work with SRFs.
! */
! Assert(rel->gpi != NULL);
! if (rel->gpi->group_exprs != NULL)
! {
! ListCell *lc;
!
! foreach(lc, rel->gpi->group_exprs->exprs)
! {
! GroupedVar *gvar = lfirst_node(GroupedVar, lc);
!
! if (IsA(gvar->gvexpr, FuncExpr))
! {
! FuncExpr *fexpr = (FuncExpr *) gvar->gvexpr;
!
! if (fexpr->funcretset)
! return;
! }
! }
! }
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL,
! true))
! return;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost, pathkeys,
! true))
! return;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path, true);
! else
! add_partial_path(rel, (Path *) agg_path, true);
! }
}
/*
*************** set_append_rel_size(PlannerInfo *root, R
*** 1137,1142 ****
--- 1284,1365 ----
adjust_appendrel_attrs(root,
(Node *) rel->joininfo,
1, &appinfo);
+ childrel->reltarget->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) rel->reltarget->exprs,
+ 1, &appinfo);
+
+ /*
+ * Also the grouped target needs to be adjusted, if one exists.
+ */
+ if (rel->gpi != NULL)
+ {
+ GroupedPathInfo *gpi;
+
+ Assert(childrel->gpi == NULL);
+
+ /*
+ * prepare_rel_for_grouping does not set "target" for a base
+ * relation and it it clears the whole "gpi" if it can't create at
+ * least "target".
+ */
+ Assert(rel->gpi->target == NULL);
+
+ childrel->gpi = gpi = makeNode(GroupedPathInfo);
+ memcpy(gpi, rel->gpi, sizeof(GroupedPathInfo));
+
+ /*
+ * add_grouping_info_to_base_rels was not sure if grouping makes
+ * sense for the parent rel, so create a separate copy of the
+ * target now.
+ */
+ gpi->target = copy_pathtarget(childrel->gpi->target);
+ Assert(gpi->target->exprs != NIL);
+ gpi->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->target->exprs,
+ 1, &appinfo);
+
+ /*
+ * Non-var grouping expressions need to be translated as well.
+ */
+ if (gpi->group_exprs != NULL)
+ {
+ gpi->group_exprs = copy_pathtarget(gpi->group_exprs);
+ gpi->group_exprs->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->group_exprs->exprs,
+ 1, &appinfo);
+ }
+
+ /*
+ * If the output of the child rel's paths should be aggregated,
+ * sortgrouprefs are also needed. (initialize_grouped_targets
+ * should have initialized sortgrouprefs of the parent rel.)
+ */
+ childrel->reltarget->sortgrouprefs =
+ rel->reltarget->sortgrouprefs;
+ }
+
+ /*
+ * We have to make child entries in the EquivalenceClass data
+ * structures as well. This is needed either if the parent
+ * participates in some eclass joins (because we will want to consider
+ * inner-indexscan joins on the individual children) or if the parent
+ * has useful pathkeys (because we should try to build MergeAppend
+ * paths that produce those sort orderings).
+ */
+ if (rel->has_eclass_joins || has_useful_pathkeys(root, rel))
+ add_child_rel_equivalences(root, appinfo, rel, childrel);
+ childrel->has_eclass_joins = rel->has_eclass_joins;
+
+ /*
+ * Note: we could compute appropriate attr_needed data for the child's
+ * variables, by transforming the parent's attr_needed through the
+ * translated_vars mapping. However, currently there's no need
+ * because attr_needed is only examined for base relations not
+ * otherrels. So we just leave the child's attr_needed empty.
+ */
/*
* If parallelism is allowable for this query in general, see whether
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1333,1338 ****
--- 1556,1563 ----
bool subpaths_valid = true;
List *partial_subpaths = NIL;
bool partial_subpaths_valid = true;
+ List *grouped_subpaths = NIL;
+ bool grouped_subpaths_valid = true;
List *all_child_pathkeys = NIL;
List *all_child_outers = NIL;
ListCell *l;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1422,1427 ****
--- 1647,1675 ----
partial_subpaths_valid = false;
/*
+ * For grouped paths, use only the unparameterized subpaths.
+ *
+ * XXX Consider if the parameterized subpaths should be processed
+ * below. It's probably not useful for sequential scans (due to
+ * repeated aggregation), but might be worthwhile for other child
+ * nodes.
+ */
+ if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
+ {
+ Path *path;
+
+ path = (Path *) linitial(childrel->gpi->pathlist);
+ if (path->param_info == NULL)
+ grouped_subpaths = accumulate_append_subpath(grouped_subpaths,
+ path);
+ else
+ grouped_subpaths_valid = false;
+ }
+ else
+ grouped_subpaths_valid = false;
+
+
+ /*
* Collect lists of all the available path orderings and
* parameterizations for all the children. We use these as a
* heuristic to indicate which sort orderings and parameterizations we
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1493,1499 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels), false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1741,1748 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels),
! false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1520,1529 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers, partitioned_rels);
add_partial_path(rel, (Path *) appendpath, false);
}
/*
* Also build unparameterized MergeAppend paths based on the collected
* list of child pathkeys.
--- 1769,1791 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers,
! partitioned_rels);
add_partial_path(rel, (Path *) appendpath, false);
}
+ /* TODO Also partial grouped paths? */
+ if (grouped_subpaths_valid)
+ {
+ Path *path;
+
+ path = (Path *) create_append_path(rel, grouped_subpaths, NULL, 0,
+ partitioned_rels);
+ /* pathtarget will produce the grouped relation.. */
+ path->pathtarget = rel->gpi->target;
+ add_path(rel, path, true);
+ }
+
/*
* Also build unparameterized MergeAppend paths based on the collected
* list of child pathkeys.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1574,1580 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels), false);
}
}
--- 1836,1843 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels),
! false);
}
}
*************** set_subquery_pathlist(PlannerInfo *root,
*** 2026,2033 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer),
! false);
}
}
--- 2289,2295 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer), false);
}
}
*************** set_worktable_pathlist(PlannerInfo *root
*** 2298,2311 ****
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
ListCell *lc;
/* If there are no partial paths, there's nothing to do here. */
! if (rel->partial_pathlist == NIL)
return;
/*
--- 2560,2580 ----
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
+ List *pathlist = NIL;
+ PathTarget *partial_target;
ListCell *lc;
+ if (!grouped)
+ pathlist = rel->partial_pathlist;
+ else if (rel->gpi != NULL)
+ pathlist = rel->gpi->partial_pathlist;
+
/* If there are no partial paths, there's nothing to do here. */
! if (pathlist == NIL)
return;
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2313,2329 ****
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(rel->partial_pathlist);
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path, false);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach(lc, rel->partial_pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
--- 2582,2604 ----
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(pathlist);
!
! if (!grouped)
! partial_target = rel->reltarget;
! else if (rel->gpi != NULL && rel->gpi->target != NULL)
! partial_target = rel->gpi->target;
!
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, partial_target,
NULL, NULL);
! add_path(rel, simple_gather_path, grouped);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach(lc, pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
*************** generate_gather_paths(PlannerInfo *root,
*** 2331,2339 ****
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, false);
}
}
--- 2606,2614 ----
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, partial_target,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, grouped);
}
}
*************** standard_join_search(PlannerInfo *root,
*** 2505,2511 ****
generate_partition_wise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
--- 2780,2787 ----
generate_partition_wise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index 096a527..2f1d8fc
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
*************** bool enable_mergejoin = true;
*** 129,134 ****
--- 129,135 ----
bool enable_hashjoin = true;
bool enable_gathermerge = true;
bool enable_partition_wise_join = false;
+ bool enable_agg_pushdown = false;
typedef struct
{
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 1dea66c..5ad1a7f
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** static bool eclass_already_used(Equivale
*** 107,119 ****
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
--- 108,121 ----
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! bool grouped);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
*************** static Const *string_to_const(const char
*** 229,235 ****
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel)
{
List *indexpaths;
List *bitindexpaths;
--- 231,237 ----
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
List *indexpaths;
List *bitindexpaths;
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,283 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
! grouped);
/*
* Identify the join clauses that can match the index. For the moment
*************** get_join_index_paths(PlannerInfo *root,
*** 667,673 ****
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
--- 669,675 ----
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths, false);
/*
* Remember we considered paths for this set of relids. We use lcons not
*************** bms_equal_any(Relids relids, List *relid
*** 736,742 ****
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths)
{
List *indexpaths;
bool skip_nonnative_saop = false;
--- 738,744 ----
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped)
{
List *indexpaths;
bool skip_nonnative_saop = false;
*************** get_index_paths(PlannerInfo *root, RelOp
*** 754,760 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
--- 756,762 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop, grouped);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
*************** get_index_paths(PlannerInfo *root, RelOp
*** 769,775 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL));
}
/*
--- 771,777 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL, grouped));
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,797 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, false);
! if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
--- 791,799 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, grouped);
! if (!grouped && index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
*************** get_index_paths(PlannerInfo *root, RelOp
*** 802,815 ****
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
--- 804,818 ----
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (!grouped && skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 861,867 ****
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop)
{
List *result = NIL;
IndexPath *ipath;
--- 864,870 ----
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop, bool grouped)
{
List *result = NIL;
IndexPath *ipath;
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 881,892 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool can_agg_sorted;
+ List *group_clauses,
+ *group_exprs,
+ *agg_exprs;
+ AggPath *agg_path;
+ double agg_input_rows;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 891,896 ****
--- 900,908 ----
case ST_BITMAPSCAN:
if (!index->amhasgetbitmap)
return NIL;
+
+ if (grouped)
+ return NIL;
break;
case ST_ANYSCAN:
/* either or both are OK */
*************** build_index_paths(PlannerInfo *root, Rel
*** 1032,1037 ****
--- 1044,1053 ----
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
*/
+ can_agg_sorted = true;
+ group_clauses = NIL;
+ group_exprs = NIL;
+ agg_exprs = NIL;
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1064,1088 ----
outer_relids,
loop_count,
false);
! if (!grouped)
! result = lappend(result, ipath);
! else if (rel->gpi != NULL && rel->gpi->target != NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root, (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
--- 1111,1142 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL &&
! rel->gpi != NULL && rel->gpi->target != NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1164,1189 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else if (can_agg_sorted &&
! rel->gpi != NULL && rel->gpi->target != NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! Assert(agg_path != NULL);
! result = lappend(result, agg_path);
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
--- 1207,1235 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL &&
! rel->gpi != NULL && rel->gpi->target != NULL)
! {
! /*
! * TODO Double-check if this is the correct input
! * value.
! */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1244,1250 ****
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL);
result = list_concat(result, indexpaths);
}
--- 1344,1351 ----
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 4841f9a..25aa1aa
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
*************** add_grouping_info_to_base_rels(PlannerIn
*** 259,264 ****
--- 259,270 ----
int i;
ListCell *lc;
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
/* No grouping in the query? */
if (!root->parse->groupClause)
return;
*************** create_grouping_expr_grouped_var_infos(P
*** 599,605 ****
}
}
-
/*
* Check if rel->reltarget allows the relation to be grouped and initialize
* the grouping target(s).
--- 605,610 ----
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 51d13ab..67579cd
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,230 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual),
! false);
}
/*
--- 223,229 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual), false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index c1939a1..2bf4c8b
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,90 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,89 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals), false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index f78e22f..84abaee
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** inheritance_planner(PlannerInfo *root)
*** 1520,1527 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)),
! false);
}
/*--------------------
--- 1520,1526 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)), false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 1930,1936 ****
newpath = (Path *) create_projection_path(root,
current_rel,
subpath,
! scanjoin_target);
lfirst(lc) = newpath;
}
}
--- 1929,1936 ----
newpath = (Path *) create_projection_path(root,
current_rel,
subpath,
! scanjoin_target,
! false);
lfirst(lc) = newpath;
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3578,3585 ****
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *grouped_rel;
PathTarget *partial_grouping_target = NULL;
! AggClauseCosts agg_partial_costs; /* parallel only */
! AggClauseCosts agg_final_costs; /* parallel only */
Size hashaggtablesize;
double dNumGroups;
double dNumPartialGroups = 0;
--- 3578,3585 ----
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *grouped_rel;
PathTarget *partial_grouping_target = NULL;
! AggClauseCosts agg_final_costs;
! bool agg_final_costs_known = false;
Size hashaggtablesize;
double dNumGroups;
double dNumPartialGroups = 0;
*************** create_grouping_paths(PlannerInfo *root,
*** 3761,3766 ****
--- 3761,3773 ----
}
/*
+ * Collect statistics about aggregates for estimating costs of performing
+ * aggregation in parallel.
+ */
+ MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
+ agg_final_costs_known = false;
+
+ /*
* Before generating paths for grouped_rel, we first generate any possible
* partial paths; that way, later code can easily consider both parallel
* and non-parallel approaches to grouping. Note that the partial paths
*************** create_grouping_paths(PlannerInfo *root,
*** 3771,3776 ****
--- 3778,3786 ----
if (try_parallel_aggregation)
{
Path *cheapest_partial_path = linitial(input_rel->partial_pathlist);
+ AggClauseCosts agg_partial_costs;
+
+ MemSet(&agg_partial_costs, 0, sizeof(AggClauseCosts));
/*
* Build target list for partial aggregate paths. These paths cannot
*************** create_grouping_paths(PlannerInfo *root,
*** 3786,3811 ****
cheapest_partial_path->rows,
gd);
- /*
- * Collect statistics about aggregates for estimating costs of
- * performing aggregation in parallel.
- */
- MemSet(&agg_partial_costs, 0, sizeof(AggClauseCosts));
- MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
if (parse->hasAggs)
{
/* partial phase */
get_agg_clause_costs(root, (Node *) partial_grouping_target->exprs,
AGGSPLIT_INITIAL_SERIAL,
&agg_partial_costs);
-
- /* final phase */
- get_agg_clause_costs(root, (Node *) target->exprs,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
- get_agg_clause_costs(root, parse->havingQual,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
}
if (can_sort)
--- 3796,3807 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3887,3893 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups), false);
}
}
}
--- 3883,3890 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3895,3911 ****
/* Build final grouping paths */
if (can_sort)
{
/*
* Use any available suitably-sorted path as input, and also consider
* sorting the cheapest-total path.
*/
! foreach(lc, input_rel->pathlist)
{
Path *path = (Path *) lfirst(lc);
bool is_sorted;
is_sorted = pathkeys_contained_in(root->group_pathkeys,
path->pathkeys);
if (path == cheapest_path || is_sorted)
{
/* Sort the cheapest-total path if it isn't already sorted */
--- 3892,3927 ----
/* Build final grouping paths */
if (can_sort)
{
+ List *pathlist = input_rel->pathlist;
+ int nplain = list_length(input_rel->pathlist);
+ int i;
+
+ /*
+ * The grouped paths created out of grouped base relations or joins
+ * can be treated almost identically here (the major difference is
+ * that their targetlists contain GroupedVars instead aggregate input
+ * vars, but this is handled by using the appropriate value of
+ * AggSplit), so process them in the same loop.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL)
+ pathlist = list_concat(list_copy(pathlist),
+ input_rel->gpi->pathlist);
+
/*
* Use any available suitably-sorted path as input, and also consider
* sorting the cheapest-total path.
*/
! i = 0;
! foreach(lc, pathlist)
{
Path *path = (Path *) lfirst(lc);
bool is_sorted;
+ bool is_grouped;
is_sorted = pathkeys_contained_in(root->group_pathkeys,
path->pathkeys);
+ is_grouped = i >= nplain;
+
if (path == cheapest_path || is_sorted)
{
/* Sort the cheapest-total path if it isn't already sorted */
*************** create_grouping_paths(PlannerInfo *root,
*** 3919,3930 ****
--- 3935,3973 ----
/* Now decide what to stick atop it */
if (parse->groupingSets)
{
+ /*
+ * No grouping should have taken place at base relation /
+ * join level, i.e. pathlist should be equal to
+ * input_rel->pathlist.
+ */
+ Assert(!is_grouped);
+
consider_groupingsets_paths(root, grouped_rel,
path, true, can_hash, target,
gd, agg_costs, dNumGroups);
}
else if (parse->hasAggs)
{
+ AggStrategy aggstrategy;
+ AggSplit aggsplit;
+
+ if (!is_grouped)
+ {
+ aggstrategy = parse->groupClause ? AGG_SORTED : AGG_PLAIN;
+ aggsplit = AGGSPLIT_SIMPLE;
+ }
+ else
+ {
+ /*
+ * Like above, no grouping of base relation is not
+ * possible w/o this.
+ */
+ Assert(parse->groupClause);
+
+ aggstrategy = AGG_SORTED;
+ aggsplit = AGGSPLIT_FINAL_DESERIAL;
+ }
+
/*
* We have aggregation, possibly with plain GROUP BY. Make
* an AggPath.
*************** create_grouping_paths(PlannerInfo *root,
*** 3934,3941 ****
grouped_rel,
path,
target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_SIMPLE,
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
--- 3977,3984 ----
grouped_rel,
path,
target,
! aggstrategy,
! aggsplit,
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
*************** create_grouping_paths(PlannerInfo *root,
*** 3962,3972 ****
Assert(false);
}
}
}
/*
! * Now generate a complete GroupAgg Path atop of the cheapest partial
! * path. We can do this using either Gather or Gather Merge.
*/
if (grouped_rel->partial_pathlist)
sort_agg_partial_grouped_paths(root,
--- 4005,4034 ----
Assert(false);
}
}
+
+ i++;
}
/*
! * Compute agg_final_costs iff the aggregation should take place in 2
! * steps.
! */
! if (parse->hasAggs &&
! (grouped_rel->partial_pathlist ||
! (input_rel->gpi != NULL && input_rel->gpi->partial_pathlist)))
! {
! get_agg_clause_costs(root, (Node *) target->exprs,
! AGGSPLIT_FINAL_DESERIAL,
! &agg_final_costs);
! get_agg_clause_costs(root, parse->havingQual,
! AGGSPLIT_FINAL_DESERIAL,
! &agg_final_costs);
!
! agg_final_costs_known = true;
! }
!
! /*
! * Gather grouped partial paths and apply AGG_SORTED to them.
*/
if (grouped_rel->partial_pathlist)
sort_agg_partial_grouped_paths(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3974,3979 ****
--- 4036,4054 ----
&agg_final_costs,
dNumGroups, grouped_rel,
partial_grouping_target, target);
+
+ /*
+ * The same for the grouped partial paths involving base relation /
+ * join grouping.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->target != NULL &&
+ input_rel->gpi->partial_pathlist)
+ sort_agg_partial_grouped_paths(root,
+ input_rel->gpi->partial_pathlist,
+ &agg_final_costs, dNumGroups,
+ grouped_rel,
+ input_rel->gpi->target,
+ target);
}
if (can_hash)
*************** create_grouping_paths(PlannerInfo *root,
*** 4020,4025 ****
--- 4095,4116 ----
}
/*
+ * Compute agg_final_costs if needed below and if not done above.
+ */
+ if (parse->hasAggs && !agg_final_costs_known &&
+ (grouped_rel->partial_pathlist ||
+ (input_rel->gpi != NULL &&
+ (input_rel->gpi->pathlist || input_rel->gpi->partial_pathlist))))
+ {
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ }
+
+ /*
* Generate a HashAgg Path atop of the cheapest partial path. Once
* again, we'll only do this if it looks as though the hash table
* won't exceed work_mem.
*************** create_grouping_paths(PlannerInfo *root,
*** 4032,4037 ****
--- 4123,4175 ----
dNumGroups, grouped_rel,
partial_grouping_target, target);
}
+
+ /*
+ * If input_rel has partially aggregated paths, perform the final
+ * aggregation.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->pathlist);
+
+ hashaggtablesize = estimate_hashagg_tablesize(path,
+ &agg_final_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * The top-level grouped_rel needs to receive the path into
+ * regular pathlist, as opposed grouped_rel->gpi->pathlist. So
+ * pass FALSE for grouped.
+ */
+ add_path(grouped_rel,
+ (Path *) create_agg_path(root, grouped_rel,
+ path,
+ target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ &agg_final_costs,
+ dNumGroups),
+ false);
+ }
+ }
+
+ /*
+ * If input_rel has partially aggregated partial paths, gather them
+ * and perform the final aggregation.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->target != NULL &&
+ input_rel->gpi->partial_pathlist != NIL)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->partial_pathlist);
+
+ hash_agg_partial_grouped_path(root, path, &agg_final_costs, dNumGroups,
+ grouped_rel, input_rel->gpi->target,
+ target);
+ }
}
/* Give a helpful error if we failed to find any implementation */
*************** create_distinct_paths(PlannerInfo *root,
*** 4864,4870 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
--- 5002,5009 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows),
! false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4891,4897 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
--- 5030,5037 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows),
! false);
}
/*
*************** adjust_paths_for_srfs(PlannerInfo *root,
*** 6004,6010 ****
newpath = (Path *) create_projection_path(root,
rel,
newpath,
! thistarget);
}
}
lfirst(lc) = newpath;
--- 6144,6151 ----
newpath = (Path *) create_projection_path(root,
rel,
newpath,
! thistarget,
! false);
}
}
lfirst(lc) = newpath;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index fa9a3f0..aab2430
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
bool has_conv_whole_rows; /* are there ConvertRowtypeExpr
* entries encapsulating a whole-row
*************** set_upper_references(PlannerInfo *root,
*** 1735,1743 ****
--- 1736,1798 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the current
+ * subplan->targetlist.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ else if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /*
+ * Partial aggregation node can have GroupedVar's on the input
+ * if those represent generic (non-Var) grouping expressions.
+ * Unlike above, the restored expressions should stay there.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1947,1952 ****
--- 2002,2008 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
itlist->has_conv_whole_rows = false;
*************** build_tlist_index(List *tlist)
*** 1967,1972 ****
--- 2023,2030 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else if (is_converted_whole_row_reference((Node *) tle->expr))
itlist->has_conv_whole_rows = true;
else
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2250,2255 ****
--- 2308,2338 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2412,2418 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
--- 2495,2502 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 3e0c313..cba5c4a
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 26,31 ****
--- 26,32 ----
#include "optimizer/planmain.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if get_grouping_expressions ends up in another module. */
#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
*************** static List *translate_sub_tlist(List *t
*** 54,60 ****
static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
!
/*****************************************************************************
* MISC. PATH UTILITIES
--- 55,63 ----
static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
! static Path *add_grouping_expressions_to_subpath(PlannerInfo *root,
! Path *subpath,
! PathTarget *group_exprs);
/*****************************************************************************
* MISC. PATH UTILITIES
*************** add_partial_path_precheck(RelOptInfo *pa
*** 971,977 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, false))
return false;
return true;
--- 974,980 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, grouped))
return false;
return true;
*************** create_unique_path(PlannerInfo *root, Re
*** 1486,1493 ****
MemoryContext oldcontext;
int numCols;
! /* Caller made a mistake if subpath isn't cheapest_total ... */
! Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
Assert(sjinfo->jointype == JOIN_SEMI);
--- 1489,1500 ----
MemoryContext oldcontext;
int numCols;
! /*
! * Caller made a mistake if subpath isn't cheapest_total (or the cheapest
! * grouped).
! */
! Assert(subpath == rel->cheapest_total_path ||
! (rel->gpi != NULL && subpath == linitial(rel->gpi->pathlist)));
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
Assert(sjinfo->jointype == JOIN_SEMI);
*************** create_hashjoin_path(PlannerInfo *root,
*** 2327,2338 ****
* 'rel' is the parent relation associated with the result
* 'subpath' is the path representing the source of data
* 'target' is the PathTarget to be computed
*/
ProjectionPath *
create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target)
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget = subpath->pathtarget;
--- 2334,2347 ----
* 'rel' is the parent relation associated with the result
* 'subpath' is the path representing the source of data
* 'target' is the PathTarget to be computed
+ * 'force_result' enforces implementation using Result plan.
*/
ProjectionPath *
create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target,
! bool force_result)
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget = subpath->pathtarget;
*************** create_projection_path(PlannerInfo *root
*** 2351,2356 ****
--- 2360,2366 ----
pathnode->path.pathkeys = subpath->pathkeys;
pathnode->subpath = subpath;
+ pathnode->force_result = force_result;
/*
* We might not need a separate Result node. If the input plan node type
*************** create_projection_path(PlannerInfo *root
*** 2361,2368 ****
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs))
{
/* No separate Result node needed */
pathnode->dummypp = true;
--- 2371,2379 ----
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (!force_result &&
! (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
*************** apply_projection_to_path(PlannerInfo *ro
*** 2432,2438 ****
* separate ProjectionPath.
*/
if (!is_projection_capable_path(path))
! return (Path *) create_projection_path(root, rel, path, target);
/*
* We can just jam the desired tlist into the existing path, being sure to
--- 2443,2450 ----
* separate ProjectionPath.
*/
if (!is_projection_capable_path(path))
! return (Path *) create_projection_path(root, rel, path, target,
! false);
/*
* We can just jam the desired tlist into the existing path, being sure to
*************** apply_projection_to_path(PlannerInfo *ro
*** 2470,2476 ****
create_projection_path(root,
gpath->subpath->parent,
gpath->subpath,
! target);
}
else if (path->parallel_safe &&
!is_parallel_safe(root, (Node *) target->exprs))
--- 2482,2489 ----
create_projection_path(root,
gpath->subpath->parent,
gpath->subpath,
! target,
! false);
}
else if (path->parallel_safe &&
!is_parallel_safe(root, (Node *) target->exprs))
*************** create_agg_path(PlannerInfo *root,
*** 2766,2771 ****
--- 2779,2969 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * first_call indicates whether the function is being called first time for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * group_clauses, group_exprs and agg_exprs are pointers to lists we populate
+ * when called first time for particular index, and that user passes for
+ * subsequent calls.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+ Assert(rel->gpi->target != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /* Add generic grouping expressions to the subpath if there are some. */
+ if (rel->gpi->group_exprs != NULL)
+ subpath = add_grouping_expressions_to_subpath(root, subpath,
+ rel->gpi->group_exprs);
+
+ if (first_call)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ if (first_call)
+ get_grouping_expressions(root, rel->gpi->target,
+ rel->gpi->sortgroupclauses, group_clauses,
+ group_exprs, agg_exprs);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows, NULL);
+
+ /* TODO HAVING qual. */
+ Assert(*group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, rel->gpi->target,
+ AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL, &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Appy partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ *
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+ Assert(rel->gpi->target != NULL);
+
+ /* Add generic grouping expressions to the subpath if there are some. */
+ if (rel->gpi->group_exprs != NULL)
+ subpath = add_grouping_expressions_to_subpath(root, subpath,
+ rel->gpi->group_exprs);
+
+ if (first_call)
+ {
+ /*
+ * Find one grouping clause per grouping column.
+ *
+ * All that create_agg_plan eventually needs of the clause is
+ * tleSortGroupRef, so we don't have to care that the clause
+ * expression might differ from texpr, in case texpr was derived from
+ * EC.
+ */
+ get_grouping_expressions(root, rel->gpi->target,
+ rel->gpi->sortgroupclauses,
+ group_clauses,
+ group_exprs, agg_exprs);
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows,
+ NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(*group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ rel->gpi->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
*************** reparameterize_pathlist_by_child(Planner
*** 3838,3840 ****
--- 4036,4079 ----
return result;
}
+
+ /*
+ * Add (non-Var) grouping expressions contained in group_exprs target to a
+ * subpath. The passed in subpath should not be touched so that it can still
+ * be used as a subpath for non-grouped paths. Therefore we wrap it into
+ * ResultPath, to which we actually add those expressions.
+ */
+ static Path *
+ add_grouping_expressions_to_subpath(PlannerInfo *root, Path *subpath,
+ PathTarget *group_exprs)
+ {
+ PathTarget *target;
+ ListCell *lc;
+ int i;
+
+ /*
+ * The subpath must stay intact because, besides producing input data for
+ * aggregation, it can participate in a plan that does not use aggregation
+ * push-down. For the same reason we pass force_result=true to
+ * create_projection_path below.
+ */
+ target = copy_pathtarget(subpath->pathtarget);
+
+ Assert(group_exprs->sortgrouprefs != NULL);
+ i = 0;
+ foreach(lc, group_exprs->exprs)
+ {
+ Index sortgroupref;
+ GroupedVar *gvar;
+
+ sortgroupref = group_exprs->sortgrouprefs[i++];
+ gvar = lfirst_node(GroupedVar, lc);
+ add_column_to_pathtarget(target, (Expr *) gvar, sortgroupref);
+ }
+
+ return (Path *) create_projection_path(root,
+ subpath->parent,
+ subpath,
+ target,
+ true);
+ }
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 167e7a8..2197ac4
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 27,32 ****
--- 27,33 ----
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
#include "parser/parse_oper.h"
#include "utils/hsearch.h"
*************** build_simple_rel(PlannerInfo *root, int
*** 126,131 ****
--- 127,133 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->gpi = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index abdc5d9..3445c98
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,413 ****
--- 408,501 ----
return result;
}
+ /*
+ * get_sortgrouplist_clauses
+ *
+ * Given a "grouped target" (i.e. target where each non-GroupedVar
+ * element must have sortgroupref set), build a list of the referencing
+ * SortGroupClauses, a list of the corresponding grouping expressions and
+ * a list of aggregate expressions.
+ *
+ * target_group_clauses can contain a list of target-specific
+ * SortGroupClause's, which correspond to grouping expressions possibly
+ * added to the target (i.e. w/o being present in
+ * root->query->groupClause, see initialize_grouped_targets for
+ * details.).
+ */
+ /* Refine the function name. */
+ void
+ get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List *target_group_clauses,
+ List **grouping_clauses, List **grouping_exprs,
+ List **agg_exprs)
+ {
+ ListCell *l;
+ int i = 0;
+
+ /* The target should contain at least one grouping column. */
+ Assert(target->sortgrouprefs != NULL);
+
+ foreach(l, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(l);
+
+ if (IsA(texpr, GroupedVar) &&
+ IsA(((GroupedVar *) texpr)->gvexpr, Aggref))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ cl = get_sortgroupref_clause_noerr(sortgroupref, root->parse->groupClause);
+
+ /*
+ * If query does not have this clause, it must be target-specific.
+ */
+ if (cl == NULL)
+ cl = get_sortgroupref_clause(sortgroupref, target_group_clauses);
+
+ *grouping_clauses = list_append_unique(*grouping_clauses, cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ *grouping_exprs = list_append_unique(*grouping_exprs, texpr);
+ }
+
+ /* Now collect the aggregates. */
+ while (l != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(l));
+
+ /* Currently, GroupedVarInfo can only represent aggregate. */
+ Assert(gvar->agg_partial != NULL);
+ *agg_exprs = lappend(*agg_exprs, gvar->agg_partial);
+ l = lnext(l);
+ }
+ }
+
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 871,927 ----
}
/*
+ * Replace each "grouped var" in the source targetlist with the original
+ * expression.
+ *
+ * TODO Think of more suitable name undo_grouped_var_substitutions? Also note
+ * that the partial aggregate is retrieved, not the original one.
+ */
+ List *
+ restore_grouping_expressions(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te,
+ *te_new;
+ Expr *expr_new = NULL;
+
+ te = lfirst_node(TargetEntry, l);
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Partial aggregate should appear in the targetlist so that
+ * it looks as if convert_combining_aggrefs arranged it.
+ */
+ expr_new = (Expr *) gvar->agg_partial;
+ }
+ else
+ expr_new = gvar->gvexpr;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
* For each aggregate add GroupedVar to the grouped target.
*
* Caller passes the aggregates in the form of GroupedVarInfos so that we
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index cc6cec7..f8cab99
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7637,7642 ****
--- 7637,7659 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+ Expr *expr = gvar->gvexpr;
+
+ if (IsA(expr, Aggref))
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ else if (IsA(expr, Var))
+ (void) get_variable((Var *) expr, 0, false, context);
+ else
+ {
+ Assert(IsA(gvar->gvexpr, OpExpr));
+ get_oper_expr((OpExpr *) expr, context);
+ }
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 9113,9122 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 9130,9147 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index ca76b0d..e411239
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_bool ConfigureNames
*** 921,926 ****
--- 921,936 ----
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"geqo", PGC_USERSET, QUERY_TUNING_GEQO,
gettext_noop("Enables genetic query optimization."),
gettext_noop("This algorithm attempts to do planning without "
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index a18ee93..e270020
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct HashPath
*** 1527,1538 ****
--- 1527,1542 ----
* ProjectionPath node, which is marked dummy to indicate that we intend to
* assign the work to the input plan node. The estimated cost for the
* ProjectionPath node will account for whether a Result will be used or not.
+ *
+ * force_result field tells that the Result node must be used for some reason
+ * even though the subpath could normally handle the projection.
*/
typedef struct ProjectionPath
{
Path path;
Path *subpath; /* path representing input source */
bool dummypp; /* true if no separate Result is needed */
+ bool force_result; /* Is Result node required? */
} ProjectionPath;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
new file mode 100644
index 306d923..8424d3a
*** a/src/include/optimizer/cost.h
--- b/src/include/optimizer/cost.h
*************** extern bool enable_mergejoin;
*** 68,73 ****
--- 68,74 ----
extern bool enable_hashjoin;
extern bool enable_gathermerge;
extern bool enable_partition_wise_join;
+ extern bool enable_agg_pushdown;
extern int constraint_exclusion;
extern double clamp_row_est(double nrows);
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 5e7e97a..d454df3
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern HashPath *create_hashjoin_path(Pl
*** 159,165 ****
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target);
extern Path *apply_projection_to_path(PlannerInfo *root,
RelOptInfo *rel,
Path *path,
--- 159,166 ----
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target,
! bool force_result);
extern Path *apply_projection_to_path(PlannerInfo *root,
RelOptInfo *rel,
Path *path,
*************** extern AggPath *create_agg_path(PlannerI
*** 195,200 ****
--- 196,215 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index ea886b6..60e6ecd
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void set_dummy_rel_pathlist(RelOp
*** 53,59 ****
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
--- 53,64 ----
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
!
! extern void create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
! Path *subpath, bool precheck, bool partial,
! AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
*************** extern void debug_print_rel(PlannerInfo
*** 69,75 ****
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
--- 74,81 ----
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index a725818..29d1b46
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern void split_pathtarget_at_srfs(Pla
*** 69,74 ****
--- 69,77 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
new file mode 100644
index cd1f7f3..3913847
*** a/src/test/regress/expected/sysviews.out
--- b/src/test/regress/expected/sysviews.out
*************** select count(*) >= 0 as ok from pg_prepa
*** 72,77 ****
--- 72,78 ----
select name, setting from pg_settings where name like 'enable%';
name | setting
----------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
*************** select name, setting from pg_settings wh
*** 85,91 ****
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (13 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
--- 86,92 ----
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (14 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
agg_pushdown_v4/06_join.diff
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index e0dc8d9..30ac459
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 269,274 ****
--- 269,275 ----
/* Create GatherPaths for any useful partial paths for rel */
generate_gather_paths(root, joinrel, false);
+ generate_gather_paths(root, joinrel, true);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 176c568..a125c57
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** static void set_append_rel_pathlist(Plan
*** 97,103 ****
static void generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels);
static Path *get_cheapest_parameterized_child_path(PlannerInfo *root,
RelOptInfo *rel,
Relids required_outer);
--- 97,104 ----
static void generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels,
! bool grouped);
static Path *get_cheapest_parameterized_child_path(PlannerInfo *root,
RelOptInfo *rel,
Relids required_outer);
*************** set_append_rel_size(PlannerInfo *root, R
*** 1290,1345 ****
1, &appinfo);
/*
! * Also the grouped target needs to be adjusted, if one exists.
*/
if (rel->gpi != NULL)
! {
! GroupedPathInfo *gpi;
!
! Assert(childrel->gpi == NULL);
!
! /*
! * prepare_rel_for_grouping does not set "target" for a base
! * relation and it it clears the whole "gpi" if it can't create at
! * least "target".
! */
! Assert(rel->gpi->target == NULL);
!
! childrel->gpi = gpi = makeNode(GroupedPathInfo);
! memcpy(gpi, rel->gpi, sizeof(GroupedPathInfo));
!
! /*
! * add_grouping_info_to_base_rels was not sure if grouping makes
! * sense for the parent rel, so create a separate copy of the
! * target now.
! */
! gpi->target = copy_pathtarget(childrel->gpi->target);
! Assert(gpi->target->exprs != NIL);
! gpi->target->exprs = (List *)
! adjust_appendrel_attrs(root,
! (Node *) gpi->target->exprs,
! 1, &appinfo);
!
! /*
! * Non-var grouping expressions need to be translated as well.
! */
! if (gpi->group_exprs != NULL)
! {
! gpi->group_exprs = copy_pathtarget(gpi->group_exprs);
! gpi->group_exprs->exprs = (List *)
! adjust_appendrel_attrs(root,
! (Node *) gpi->group_exprs->exprs,
! 1, &appinfo);
! }
!
! /*
! * If the output of the child rel's paths should be aggregated,
! * sortgrouprefs are also needed. (initialize_grouped_targets
! * should have initialized sortgrouprefs of the parent rel.)
! */
! childrel->reltarget->sortgrouprefs =
! rel->reltarget->sortgrouprefs;
! }
/*
* We have to make child entries in the EquivalenceClass data
--- 1291,1301 ----
1, &appinfo);
/*
! * Setup GroupedPathInfo for the child relation if the parent has
! * some.
*/
if (rel->gpi != NULL)
! build_chiid_rel_gpi(root, childrel, rel, 1, &appinfo);
/*
* We have to make child entries in the EquivalenceClass data
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1793,1799 ****
if (subpaths_valid)
generate_mergeappend_paths(root, rel, live_childrels,
all_child_pathkeys,
! partitioned_rels);
/*
* Build Append paths for each parameterization seen among the child rels.
--- 1749,1759 ----
if (subpaths_valid)
generate_mergeappend_paths(root, rel, live_childrels,
all_child_pathkeys,
! partitioned_rels, false);
! if (grouped_subpaths_valid)
! generate_mergeappend_paths(root, rel, live_childrels,
! all_child_pathkeys,
! partitioned_rels, true);
/*
* Build Append paths for each parameterization seen among the child rels.
*************** static void
*** 1868,1876 ****
generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels)
{
ListCell *lcp;
foreach(lcp, all_child_pathkeys)
{
--- 1828,1838 ----
generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels,
! bool grouped)
{
ListCell *lcp;
+ PathTarget *target = NULL;
foreach(lcp, all_child_pathkeys)
{
*************** generate_mergeappend_paths(PlannerInfo *
*** 1879,1901 ****
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
Path *cheapest_startup,
*cheapest_total;
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
TOTAL_COST,
--- 1841,1872 ----
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
+ Path *path;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
+ List *pathlist;
Path *cheapest_startup,
*cheapest_total;
+ if (grouped &&
+ (childrel->gpi == NULL || childrel->gpi->pathlist == NIL))
+ return;
+
+ pathlist = !grouped ? childrel->pathlist :
+ childrel->gpi->pathlist;
+
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
TOTAL_COST,
*************** generate_mergeappend_paths(PlannerInfo *
*** 1927,1948 ****
accumulate_append_subpath(total_subpaths, cheapest_total);
}
/* ... and build the MergeAppend paths */
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels),
! false);
}
}
--- 1898,1944 ----
accumulate_append_subpath(total_subpaths, cheapest_total);
}
+ /*
+ * Special target is needed if the path output should be grouped.
+ */
+ if (grouped)
+ {
+ Assert(rel->gpi != NULL);
+ target = rel->gpi->target;
+ }
+
/* ... and build the MergeAppend paths */
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
!
! /* pathtarget will produce the grouped relation.. */
! if (grouped)
! {
! Assert(rel->gpi != NULL && rel->gpi->target != NULL);
! path->pathtarget = rel->gpi->target;
! }
!
! add_path(rel, path, grouped);
!
if (startup_neq_total)
! {
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
! if (grouped)
! path->pathtarget = rel->gpi->target;
! add_path(rel, path, grouped);
! }
!
}
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index ff784b5..f9c2fb1
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***************
*** 22,27 ****
--- 22,28 ----
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
+ #include "optimizer/tlist.h"
/* Hook for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
*************** set_join_pathlist_hook_type set_join_pat
*** 39,44 ****
--- 40,49 ----
#define PATH_PARAM_BY_REL(path, rel) \
(PATH_PARAM_BY_REL_SELF(path, rel) || PATH_PARAM_BY_PARENT(path, rel))
+ #define REL_HAS_GROUPED_PATHS(rel) ((rel)->gpi && (rel)->gpi->pathlist)
+ #define REL_HAS_PARTIAL_GROUPED_PATHS(rel) \
+ ((rel)->gpi && (rel)->gpi->partial_pathlist)
+
static void try_partial_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
Path *outer_path,
*************** static void try_partial_mergejoin_path(P
*** 48,57 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 53,73 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
+ static void sort_inner_and_outer_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ bool grouped_outer,
+ bool grouped_inner,
+ bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** static void consider_parallel_nestloop(P
*** 60,73 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 76,92 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped_outer,
! bool do_aggregate);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** static void generate_mergejoin_paths(Pla
*** 87,93 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 106,115 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
/*
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 302,309 ****
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
! hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 324,331 ----
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
! hash_inner_and_outer(root, joinrel, outerrel, innerrel, jointype,
! &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 364,370 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
--- 386,394 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
*************** try_nestloop_path(PlannerInfo *root,
*** 374,379 ****
--- 398,417 ----
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi || !grouped);
+
+ /*
+ * Aggregation target is necessary to produce grouped join output.
+ */
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* Paths are parameterized by top-level parents, so run parameterization
*************** try_nestloop_path(PlannerInfo *root,
*** 420,461 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
/*
! * If the inner path is parameterized, it is parameterized by the
! * topmost parent of the outer rel, not the outer rel itself. Fix
! * that.
*/
! if (PATH_PARAM_BY_PARENT(inner_path, outer_path->parent))
{
! inner_path = reparameterize_path_by_child(root, inner_path,
! outer_path->parent);
!
! /*
! * If we could not translate the path, we can't create nest loop
! * path.
! */
! if (!inner_path)
! {
! bms_free(required_outer);
! return;
! }
}
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer), false);
}
else
{
--- 458,512 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! /*
! * If the inner path is parameterized, it is parameterized by the topmost
! * parent of the outer rel, not the outer rel itself. Fix that.
! */
! if (PATH_PARAM_BY_PARENT(inner_path, outer_path->parent))
{
+ inner_path = reparameterize_path_by_child(root, inner_path,
+ outer_path->parent);
+
/*
! * If we could not translate the path, we can't create nest loop path.
*/
! if (!inner_path)
{
! bms_free(required_outer);
! return;
}
+ }
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! required_outer, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate && required_outer == NULL)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
! {
! add_path(joinrel, join_path, grouped);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 476,484 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 527,545 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_nestloop_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 513,520 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/*
--- 574,635 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! NULL, join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_nestloop_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
! return;
!
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target == NULL)
return;
/*
*************** try_partial_nestloop_path(PlannerInfo *r
*** 533,550 ****
return;
}
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL), false);
}
/*
--- 648,722 ----
return;
}
! if (!partial)
! try_nestloop_path(root, joinrel, outer_path, inner_path, pathkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_nestloop_path(root, joinrel, outer_path, inner_path,
! pathkeys, jointype, extra, true,
! do_aggregate);
! }
!
! static void
! try_nestloop_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! grouped_join,
! do_aggregate);
! else
! try_partial_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! grouped_join,
! do_aggregate);
! }
! else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! partial,
! do_aggregate);
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 563,600 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
! if (is_partial)
! {
! try_partial_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra);
! return;
! }
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 735,773 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
! /* Caller should not request aggregation w/o grouped output. */
! Assert(!do_aggregate || grouped);
!
! /* GroupedPathInfo is necessary for us to produce a grouped set. */
! Assert(joinrel->gpi || !grouped);
!
! /*
! * Aggregation target is necessary to produce grouped join output.
! */
! Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
! !do_aggregate);
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (!bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 613,639 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys), false);
}
else
{
--- 786,832 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
{
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
! {
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 657,665 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 850,868 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_mergejoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 689,716 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys), false);
}
/*
--- 892,1053 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_mergejoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target == NULL)
! return;
!
! if (!partial)
! try_mergejoin_path(root, joinrel, outer_path, inner_path, pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_mergejoin_path(root, joinrel, outer_path, inner_path,
! pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! }
!
! /*
! * Generic front-end to create combinations of full/partial and grouped/plain
! * merge joins.
! */
! static void
! try_mergejoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! false, false);
! else
! try_partial_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! false, false);
! }
! else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! do_aggregate);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 725,771 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, false))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! required_outer,
! hashclauses), false);
}
else
{
--- 1062,1146 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi || !grouped);
+
+ /*
+ * Aggregation target is necessary to produce grouped join output.
+ */
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* Check to see if proposed path is still parameterized, and reject if the
* parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (!bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
+ *
+ * TODO Need to consider aggregation here?
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist,
! required_outer, hashclauses,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
{
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, grouped))
! {
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 786,794 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1161,1179 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_hashjoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 811,833 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL, false))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! NULL,
! hashclauses), false);
}
/*
* clause_sides_match_join
* Determine whether a join clause is of the right form to use in this join.
--- 1196,1349 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist, NULL,
! hashclauses, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! NIL, grouped))
! {
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! /*
! * Create a new grouped hash join path by joining a grouped path to plain
! * (non-grouped) one, or by joining 2 plain relations and applying grouping on
! * the result.
! *
! * Joining of 2 grouped paths is not supported. If a grouped relation A was
! * joined to grouped relation B, then the grouping of B reduces the number of
! * times each group of A is appears in the join output. This makes difference
! * for some aggregates, e.g. sum().
! *
! * If do_aggregate is true, neither input rel is grouped so we need to
! * aggregate the join result explicitly.
! *
! * partial argument tells whether the join path should be considered partial.
! */
! static void
! try_grouped_hashjoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate
! )
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target == NULL)
! return;
!
! if (!partial)
! try_hashjoin_path(root, joinrel, outer_path, inner_path, hashclauses,
! jointype, extra, true, do_aggregate);
! else
! try_partial_hashjoin_path(root, joinrel, outer_path, inner_path,
! hashclauses, jointype, extra, true,
! do_aggregate);
! }
!
! /*
! * Generic front-end to create combinations of full/partial and grouped/plain
! * hash joins.
! */
! static void
! try_hashjoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! false, false);
! else
! try_partial_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! false, false);
! }
! else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! partial,
! do_aggregate);
! }
}
+
/*
* clause_sides_match_join
* Determine whether a join clause is of the right form to use in this join.
*************** sort_inner_and_outer(PlannerInfo *root,
*** 878,883 ****
--- 1394,1427 ----
JoinType jointype,
JoinPathExtraData *extra)
{
+ /* Plain (non-grouped) join. */
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, false);
+
+ /* Use all the supported strategies to generate grouped join. */
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, true, false, false);
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, true, false);
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, true);
+ }
+
+ /*
+ * TODO As merge_pathkeys shouldn't differ across calls, use a separate
+ * function to derive them and pass them here in a list.
+ */
+ static void
+ sort_inner_and_outer_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ bool grouped_outer,
+ bool grouped_inner,
+ bool do_aggregate)
+ {
JoinType save_jointype = jointype;
Path *outer_path;
Path *inner_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 899,906 ****
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! outer_path = outerrel->cheapest_total_path;
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 1443,1467 ----
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! if (grouped_outer)
! {
! if (REL_HAS_GROUPED_PATHS(outerrel))
! outer_path = linitial(outerrel->gpi->pathlist);
! else
! return;
! }
! else
! outer_path = outerrel->cheapest_total_path;
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** sort_inner_and_outer(PlannerInfo *root,
*** 918,923 ****
--- 1479,1494 ----
*/
if (jointype == JOIN_UNIQUE_OUTER)
{
+ /*
+ * TODO This is just a temporary limitation. Before lifting it, make
+ * sure that the UniquePath does emit GroupedVars. Also try to avoid
+ * the unique-ification if outer_path comes directly from AggPath
+ * (i.e. it's not grouped path combined with plain one) and the
+ * grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_outer)
+ return;
+
outer_path = (Path *) create_unique_path(root, outerrel,
outer_path, extra->sjinfo);
Assert(outer_path);
*************** sort_inner_and_outer(PlannerInfo *root,
*** 925,930 ****
--- 1496,1505 ----
}
else if (jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary limitation, like above. */
+ if (grouped_inner)
+ return;
+
inner_path = (Path *) create_unique_path(root, innerrel,
inner_path, extra->sjinfo);
Assert(inner_path);
*************** sort_inner_and_outer(PlannerInfo *root,
*** 946,958 ****
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! cheapest_partial_outer = (Path *) linitial(outerrel->partial_pathlist);
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
/*
--- 1521,1570 ----
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! if (grouped_outer)
! {
! if (REL_HAS_PARTIAL_GROUPED_PATHS(outerrel))
! cheapest_partial_outer = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
! else
! return;
! }
! else
! cheapest_partial_outer = (Path *)
! linitial(outerrel->partial_pathlist);
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else
+ {
+ Assert(innerrel->gpi != NULL);
+ inner_pathlist = innerrel->gpi->pathlist;
+ }
+
+ /*
+ * All the grouped paths should be unparameterized, so the
+ * function is overly stringent in the grouped_inner case, but
+ * still useful.
+ */
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1028,1060 ****
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra);
}
}
--- 1640,1663 ----
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_mergejoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! true, grouped_outer, grouped_inner,
! do_aggregate);
}
}
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1071,1076 ****
--- 1674,1687 ----
* some sort key requirements). So, we consider truncations of the
* mergeclause list as well as the full list. (Ideally we'd consider all
* subsets of the mergeclause list, but that seems way too expensive.)
+ *
+ * grouped_outer - is outerpath grouped?
+ * grouped_inner - use grouped paths of innerrel?
+ * do_aggregate - apply (partial) aggregation to the output?
+ *
+ * TODO If subsequent calls often differ only by the 3 arguments above,
+ * consider a workspace structure to share useful info (eg merge clauses)
+ * across calls.
*/
static void
generate_mergejoin_paths(PlannerInfo *root,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1082,1088 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1693,1702 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1133,1149 ****
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1747,1764 ----
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner, do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1199,1214 ****
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
List *newclauses = NIL;
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
--- 1814,1835 ----
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
+ List *inner_pathlist = NIL;
Path *innerpath;
List *newclauses = NIL;
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else if (innerrel->gpi != NULL)
+ inner_pathlist = innerrel->gpi->pathlist;
+
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1231,1251 ****
}
else
newclauses = mergeclauses;
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
--- 1852,1876 ----
}
else
newclauses = mergeclauses;
!
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
!
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1276,1292 ****
else
newclauses = mergeclauses;
}
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1901,1919 ----
else
newclauses = mergeclauses;
}
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1299,1341 ****
}
}
/*
! * match_unsorted_outer
! * Creates possible join paths for processing a single join relation
! * 'joinrel' by employing either iterative substitution or
! * mergejoining on each of its possible outer paths (considering
! * only outer paths that are already ordered well enough for merging).
! *
! * We always generate a nestloop path for each available outer path.
! * In fact we may generate as many as five: one on the cheapest-total-cost
! * inner path, one on the same with materialization, one on the
! * cheapest-startup-cost inner path (if different), one on the
! * cheapest-total inner-indexscan path (if any), and one on the
! * cheapest-startup inner-indexscan path (if different).
! *
! * We also consider mergejoins if mergejoin clauses are available. See
! * detailed comments in generate_mergejoin_paths.
! *
! * 'joinrel' is the join relation
! * 'outerrel' is the outer join relation
! * 'innerrel' is the inner join relation
! * 'jointype' is the type of join to do
! * 'extra' contains additional input values
*/
static void
! match_unsorted_outer(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
bool useallclauses;
! Path *inner_cheapest_total = innerrel->cheapest_total_path;
Path *matpath = NULL;
ListCell *lc1;
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
--- 1926,1957 ----
}
}
+
/*
! * TODO As merge_pathkeys shouldn't differ across calls, use a separate
! * function to derive them and pass them here in a list.
*/
static void
! match_unsorted_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
bool useallclauses;
! Path *inner_cheapest_total;
Path *matpath = NULL;
ListCell *lc1;
+ List *outer_pathlist;
+
+ /* At most one input path may be grouped. */
+ Assert(!(grouped_outer && grouped_inner));
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
*************** match_unsorted_outer(PlannerInfo *root,
*** 1372,1384 ****
--- 1988,2025 ----
break;
}
+ if (grouped_outer)
+ {
+ if (REL_HAS_GROUPED_PATHS(outerrel))
+ outer_pathlist = outerrel->gpi->pathlist;
+ else
+ return;
+ }
+ else
+ outer_pathlist = outerrel->pathlist;
+
+ if (grouped_inner)
+ {
+ if (REL_HAS_GROUPED_PATHS(innerrel))
+ inner_cheapest_total = linitial(innerrel->gpi->pathlist);
+ else
+ return;
+ }
+ else
+ inner_cheapest_total = innerrel->cheapest_total_path;
+
/*
* If inner_cheapest_total is parameterized by the outer rel, ignore it;
* we will consider it below as a member of cheapest_parameterized_paths,
* but the other possibilities considered in this routine aren't usable.
*/
if (PATH_PARAM_BY_REL(inner_cheapest_total, outerrel))
+ {
+ /* Grouped path should not be parameterized at all. */
+ Assert(!grouped_inner);
+
inner_cheapest_total = NULL;
+ }
/*
* If we need to unique-ify the inner path, we will consider only the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1389,1404 ****
/* No way to do this with an inner path parameterized by outer rel */
if (inner_cheapest_total == NULL)
return;
inner_cheapest_total = (Path *)
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK)
{
/*
* Consider materializing the cheapest inner path, unless
* enable_material is off or the path in question materializes its
* output anyway.
*/
if (enable_material && inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
--- 2030,2060 ----
/* No way to do this with an inner path parameterized by outer rel */
if (inner_cheapest_total == NULL)
return;
+
+ /*
+ * TODO This is just a temporary limitation. Before lifting it, make
+ * sure that the UniquePath does emit GroupedVars. Also try to avoid
+ * the unique-ification if outer_path comes directly from AggPath
+ * (i.e. it's not grouped path combined with plain one) and the
+ * grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_inner)
+ return;
+
inner_cheapest_total = (Path *)
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK && !grouped_inner)
{
/*
* Consider materializing the cheapest inner path, unless
* enable_material is off or the path in question materializes its
* output anyway.
+ *
+ * TODO Verify that this is ok even if grouped_inner is true (at least
+ * make sure that mathpath does contain GroupedVars) and remove
+ * grouped_inner from the condition above.
*/
if (enable_material && inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
*************** match_unsorted_outer(PlannerInfo *root,
*** 1406,1412 ****
create_material_path(innerrel, inner_cheapest_total);
}
! foreach(lc1, outerrel->pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
--- 2062,2068 ----
create_material_path(innerrel, inner_cheapest_total);
}
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1426,1431 ****
--- 2082,2092 ----
{
if (outerpath != outerrel->cheapest_total_path)
continue;
+
+ /* TODO Temporary restriction, see above. */
+ if (grouped_outer)
+ continue;
+
outerpath = (Path *) create_unique_path(root, outerrel,
outerpath, extra->sjinfo);
Assert(outerpath);
*************** match_unsorted_outer(PlannerInfo *root,
*** 1441,1457 ****
if (save_jointype == JOIN_UNIQUE_INNER)
{
/*
* Consider nestloop join, but only with the unique-ified cheapest
* inner path
*/
! try_nestloop_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra);
}
else if (nestjoinOK)
{
--- 2102,2124 ----
if (save_jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary restriction, see above. */
+ if (grouped_inner)
+ continue;
+
/*
* Consider nestloop join, but only with the unique-ified cheapest
* inner path
*/
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1463,1490 ****
*/
ListCell *lc2;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
! Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra);
}
! /* Also consider materialized form of the cheapest inner path */
if (matpath != NULL)
! try_nestloop_path(root,
! joinrel,
! outerpath,
! matpath,
! merge_pathkeys,
! jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 2130,2169 ----
*/
ListCell *lc2;
! /*
! * There are no grouped paths in cheapest_parameterized_paths.
! */
! if (!grouped_inner)
{
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
! }
}
! /*
! * Also consider materialized form of the cheapest inner path.
! */
if (matpath != NULL)
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! matpath,
! merge_pathkeys,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1499,1505 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 2178,2185 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1510,1526 ****
* we handle extra_lateral_rels, since partial paths must not be
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
! bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 2190,2212 ----
* we handle extra_lateral_rels, since partial paths must not be
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
+ *
+ * grouped_inner must be false because we're not interested in nest loop
+ * joins with the grouped path on the inner side (i.e. repeated
+ * aggregation).
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
! bms_is_empty(joinrel->lateral_relids) &&
! !grouped_inner)
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra,
! grouped_outer, do_aggregate);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1540,1550 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
/*
* consider_parallel_mergejoin
* Try to build partial paths for a joinrel by joining a partial path
* for the outer relation to a complete path for the inner relation.
--- 2226,2283 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total,
! grouped_outer, do_aggregate);
}
}
/*
+ * match_unsorted_outer
+ * Creates possible join paths for processing a single join relation
+ * 'joinrel' by employing either iterative substitution or
+ * mergejoining on each of its possible outer paths (considering
+ * only outer paths that are already ordered well enough for merging).
+ *
+ * We always generate a nestloop path for each available outer path.
+ * In fact we may generate as many as five: one on the cheapest-total-cost
+ * inner path, one on the same with materialization, one on the
+ * cheapest-startup-cost inner path (if different), one on the
+ * cheapest-total inner-indexscan path (if any), and one on the
+ * cheapest-startup inner-indexscan path (if different).
+ *
+ * We also consider mergejoins if mergejoin clauses are available. See
+ * detailed comments in generate_mergejoin_paths.
+ *
+ * 'joinrel' is the join relation
+ * 'outerrel' is the outer join relation
+ * 'innerrel' is the inner join relation
+ * 'jointype' is the type of join to do
+ * 'extra' contains additional input values
+ * 'grouped' indicates that the at least one relation in the join has been
+ * aggregated.
+ */
+ static void
+ match_unsorted_outer(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra)
+ {
+ /* Plain (non-grouped) join. */
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, false);
+
+ /* Use all the supported strategies to generate grouped join. */
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, true, false, false);
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, true, false);
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, true);
+ }
+
+ /*
* consider_parallel_mergejoin
* Try to build partial paths for a joinrel by joining a partial path
* for the outer relation to a complete path for the inner relation.
*************** match_unsorted_outer(PlannerInfo *root,
*** 1556,1561 ****
--- 2289,2298 ----
* 'extra' contains additional input values
* 'inner_cheapest_total' cheapest total path for innerrel
*/
+ /*
+ * TODO Store merge_pathkeys across calls if these (the calls) only differ in
+ * grouped_outer or do_aggregate.
+ */
static void
consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
*************** consider_parallel_mergejoin(PlannerInfo
*** 1563,1574 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
/* generate merge join path for each partial outer path */
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
--- 2300,2335 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped_outer,
! bool do_aggregate)
{
ListCell *lc1;
+ List *outer_pathlist;
+ bool grouped = grouped_outer || do_aggregate;
+
+ if (!grouped || do_aggregate)
+ {
+ /*
+ * If creating grouped paths by explicit aggregation, the input paths
+ * must be plain.
+ */
+ outer_pathlist = outerrel->partial_pathlist;
+ }
+ else if (outerrel->gpi != NULL)
+ {
+ /*
+ * Only the outer paths are accepted as grouped when we try to combine
+ * grouped and plain ones. Grouped inner path implies repeated
+ * aggregation, which doesn't sound as a good idea.
+ */
+ outer_pathlist = outerrel->gpi->partial_pathlist;
+ }
+ else
+ return;
/* generate merge join path for each partial outer path */
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
*************** consider_parallel_mergejoin(PlannerInfo
*** 1579,1587 ****
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
! extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 2340,2349 ----
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, grouped_outer, false, do_aggregate);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1596,1616 ****
* 'jointype' is the type of join to do
* 'extra' contains additional input values
*/
static void
consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
--- 2358,2405 ----
* 'jointype' is the type of join to do
* 'extra' contains additional input values
*/
+ /*
+ * TODO Store pathkeys across calls if these (the calls) only differ in
+ * grouped_outer or do_aggregate.
+ */
static void
consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer, bool do_aggregate)
{
JoinType save_jointype = jointype;
+ List *outer_pathlist;
ListCell *lc1;
+ bool grouped = grouped_outer || do_aggregate;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! if (!grouped || do_aggregate)
! {
! /*
! * If creating grouped paths by explicit aggregation, the input paths
! * must be plain.
! */
! outer_pathlist = outerrel->partial_pathlist;
! }
! else if (outerrel->gpi != NULL)
! {
! /*
! * Only the outer paths are accepted as grouped when we try to combine
! * grouped and plain ones. Grouped inner path implies repeated
! * aggregation, which doesn't sound as a good idea.
! */
! outer_pathlist = outerrel->gpi->partial_pathlist;
! }
! else
! return;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1641,1647 ****
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
--- 2430,2436 ----
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER && !grouped)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1651,1680 ****
Assert(innerpath);
}
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
/*
! * hash_inner_and_outer
! * Create hashjoin join paths by explicitly hashing both the outer and
! * inner keys of each available hash clause.
! *
! * 'joinrel' is the join relation
! * 'outerrel' is the outer join relation
! * 'innerrel' is the inner join relation
! * 'jointype' is the type of join to do
! * 'extra' contains additional input values
*/
static void
! hash_inner_and_outer(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
--- 2440,2468 ----
Assert(innerpath);
}
! try_nestloop_path_common(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! true, grouped_outer, false,
! do_aggregate);
}
}
}
/*
! * TODO hashclauses (and mabye some other info) should be shared across calls
! * if only some of the following arguments change: partial, grouped_outer,
! * grouped_inner, do_aggregate.
*/
static void
! hash_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1721,1729 ****
* outer paths. There's no need to consider any but the
* cheapest-total-cost inner path, however.
*/
! Path *cheapest_startup_outer = outerrel->cheapest_startup_path;
! Path *cheapest_total_outer = outerrel->cheapest_total_path;
! Path *cheapest_total_inner = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 2509,2544 ----
* outer paths. There's no need to consider any but the
* cheapest-total-cost inner path, however.
*/
! Path *cheapest_startup_outer,
! *cheapest_total_outer,
! *cheapest_total_inner;
!
! if (grouped_outer)
! {
! if (REL_HAS_GROUPED_PATHS(outerrel))
! {
! cheapest_total_outer = linitial(outerrel->gpi->pathlist);
! /* No parameterized grouped paths. */
! cheapest_startup_outer = NULL;
! }
! else
! return;
! }
! else
! {
! cheapest_total_outer = outerrel->cheapest_total_path;
! cheapest_startup_outer = outerrel->cheapest_startup_path;
! }
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! cheapest_total_inner = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! cheapest_total_inner = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1738,1780 ****
/* Unique-ify if need be; we ignore parameterized possibilities */
if (jointype == JOIN_UNIQUE_OUTER)
{
cheapest_total_outer = (Path *)
create_unique_path(root, outerrel,
cheapest_total_outer, extra->sjinfo);
Assert(cheapest_total_outer);
jointype = JOIN_INNER;
! try_hashjoin_path(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
{
cheapest_total_inner = (Path *)
create_unique_path(root, innerrel,
cheapest_total_inner, extra->sjinfo);
Assert(cheapest_total_inner);
jointype = JOIN_INNER;
! try_hashjoin_path(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
}
else
{
--- 2553,2618 ----
/* Unique-ify if need be; we ignore parameterized possibilities */
if (jointype == JOIN_UNIQUE_OUTER)
{
+ /*
+ * TODO This is just a temporary limitation. Before lifting it,
+ * make sure that the UniquePath does emit GroupedVars. Also try
+ * to avoid the unique-ification if the outer path comes directly
+ * from AggPath (i.e. it's not grouped path combined with plain
+ * one) and the grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_outer)
+ return;
+
cheapest_total_outer = (Path *)
create_unique_path(root, outerrel,
cheapest_total_outer, extra->sjinfo);
Assert(cheapest_total_outer);
jointype = JOIN_INNER;
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary restriction, see above. */
+ if (grouped_inner)
+ return;
+
cheapest_total_inner = (Path *)
create_unique_path(root, innerrel,
cheapest_total_inner, extra->sjinfo);
Assert(cheapest_total_inner);
jointype = JOIN_INNER;
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
!
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
}
else
{
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1784,1805 ****
* pairings of cheapest-total paths including parameterized ones.
* There is no use in generating parameterized paths on the basis
* of possibly cheap startup cost, so this is sufficient.
*/
ListCell *lc1;
ListCell *lc2;
if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
Path *outerpath = (Path *) lfirst(lc1);
/*
* We cannot use an outer path that is parameterized by the
--- 2622,2656 ----
* pairings of cheapest-total paths including parameterized ones.
* There is no use in generating parameterized paths on the basis
* of possibly cheap startup cost, so this is sufficient.
+ *
+ * For if either side of the join is grouped, we simply use
+ * rel->gpi->pathlist. (cheapest_startup_outer should be NULL if
+ * grouped_outer.)
*/
ListCell *lc1;
ListCell *lc2;
+ List *outer_pathlist;
if (cheapest_startup_outer != NULL)
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
! outer_pathlist = !grouped_outer ?
! outerrel->cheapest_parameterized_paths :
! outerrel->gpi->pathlist;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
+ List *inner_pathlist;
/*
* We cannot use an outer path that is parameterized by the
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1808,1814 ****
if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
Path *innerpath = (Path *) lfirst(lc2);
--- 2659,2669 ----
if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! inner_pathlist = !grouped_inner ?
! innerrel->cheapest_parameterized_paths :
! innerrel->gpi->pathlist;
!
! foreach(lc2, inner_pathlist)
{
Path *innerpath = (Path *) lfirst(lc2);
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1823,1835 ****
innerpath == cheapest_total_inner)
continue; /* already tried it */
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra);
}
}
}
--- 2678,2693 ----
innerpath == cheapest_total_inner)
continue; /* already tried it */
! try_hashjoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1852,1883 ****
Path *cheapest_partial_outer;
Path *cheapest_safe_inner = NULL;
! cheapest_partial_outer =
! (Path *) linitial(outerrel->partial_pathlist);
/*
* Normally, given that the joinrel is parallel-safe, the cheapest
* total inner path will also be parallel-safe, but if not, we'll
! * have to search for the cheapest safe, unparameterized inner
! * path. If doing JOIN_UNIQUE_INNER, we can't use any alternative
! * inner path.
*/
if (cheapest_total_inner->parallel_safe)
cheapest_safe_inner = cheapest_total_inner;
else if (save_jointype != JOIN_UNIQUE_INNER)
! cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra);
}
}
}
/*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
--- 2710,2805 ----
Path *cheapest_partial_outer;
Path *cheapest_safe_inner = NULL;
! if (grouped_outer)
! {
! if (REL_HAS_PARTIAL_GROUPED_PATHS(outerrel))
! {
! cheapest_partial_outer =
! (Path *) linitial(outerrel->gpi->partial_pathlist);
! }
! else
! return;
! }
! else
! cheapest_partial_outer =
! (Path *) linitial(outerrel->partial_pathlist);
/*
* Normally, given that the joinrel is parallel-safe, the cheapest
* total inner path will also be parallel-safe, but if not, we'll
! * have to search cheapest_parameterized_paths for the cheapest
! * safe, unparameterized inner path. If doing JOIN_UNIQUE_INNER,
! * we can't use any alternative inner path.
*/
if (cheapest_total_inner->parallel_safe)
cheapest_safe_inner = cheapest_total_inner;
else if (save_jointype != JOIN_UNIQUE_INNER)
! {
! ListCell *lc;
! List *inner_pathlist;
!
! inner_pathlist = !grouped_inner ?
! innerrel->cheapest_parameterized_paths :
! innerrel->gpi->pathlist;
!
! foreach(lc, inner_pathlist)
! {
! Path *innerpath = (Path *) lfirst(lc);
!
! if (innerpath->parallel_safe &&
! bms_is_empty(PATH_REQ_OUTER(innerpath)))
! {
! cheapest_safe_inner = innerpath;
! break;
! }
! }
! }
if (cheapest_safe_inner != NULL)
! try_hashjoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra,
! true,
! grouped_outer, grouped_inner,
! do_aggregate);
}
}
}
/*
+ * hash_inner_and_outer
+ * Create hashjoin join paths by explicitly hashing both the outer and
+ * inner keys of each available hash clause.
+ *
+ * 'joinrel' is the join relation
+ * 'outerrel' is the outer join relation
+ * 'innerrel' is the inner join relation
+ * 'jointype' is the type of join to do
+ * 'extra' contains additional input values
+ */
+ static void
+ hash_inner_and_outer(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra)
+ {
+ /* Plain (non-grouped) join. */
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, false);
+
+ /* Use all the supported strategies to generate grouped join. */
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, true, false, false);
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, true, false);
+ hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, false, false, true);
+ }
+
+ /*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 382c0f0..55fab19
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** try_partition_wise_join(PlannerInfo *roo
*** 1403,1409 ****
(List *) adjust_appendrel_attrs(root,
(Node *) parent_restrictlist,
nappinfos, appinfos);
- pfree(appinfos);
child_joinrel = joinrel->part_rels[cnt_parts];
if (!child_joinrel)
--- 1403,1408 ----
*************** try_partition_wise_join(PlannerInfo *roo
*** 1413,1419 ****
--- 1412,1426 ----
child_sjinfo,
child_sjinfo->jointype);
joinrel->part_rels[cnt_parts] = child_joinrel;
+
+ /*
+ * If the parent join can be grouped, so can be its children.
+ */
+ if (joinrel->gpi != NULL)
+ build_chiid_rel_gpi(root, child_joinrel, joinrel, nappinfos,
+ appinfos);
}
+ pfree(appinfos);
Assert(bms_equal(child_joinrel->relids, child_joinrelids));
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index 4b49748..0bbeeda
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** use_physical_tlist(PlannerInfo *root, Pa
*** 808,813 ****
--- 808,819 ----
return false;
/*
+ * Grouped relation's target list contains GroupedVars.
+ */
+ if (rel->gpi != NULL)
+ return false;
+
+ /*
* If a bitmap scan's tlist is empty, keep it as-is. This may allow the
* executor to skip heap page fetches, and in any case, the benefit of
* using a physical tlist instead would be minimal.
*************** create_projection_plan(PlannerInfo *root
*** 1585,1592 ****
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
--- 1591,1599 ----
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (!best_path->force_result &&
! (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist)))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 25aa1aa..04cb908
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
*************** create_grouping_expr_grouped_var_infos(P
*** 601,606 ****
--- 601,616 ----
/* Find out where the aggregate should be evaluated. */
gvi->gv_eval_at = pull_varnos((Node *) expr);
+ /*
+ * If the grouping column is a plain Var, it has GroupedVarInfo (XXX
+ * should it be so?) but the actual Var will appear in the target.
+ * Thus set_rel_width should get better estimate from statistics. Do
+ * not waste cycles here to get less accurate value.
+ */
+ if (!IsA(gvi->gvexpr, Var))
+ gvi->gv_width = get_typavgwidth(exprType((Node *) gvi->gvexpr),
+ exprTypmod((Node *) gvi->gvexpr));
+
root->grouped_var_list = lappend(root->grouped_var_list, gvi);
}
}
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index cba5c4a..33b66c4
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
*************** create_append_path(RelOptInfo *rel, List
*** 1307,1317 ****
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
List *subpaths,
List *pathkeys,
Relids required_outer,
--- 1307,1319 ----
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode. target can be supplied by caller. If NULL is passed, the field
! * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** create_merge_append_path(PlannerInfo *ro
*** 1324,1330 ****
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1326,1332 ----
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = target ? target : rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** calc_non_nestloop_required_outer(Path *o
*** 2115,2120 ****
--- 2117,2123 ----
* 'restrict_clauses' are the RestrictInfo nodes to apply at the join
* 'pathkeys' are the path keys of the new join path
* 'required_outer' is the set of required outer rels
+ * 'target' can be passed to override that of joinrel.
*
* Returns the resulting path node.
*/
*************** create_nestloop_path(PlannerInfo *root,
*** 2128,2134 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
--- 2131,2138 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
*************** create_nestloop_path(PlannerInfo *root,
*** 2161,2167 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2165,2171 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target == NULL ? joinrel->reltarget : target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2219,2231 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2223,2237 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2270,2275 ****
--- 2276,2282 ----
* 'required_outer' is the set of required outer rels
* 'hashclauses' are the RestrictInfo nodes to use as hash clauses
* (this should be a subset of the restrict_clauses list)
+ * 'target' can be passed to override that of joinrel.
*/
HashPath *
create_hashjoin_path(PlannerInfo *root,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2281,2293 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2288,2302 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 2197ac4..b018f02
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
*************** build_join_rel(PlannerInfo *root,
*** 537,542 ****
--- 537,543 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 590,595 ****
--- 591,626 ----
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
+ * If grouping is not applicable at relation level, try to build grouped
+ * targets --- both for aggregation and for joining grouped relation to
+ * non-grouped one.
+ */
+ if (root->grouped_var_list != NIL)
+ {
+ /*
+ * TODO Consider if placeholders make sense here. If not, also make
+ * the related code below conditional.
+ */
+ prepare_rel_for_grouping(root, joinrel);
+
+ /*
+ * If the relation appears to be eligible for grouping, joinrel->gpi
+ * has been initialized. Compute cost of the grouping target.
+ */
+
+ /*
+ * TODO Store the costs of GroupedVars separate in GroupedPathInfo and
+ * only add it to the join cost if its result is actually aggregated.
+ * In contrast, if the grouped join is formed by joining a group
+ * relation to non-grouped one, the cost of GroupedVar should not
+ * included (but width should) because the grouped input relation was
+ * in charge of evaluation.
+ */
+ if (joinrel->gpi != NULL)
+ set_pathtarget_cost_width(root, joinrel->gpi->target);
+ }
+
+ /*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
* now we can finish computing that. This is much like the computation of
*************** build_child_join_rel(PlannerInfo *root,
*** 712,717 ****
--- 743,749 ----
joinrel->cheapest_parameterized_paths = NIL;
joinrel->direct_lateral_relids = NULL;
joinrel->lateral_relids = NULL;
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_child_join_rel(PlannerInfo *root,
*** 760,766 ****
(Node *) parent_joinrel->joininfo,
nappinfos,
appinfos);
- pfree(appinfos);
/*
* Lateral relids referred in child join will be same as that referred in
--- 792,797 ----
*************** build_child_join_rel(PlannerInfo *root,
*** 796,805 ****
--- 827,891 ----
/* Add the relation to the PlannerInfo. */
add_join_rel(root, joinrel);
+ pfree(appinfos);
+
return joinrel;
}
/*
+ * Initialize GroupedPathInfo of a child relation according to that of the
+ * parent.
+ */
+ void
+ build_chiid_rel_gpi(PlannerInfo *root, RelOptInfo *child, RelOptInfo *parent,
+ int nappinfos, AppendRelInfo **appinfos)
+ {
+ GroupedPathInfo *gpi,
+ *gpi_parent;
+
+ Assert(child->gpi == NULL);
+
+ gpi_parent = parent->gpi;
+ child->gpi = gpi = makeNode(GroupedPathInfo);
+
+ /*
+ * Create grouping target translated to the child varnos.
+ *
+ * We need a copy of the target so that the translation does not affect
+ * the parent.
+ */
+ gpi->target = copy_pathtarget(gpi_parent->target);
+ gpi->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->target->exprs,
+ nappinfos, appinfos);
+
+ /*
+ * Non-var grouping expressions need to be translated as well.
+ */
+ if (gpi_parent->group_exprs != NULL)
+ {
+ gpi->group_exprs = copy_pathtarget(gpi_parent->group_exprs);
+ gpi->group_exprs->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->group_exprs->exprs,
+ nappinfos, appinfos);
+ }
+
+ /*
+ * sortgroupclauses need no kind of translation, so just copy the pointer.
+ */
+ gpi->sortgroupclauses = gpi_parent->sortgroupclauses;
+
+ /*
+ * If the output of the child rel's paths should be aggregated,
+ * sortgrouprefs are also needed. (initialize_grouped_targets should have
+ * initialized sortgrouprefs of the parent rel.)
+ */
+ child->reltarget->sortgrouprefs = parent->reltarget->sortgrouprefs;
+ }
+
+ /*
* min_join_parameterization
*
* Determine the minimum possible parameterization of a joinrel, that is, the
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index d454df3..c825d7c
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern AppendPath *create_append_path(Re
*** 70,75 ****
--- 70,76 ----
List *partitioned_rels);
extern MergeAppendPath *create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** extern NestPath *create_nestloop_path(Pl
*** 129,135 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 130,137 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern MergePath *create_mergejoin_path(
*** 143,149 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 145,152 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern HashPath *create_hashjoin_path(Pl
*** 154,160 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
--- 157,164 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
*************** extern RelOptInfo *build_child_join_rel(
*** 313,317 ****
--- 317,324 ----
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
+ extern void build_chiid_rel_gpi(PlannerInfo *root, RelOptInfo *child,
+ RelOptInfo *parent, int nappinfos,
+ AppendRelInfo **appinfos);
extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index ...cac70a0
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
***************
*** 0 ****
--- 1,299 ----
+ BEGIN;
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ CREATE INDEX ON agg_pushdown_child1(parent);
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ ANALYZE;
+ SET enable_agg_pushdown TO on;
+ -- The supposed advantage of pushing aggregation down below joins is that
+ -- joins will then receive less input tuples. Let's stress this difference.
+ SET cpu_tuple_cost TO 0.1;
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (8 rows)
+
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (8 rows)
+
+ SET enable_seqscan TO on;
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (12 rows)
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ----------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+ -> Hash
+ -> Seq Scan on agg_pushdown_parent p
+ (13 rows)
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (14 rows)
+
+ -- Generic grouping expression.
+ -- Make any processing of the grouping expression expensive. If the final
+ -- relation output needs to be sorted before GroupAggregate is applied, the
+ -- (p.i / 2) expression is also evaluated by the Sort node. Thus the plan that
+ -- uses the partial aggregation is preferrable because it emits the value of
+ -- the exprssion and no node above has to evaluate it again.
+ SET cpu_operator_cost TO 0.1;
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Result
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (15 rows)
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+ SET enable_agg_pushdown TO on;
+ SET cpu_tuple_cost TO 0.1;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ SET enable_seqscan TO on;
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (14 rows)
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ -- TODO Force the Hash Join node to stay below Gather. Does each parallel
+ -- worker still use a separate hash table? Try again once this limitation has
+ -- been fixed.
+ -- EXPLAIN (COSTS off)
+ -- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ -- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ -- c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (16 rows)
+
+ SET cpu_operator_cost TO 0.1;
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Gather
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Result
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (17 rows)
+
+ ROLLBACK;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
new file mode 100644
index aa5e6af..0f9af11
*** a/src/test/regress/parallel_schedule
--- b/src/test/regress/parallel_schedule
*************** test: rules psql_crosstab amutils
*** 98,103 ****
--- 98,106 ----
test: select_parallel
test: write_parallel
+ # this one runs parallel workers too
+ test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
new file mode 100644
index 3866314..7d3464e
*** a/src/test/regress/serial_schedule
--- b/src/test/regress/serial_schedule
*************** test: stats_ext
*** 135,140 ****
--- 135,141 ----
test: rules
test: psql_crosstab
test: select_parallel
+ test: agg_pushdown
test: write_parallel
test: publication
test: subscription
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index ...f5b9b24
*** a/src/test/regress/sql/agg_pushdown.sql
--- b/src/test/regress/sql/agg_pushdown.sql
***************
*** 0 ****
--- 1,155 ----
+ BEGIN;
+
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ CREATE INDEX ON agg_pushdown_child1(parent);
+
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ ANALYZE;
+
+ SET enable_agg_pushdown TO on;
+
+ -- The supposed advantage of pushing aggregation down below joins is that
+ -- joins will then receive less input tuples. Let's stress this difference.
+ SET cpu_tuple_cost TO 0.1;
+
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ SET enable_seqscan TO on;
+
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- Generic grouping expression.
+
+ -- Make any processing of the grouping expression expensive. If the final
+ -- relation output needs to be sorted before GroupAggregate is applied, the
+ -- (p.i / 2) expression is also evaluated by the Sort node. Thus the plan that
+ -- uses the partial aggregation is preferrable because it emits the value of
+ -- the exprssion and no node above has to evaluate it again.
+ SET cpu_operator_cost TO 0.1;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+
+ SET enable_agg_pushdown TO on;
+ SET cpu_tuple_cost TO 0.1;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ SET enable_seqscan TO on;
+
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ -- TODO Force the Hash Join node to stay below Gather. Does each parallel
+ -- worker still use a separate hash table? Try again once this limitation has
+ -- been fixed.
+
+ -- EXPLAIN (COSTS off)
+ -- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ -- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ -- c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET cpu_operator_cost TO 0.1;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ ROLLBACK;
+
agg_pushdown_v4/07_avoid_agg_finalization.diff
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index dfab262..06421e4
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyAggref(const Aggref *from)
*** 1358,1363 ****
--- 1358,1364 ----
COPY_SCALAR_FIELD(inputcollid);
COPY_SCALAR_FIELD(aggtranstype);
COPY_SCALAR_FIELD(aggcombinefn);
+ COPY_SCALAR_FIELD(aggfinalfn);
COPY_NODE_FIELD(aggargtypes);
COPY_NODE_FIELD(aggdirectargs);
COPY_NODE_FIELD(args);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index ec83734..fd52c2c
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outAggref(StringInfo str, const Aggref
*** 1132,1137 ****
--- 1132,1138 ----
WRITE_OID_FIELD(inputcollid);
WRITE_OID_FIELD(aggtranstype);
WRITE_OID_FIELD(aggcombinefn);
+ WRITE_OID_FIELD(aggfinalfn);
WRITE_NODE_FIELD(aggargtypes);
WRITE_NODE_FIELD(aggdirectargs);
WRITE_NODE_FIELD(args);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index fc739b7..d7ed26b
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readAggref(void)
*** 601,606 ****
--- 601,607 ----
READ_OID_FIELD(inputcollid);
READ_OID_FIELD(aggtranstype);
READ_OID_FIELD(aggcombinefn);
+ READ_OID_FIELD(aggfinalfn);
READ_NODE_FIELD(aggargtypes);
READ_NODE_FIELD(aggdirectargs);
READ_NODE_FIELD(args);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index a125c57..c278acb
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 704,709 ****
--- 704,712 ----
/* Consider sequential scan, both plain and grouped. */
seq_path = create_seqscan_path(root, rel, required_outer, 0);
+ /* Try to compute unique keys. */
+ make_uniquekeys(root, seq_path);
+
add_path(rel, seq_path, false);
if (rel->gpi != NULL && rel->gpi->target != NULL &&
*************** create_grouped_path(PlannerInfo *root, R
*** 838,851 ****
&group_clauses,
&group_exprs,
&agg_exprs,
! subpath->rows);
else if (aggstrategy == AGG_SORTED)
agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
true,
&group_clauses,
&group_exprs,
&agg_exprs,
! subpath->rows);
else
elog(ERROR, "unexpected strategy %d", aggstrategy);
--- 841,856 ----
&group_clauses,
&group_exprs,
&agg_exprs,
! subpath->rows,
! partial);
else if (aggstrategy == AGG_SORTED)
agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
true,
&group_clauses,
&group_exprs,
&agg_exprs,
! subpath->rows,
! partial);
else
elog(ERROR, "unexpected strategy %d", aggstrategy);
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1739,1744 ****
--- 1744,1753 ----
partitioned_rels);
/* pathtarget will produce the grouped relation.. */
path->pathtarget = rel->gpi->target;
+
+ /* Try to compute unique keys. */
+ make_uniquekeys(root, path);
+
add_path(rel, path, true);
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1916,1921 ****
--- 1925,1933 ----
NULL,
partitioned_rels);
+ /* Try to compute unique keys. */
+ make_uniquekeys(root, path);
+
/* pathtarget will produce the grouped relation.. */
if (grouped)
{
*************** generate_mergeappend_paths(PlannerInfo *
*** 1934,1939 ****
--- 1946,1952 ----
pathkeys,
NULL,
partitioned_rels);
+ make_uniquekeys(root, path);
if (grouped)
path->pathtarget = rel->gpi->target;
add_path(rel, path, grouped);
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 5ad1a7f..8be754b
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
*************** get_index_paths(PlannerInfo *root, RelOp
*** 791,797 ****
--- 791,806 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
+ {
+ /*
+ * In case grouping may be pushed down, try to identify unique
+ * keys to possibly avoid final aggregation.
+ */
+ if (root->grouped_var_list != NIL)
+ make_uniquekeys(root, (Path *) ipath);
+
add_path(rel, (Path *) ipath, grouped);
+ }
if (!grouped && index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
*************** build_index_paths(PlannerInfo *root, Rel
*** 1076,1082 ****
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows);
if (agg_path != NULL)
result = lappend(result, agg_path);
--- 1085,1092 ----
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows,
! false);
if (agg_path != NULL)
result = lappend(result, agg_path);
*************** build_index_paths(PlannerInfo *root, Rel
*** 1126,1132 ****
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows);
/*
* If create_agg_sorted_path succeeded once, it should
--- 1136,1143 ----
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows,
! true);
/*
* If create_agg_sorted_path succeeded once, it should
*************** build_index_paths(PlannerInfo *root, Rel
*** 1179,1185 ****
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows);
Assert(agg_path != NULL);
result = lappend(result, agg_path);
--- 1190,1197 ----
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows,
! false);
Assert(agg_path != NULL);
result = lappend(result, agg_path);
*************** build_index_paths(PlannerInfo *root, Rel
*** 1225,1231 ****
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows);
Assert(agg_path != NULL);
add_partial_path(rel, (Path *) agg_path, grouped);
}
--- 1237,1244 ----
&group_clauses,
&group_exprs,
&agg_exprs,
! agg_input_rows,
! true);
Assert(agg_path != NULL);
add_partial_path(rel, (Path *) agg_path, grouped);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index f9c2fb1..fdcd6d1
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** try_nestloop_path(PlannerInfo *root,
*** 506,511 ****
--- 506,518 ----
workspace.startup_cost, workspace.total_cost,
pathkeys, required_outer, grouped))
{
+ /*
+ * For a grouped path try to identify unique keys, to possibly avoid
+ * final aggregation.
+ */
+ if (grouped)
+ make_uniquekeys(root, (Path *) join_path);
+
add_path(joinrel, join_path, grouped);
}
else
*************** try_mergejoin_path(PlannerInfo *root,
*** 826,831 ****
--- 833,845 ----
workspace.startup_cost, workspace.total_cost,
pathkeys, required_outer, grouped))
{
+ /*
+ * For a grouped path try to identify unique keys, to possibly avoid
+ * final aggregation.
+ */
+ if (grouped)
+ make_uniquekeys(root, (Path *) join_path);
+
add_path(joinrel, (Path *) join_path, grouped);
}
else
*************** try_hashjoin_path(PlannerInfo *root,
*** 1140,1145 ****
--- 1154,1166 ----
workspace.startup_cost, workspace.total_cost,
NIL, required_outer, grouped))
{
+ /*
+ * For a grouped path try to identify unique keys, to possibly avoid
+ * final aggregation.
+ */
+ if (grouped)
+ make_uniquekeys(root, (Path *) join_path);
+
add_path(joinrel, (Path *) join_path, grouped);
}
else
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
new file mode 100644
index 9d83a5c..d845f7b
*** a/src/backend/optimizer/path/pathkeys.c
--- b/src/backend/optimizer/path/pathkeys.c
*************** has_useful_pathkeys(PlannerInfo *root, R
*** 1563,1565 ****
--- 1563,1719 ----
return true; /* might be able to use them for ordering */
return false; /* definitely useless */
}
+
+ /*
+ * Add a new set of unique keys to path's list of unique key sets. If an
+ * identical set is already there --- free new_set instead of adding it.
+ */
+ void
+ add_uniquekeys_to_path(Path *path, Bitmapset *new_set)
+ {
+ ListCell *lc;
+
+ foreach(lc, path->uniquekeys)
+ {
+ Bitmapset *set = (Bitmapset *) lfirst(lc);
+
+ if (bms_equal(new_set, set))
+ break;
+ }
+ if (lc == NULL)
+ path->uniquekeys = lappend(path->uniquekeys, new_set);
+ else
+ bms_free(new_set);
+ }
+
+ /*
+ * Return true if path output is grouped in a way described by
+ * root->group_pathkeys. AGGSPLIT_FINAL_DESERIAL can be skipped in such a
+ * case.
+ */
+ bool
+ match_path_to_group_pathkeys(PlannerInfo *root, Path *path)
+ {
+ Bitmapset *uniquekeys_all = NULL;
+ ListCell *l1;
+ int i;
+ bool *is_group_expr;
+
+ /*
+ * group_pathkeys are essential for this function.
+ */
+ if (root->group_pathkeys == NIL)
+ return false;
+
+ /*
+ * The path is not aware of being unique.
+ */
+ if (path->uniquekeys == NIL)
+ return false;
+
+ /*
+ * There can be multiple known unique key sets. Gather pathkeys of all the
+ * unique expressions the sets may reference.
+ */
+ foreach(l1, path->uniquekeys)
+ {
+ Bitmapset *set = (Bitmapset *) lfirst(l1);
+
+ uniquekeys_all = bms_union(uniquekeys_all, set);
+ }
+
+ /*
+ * Find pathkeys for the expressions.
+ */
+ is_group_expr = (bool *)
+ palloc0(list_length(path->pathtarget->exprs) * sizeof(bool));
+
+ i = 0;
+ foreach(l1, path->pathtarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+
+ if (bms_is_member(i, uniquekeys_all))
+ {
+ ListCell *l2;
+ bool found = false;
+
+ /*
+ * This is an unique expression, so find its pathkey.
+ */
+ foreach(l2, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l2);
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *l3;
+ EquivalenceMember *em = NULL;
+
+ if (ec->ec_below_outer_join)
+ continue;
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(l3, ec->ec_members)
+ {
+ em = lfirst_node(EquivalenceMember, l3);
+
+ if (em->em_nullable_relids)
+ continue;
+
+ if (equal(em->em_expr, expr))
+ {
+ found = true;
+ break;
+ }
+ }
+ if (found)
+ break;
+
+ }
+ is_group_expr[i] = found;
+ }
+
+ i++;
+ }
+
+ /*
+ * Now check the unique key sets and see if any one matches all items of
+ * group_pathkeys.
+ */
+ foreach(l1, path->uniquekeys)
+ {
+ Bitmapset *set = (Bitmapset *) lfirst(l1);
+ bool found = false;
+
+ /*
+ * Collect PKs associated with this set.
+ */
+ for (i = 0; i < list_length(path->pathtarget->exprs); i++)
+ {
+ if (bms_is_member(i, set))
+ {
+ /*
+ * If the set misses a single grouping path key, at least one
+ * expression of the unique key is outside the grouping
+ * expressions, and thus the grouping expressions are not
+ * guaranteed to be unique. Thus the avoidance of final
+ * aggregation is not justified.
+ */
+ if (!is_group_expr[i])
+ {
+ found = true;
+ break;
+ }
+ }
+ }
+
+ /*
+ * No problem with this set. No need to check the other ones.
+ */
+ if (!found)
+ return true;
+ }
+
+ /* No match found. */
+ return false;
+ }
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index 0bbeeda..6cd9c7c
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** prepare_sort_from_pathkeys(Plan *lefttre
*** 5671,5676 ****
--- 5671,5709 ----
foreach(j, tlist)
{
tle = (TargetEntry *) lfirst(j);
+
+ /*
+ * Does this happen to be an aggregate final function
+ * referencing the partial aggregate? This can replace the
+ * final aggregation if the path generates an unique set of
+ * grouping keys.
+ */
+ if (IS_AGGFINALFN_STANDALONE(tle->expr))
+ {
+ FuncExpr *fexpr = castNode(FuncExpr, tle->expr);
+ GroupedVar *gvar = linitial_node(GroupedVar, fexpr->args);
+ Aggref *aggref = castNode(Aggref, gvar->gvexpr);
+
+ Assert(fexpr->funcid == aggref->aggfinalfn);
+
+ tle = flatCopyTargetEntry(tle);
+ tle->expr = (Expr *) aggref;
+ }
+
+ /*
+ * Aggregate w/o aggfinalfn?
+ */
+ else if (IsA(tle->expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, tle->expr);
+
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ tle = flatCopyTargetEntry(tle);
+ tle->expr = gvar->gvexpr;
+ }
+ }
+
em = find_ec_member_for_tle(ec, tle, relids);
if (em)
{
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 84abaee..6b17d96
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static PathTarget *make_sort_input_targe
*** 193,198 ****
--- 193,200 ----
bool *have_postponed_srfs);
static void adjust_paths_for_srfs(PlannerInfo *root, RelOptInfo *rel,
List *targets, List *targets_contain_srfs);
+ static Path *adjust_path_for_unique_group_keys(PlannerInfo *root, Path *path,
+ PathTarget *target);
/*****************************************************************************
*************** grouping_planner(PlannerInfo *root, bool
*** 1974,1979 ****
--- 1976,1982 ----
grouping_target,
&agg_costs,
gset_data);
+
/* Fix things up if grouping_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
*************** create_grouping_paths(PlannerInfo *root,
*** 4172,4177 ****
--- 4175,4206 ----
}
}
+ /*
+ * If input_rel has partially aggregated paths which emit an unique set of
+ * grouping keys, we only need to call aggfinalfn on each aggregate state
+ * instead of doing the final aggregation
+ */
+ if (input_rel->gpi != NULL && !parse->groupingSets)
+ {
+ foreach(lc, input_rel->gpi->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * If aggregation could have been pushed down and if the path
+ * generates unique grouping keys, replace aggregates with the
+ * aggregate final functions alone.
+ */
+ if ((parse->groupClause || parse->hasAggs || root->hasHavingQual)
+ && root->grouped_var_list != NIL &&
+ match_path_to_group_pathkeys(root, path))
+ add_path(grouped_rel,
+ adjust_path_for_unique_group_keys(root, path,
+ target),
+ false);
+ }
+ }
+
/* Give a helpful error if we failed to find any implementation */
if (grouped_rel->pathlist == NIL)
ereport(ERROR,
*************** adjust_paths_for_srfs(PlannerInfo *root,
*** 6153,6158 ****
--- 6182,6267 ----
}
/*
+ * Project a path that generates unique grouping keys to a new one whose
+ * target, instead of the (partial) aggregates, contains the aggregates' final
+ * functions, each referencing the corresponding partial aggregate.
+ */
+ static Path *
+ adjust_path_for_unique_group_keys(PlannerInfo *root, Path *path,
+ PathTarget *target)
+ {
+ ListCell *lc1;
+ PathTarget *target_new;
+ List *exprs = NIL;
+
+ target_new = copy_pathtarget(target);
+
+ /*
+ * Replace (partial) aggregates with aggfinalfn function calls.
+ */
+ foreach(lc1, target_new->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc1);
+
+ if (IsA(expr, Aggref))
+ {
+ ListCell *lc2;
+ Aggref *aggref = castNode(Aggref, expr);
+ GroupedVar *gvar = NULL;
+
+ /*
+ * Find GroupedVar for this aggregate --- the input path should
+ * contain it too. Put it to the target for set_upper_references
+ * to be able to match the output and input target.
+ */
+ foreach(lc2, path->pathtarget->exprs)
+ {
+ Expr *texpr = (Expr *) lfirst(lc2);
+
+ if (IsA(texpr, GroupedVar))
+ {
+ gvar = castNode(GroupedVar, texpr);
+
+ if (equal(gvar->gvexpr, aggref))
+ break;
+ }
+ }
+ /* The GroupedVar should have been found. */
+ Assert(lc2 != NULL);
+
+ /*
+ * Arrange for the call of aggfinalfn or let the transient state
+ * appear on the output unprocessed.
+ */
+ if (OidIsValid(aggref->aggfinalfn))
+ {
+ FuncExpr *fexpr = makeNode(FuncExpr);
+
+ fexpr = makeNode(FuncExpr);
+ fexpr->funcid = aggref->aggfinalfn;
+ fexpr->funcresulttype = aggref->aggtype;
+ fexpr->funcretset = false;
+ fexpr->funcvariadic = aggref->aggvariadic;
+ fexpr->funcformat = COERCE_EXPLICIT_CALL;
+ fexpr->funccollid = aggref->aggcollid;
+ fexpr->inputcollid = aggref->inputcollid;
+ fexpr->args = list_make1(gvar);
+ fexpr->location = -1;
+ expr = (Expr *) fexpr;
+ }
+ else
+ expr = (Expr *) gvar;
+ }
+
+ exprs = lappend(exprs, expr);
+ }
+ target_new->exprs = exprs;
+
+ return (Path *) create_projection_path(root, path->parent, path,
+ target_new, false);
+ }
+
+ /*
* expression_planner
* Perform planner's transformations on a standalone expression.
*
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index aab2430..99aabd5
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** set_upper_references(PlannerInfo *root,
*** 1769,1775 ****
* level.
*/
subplan->targetlist =
! restore_grouping_expressions(root, subplan->targetlist);
}
else if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL)
{
--- 1769,1776 ----
* level.
*/
subplan->targetlist =
! restore_grouping_expressions(root, subplan->targetlist,
! true);
}
else if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL)
{
*************** set_upper_references(PlannerInfo *root,
*** 1779,1787 ****
* Unlike above, the restored expressions should stay there.
*/
subplan->targetlist =
! restore_grouping_expressions(root, subplan->targetlist);
}
}
}
subplan_itlist = build_tlist_index(subplan->targetlist);
--- 1780,1805 ----
* Unlike above, the restored expressions should stay there.
*/
subplan->targetlist =
! restore_grouping_expressions(root, subplan->targetlist,
! true);
}
}
+ else if (IsA(plan, Result))
+ {
+ /*
+ * If standalone aggfinalfn function is used (e.g. in the target
+ * of a Sort node) instead of AGGSPLIT_FINAL_DESERIAL aggregate,
+ * projection to the aggregate might have been added. Replace the
+ * function with the aggregate.
+ *
+ * TODO Find out if the Result plan is in parallel worker. In that
+ * case we'd better pass "true" for agg_partial.
+ */
+ sub_tlist_save = subplan->targetlist;
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist,
+ false);
+ }
}
subplan_itlist = build_tlist_index(subplan->targetlist);
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 33b66c4..3eb0bf4
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
*************** static List *reparameterize_pathlist_by_
*** 58,63 ****
--- 58,66 ----
static Path *add_grouping_expressions_to_subpath(PlannerInfo *root,
Path *subpath,
PathTarget *group_exprs);
+ static void make_uniquekeys_for_unique_index(PathTarget *reltarget,
+ IndexOptInfo *index, Path *path);
+ static void make_uniquekeys_for_append_path(PlannerInfo *root, Path *path);
/*****************************************************************************
* MISC. PATH UTILITIES
*************** create_material_path(RelOptInfo *rel, Pa
*** 1458,1463 ****
--- 1461,1467 ----
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = subpath->pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
*************** create_projection_path(PlannerInfo *root
*** 2367,2372 ****
--- 2371,2377 ----
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order */
pathnode->path.pathkeys = subpath->pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
pathnode->force_result = force_result;
*************** create_sort_path(PlannerInfo *root,
*** 2604,2609 ****
--- 2609,2615 ----
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
*************** create_agg_path(PlannerInfo *root,
*** 2789,2795 ****
/*
* Apply partial AGG_SORTED aggregation path to subpath if it's suitably
! * sorted.
*
* first_call indicates whether the function is being called first time for
* given index --- since the target should not change, we can skip the check
--- 2795,2802 ----
/*
* Apply partial AGG_SORTED aggregation path to subpath if it's suitably
! * sorted. ("partial" in the function name refers to AGGSPLIT_INITIAL_SERIAL
! * strategy, as opposed to parallel processing.)
*
* first_call indicates whether the function is being called first time for
* given index --- since the target should not change, we can skip the check
*************** AggPath *
*** 2805,2811 ****
create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
bool first_call,
List **group_clauses, List **group_exprs,
! List **agg_exprs, double input_rows)
{
RelOptInfo *rel;
AggClauseCosts agg_costs;
--- 2812,2819 ----
create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
bool first_call,
List **group_clauses, List **group_exprs,
! List **agg_exprs, double input_rows,
! bool parallel)
{
RelOptInfo *rel;
AggClauseCosts agg_costs;
*************** create_partial_agg_sorted_path(PlannerIn
*** 2879,2898 ****
AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
*group_clauses, NIL, &agg_costs, dNumGroups);
return result;
}
/*
! * Appy partial AGG_HASHED aggregation to subpath.
*
* Arguments have the same meaning as those of create_agg_sorted_path.
- *
*/
AggPath *
create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
bool first_call,
List **group_clauses, List **group_exprs,
! List **agg_exprs, double input_rows)
{
RelOptInfo *rel;
bool can_hash;
--- 2887,2914 ----
AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
*group_clauses, NIL, &agg_costs, dNumGroups);
+ /*
+ * Check if there's a chance to avoid final aggregation.
+ */
+ if (!parallel)
+ make_uniquekeys(root, (Path *) result);
+
return result;
}
/*
! * Appy partial AGG_HASHED aggregation to subpath. ("partial" in the function
! * name refers to AGGSPLIT_INITIAL_SERIAL strategy, as opposed to parallel
! * processing.)
*
* Arguments have the same meaning as those of create_agg_sorted_path.
*/
AggPath *
create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
bool first_call,
List **group_clauses, List **group_exprs,
! List **agg_exprs, double input_rows,
! bool parallel)
{
RelOptInfo *rel;
bool can_hash;
*************** create_partial_agg_hashed_path(PlannerIn
*** 2969,2974 ****
--- 2985,2996 ----
}
}
+ /*
+ * Check if there's a chance to avoid final aggregation.
+ */
+ if (result != NULL && !parallel)
+ make_uniquekeys(root, (Path *) result);
+
return result;
}
*************** add_grouping_expressions_to_subpath(Plan
*** 4086,4088 ****
--- 4108,4678 ----
target,
true);
}
+
+ /*
+ * Find out if path produces an unique set of expressions and set uniquekeys
+ * accordingly.
+ *
+ * TODO Check if any expression of any unique key isn't nullable, whether in
+ * the table / index or by an outer join.
+ */
+ void
+ make_uniquekeys(PlannerInfo *root, Path *path)
+ {
+ RelOptInfo *rel;
+
+ /*
+ * The unique keys are not interesting if there's no chance to push
+ * aggregation down to base relations / joins.
+ */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Do not accept repeated calls of the function on the same path.
+ */
+ if (path->uniquekeys != NIL)
+ return;
+
+ rel = path->parent;
+
+ /*
+ * Base relations.
+ */
+ if (IsA(path, IndexPath) ||
+ (IsA(path, Path) &&path->pathtype == T_SeqScan) ||
+ IsA(path, AppendPath) ||IsA(path, MergeAppendPath))
+ {
+ ListCell *lc;
+
+ /*
+ * Derive grouping keys from unique indexes.
+ */
+ if (IsA(path, IndexPath) ||IsA(path, Path))
+ {
+ foreach(lc, rel->indexlist)
+ {
+ IndexOptInfo *index = lfirst_node(IndexOptInfo, lc);
+
+ make_uniquekeys_for_unique_index(rel->reltarget, index, path);
+ }
+ }
+ else if (IsA(path, AppendPath) ||IsA(path, MergeAppendPath))
+ make_uniquekeys_for_append_path(root, path);
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(false);
+ #endif
+ return;
+ }
+
+ if (IsA(path, AggPath))
+ {
+ /*
+ * The immediate output of aggregation essentially produces an unique
+ * set of grouping keys.
+ */
+ make_uniquekeys_for_agg_path(path);
+ }
+ else if (IS_JOIN_REL(path->parent))
+ {
+ JoinPath *jpath = (JoinPath *) path;
+ Path *outerpath = jpath->outerjoinpath;
+ Path *innerpath = jpath->innerjoinpath;
+ ListCell *l1;
+
+ /*
+ * Find out if the join produces unique keys for various combinations
+ * of input sets of unique keys.
+ *
+ * TODO Implement heuristic that picks a few most useful sets on each
+ * side, to avoid exponential growth of the uniquekeys list as we
+ * proceed from lower to higher joins. Maybe also discard the
+ * resulting sets containing unique expressions which are not grouping
+ * expressions (and of course which are not aggregates) of this join's
+ * target.
+ */
+ foreach(l1, outerpath->uniquekeys)
+ {
+ Bitmapset *outerset = (Bitmapset *) lfirst(l1);
+ ListCell *l2;
+
+ foreach(l2, innerpath->uniquekeys)
+ {
+ Bitmapset *innerset = (Bitmapset *) lfirst(l2);
+ Bitmapset *joinset;
+
+ /*
+ * Given that unique keys of each input relation are contained
+ * in that relation's target, the union of the key sets should
+ * be contained in the join target.
+ */
+ joinset = bms_union(outerset, innerset);
+
+ /* Add the set to the path. */
+ add_uniquekeys_to_path((Path *) jpath, joinset);
+ }
+ }
+ }
+ #ifdef USE_ASSERT_CHECKING
+
+ /*
+ * TODO Consider other ones, e.g. UniquePath.
+ */
+ else
+ Assert(false);
+ #endif
+ }
+
+ /*
+ * Create a set of positions of expressions in reltarget if the index is
+ * unique and if reltarget contains all the index columns. Add the set to
+ * uniquekeys if identical one is not already there.
+ */
+ static void
+ make_uniquekeys_for_unique_index(PathTarget *reltarget, IndexOptInfo *index,
+ Path *path)
+ {
+ int i;
+ Bitmapset *new_set = NULL;
+
+ /*
+ * Give up if the index does not guarantee uniqueness.
+ */
+ if (!index->unique || !index->immediate ||
+ (index->indpred != NIL && !index->predOK))
+ return;
+
+ /*
+ * For the index path to be acceptable, reltarget must contain all the
+ * index columns.
+ *
+ * reltarget is not supposed to contain non-var expressions, so the index
+ * should neither.
+ */
+ if (index->indexprs != NULL)
+ return;
+
+ for (i = 0; i < index->ncolumns; i++)
+ {
+ int indkey = index->indexkeys[i];
+ ListCell *lc;
+ bool found = false;
+ int j = 0;
+
+ foreach(lc, reltarget->exprs)
+ {
+ Var *var = lfirst_node(Var, lc);
+
+ if (var->varno == index->rel->relid && var->varattno == indkey)
+ {
+ new_set = bms_add_member(new_set, j);
+ found = true;
+ break;
+ }
+
+ j++;
+ }
+
+ /*
+ * If rel needs less than the whole index key then the values of the
+ * columns matched so far can be duplicate.
+ */
+ if (!found)
+ {
+ bms_free(new_set);
+ return;
+ }
+ }
+
+ /*
+ * Add the set to the path, unless it's already there.
+ */
+ add_uniquekeys_to_path(path, new_set);
+ }
+
+ /*
+ * Create uniquekeys for a path that has Aggrefs in its target.
+ * set.
+ *
+ * Besides AggPath, ForeignPath is a known use case for this function.
+ */
+ void
+ make_uniquekeys_for_agg_path(Path *path)
+ {
+ PathTarget *target;
+ ListCell *lc;
+ Bitmapset *keyset = NULL;
+ int i = 0;
+
+ target = path->pathtarget;
+ Assert(target->sortgrouprefs != NULL);
+
+ foreach(lc, target->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, expr);
+
+ if (!IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Generic grouping expression.
+ */
+ keyset = bms_add_member(keyset, i);
+ }
+ }
+ else
+ {
+ Assert(IsA(expr, Var));
+
+ if (target->sortgrouprefs[i] > 0)
+ {
+ /*
+ * Plain Var grouping expression.
+ */
+ keyset = bms_add_member(keyset, i);
+ }
+ else
+ {
+ /*
+ * A column functionally dependent on the GROUP BY clause?
+ */
+ }
+ }
+
+ i++;
+ }
+
+ add_uniquekeys_to_path((Path *) path, keyset);
+ }
+
+ /*
+ * Create uniquekeys for a AppendPath or MergeAppendPath.
+ *
+ * Besides AggPath, ForeignPath is a known use case for this function.
+ */
+ static void
+ make_uniquekeys_for_append_path(PlannerInfo *root, Path *path)
+ {
+ RelOptInfo *rel = path->parent;
+ List *subpaths;
+ Path *subpath;
+ ListCell *l1;
+ bool first = true;
+ List *uniquekeys_common = NIL;
+
+ /*
+ * In addition to the requirement that all subpaths must have uniquekeys
+ * set, the table needs to be partitioned in a specific way. Reject
+ * non-partitioned tables immediately, the other check to follow.
+ */
+ if (!IS_PARTITIONED_REL(rel))
+ return;
+
+ if (IsA(path, AppendPath))
+ subpaths = ((AppendPath *) path)->subpaths;
+ else if (IsA(path, MergeAppendPath))
+ subpaths = ((MergeAppendPath *) path)->subpaths;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(false);
+ #endif
+
+ /*
+ * Check if each subpath has uniquekeys.
+ */
+ foreach(l1, subpaths)
+ {
+ subpath = (Path *) lfirst(l1);
+
+ /*
+ * If any subpath does not have uniquekeys, the whole append path can
+ * have them neither.
+ */
+ if (subpath->uniquekeys == NIL)
+ return;
+ }
+
+ /*
+ * Choose groupkey sets that each subpath has.
+ */
+ foreach(l1, subpaths)
+ {
+ List *common_new = NIL;
+ ListCell *l2;
+
+ subpath = (Path *) lfirst(l1);
+
+ if (uniquekeys_common == NIL)
+ {
+ /* Get the initial list from the first subpath. */
+ uniquekeys_common = subpath->uniquekeys;
+ continue;
+ }
+
+ /*
+ * Remove items missing in the current subpath from uniquekeys_common.
+ */
+ foreach(l2, uniquekeys_common)
+ {
+ Bitmapset *set1 = (Bitmapset *) lfirst(l2);
+ ListCell *l3;
+
+ /*
+ * Does the current subpath contain this set?
+ */
+ foreach(l3, subpath->uniquekeys)
+ {
+ Bitmapset *set2 = (Bitmapset *) lfirst(l3);
+
+ if (bms_equal(set1, set2))
+ {
+ common_new = lappend(common_new, set1);
+ break;
+ }
+ }
+ }
+
+ /*
+ * If there are no sets in common, the next subpaths cannot help us.
+ */
+ if (common_new == NIL)
+ return;
+
+ /*
+ * Adopt the new, possibly reduced list.
+ */
+ uniquekeys_common = common_new;
+ }
+
+ foreach(l1, subpaths)
+ {
+ subpath = (Path *) lfirst(l1);
+
+ /*
+ * The following tests should ideally be performed either on
+ * rel->reltarget or rel->gpi->target outside the iteration of
+ * subpaths. However there's no easy way to find out if path is
+ * grouped or not. So we check the target of the first subpath and
+ * assume that the other ones would pass or fail in the same way:
+ * another subpath target should be the same path target whose
+ * attributes are translated to another child relations.
+ */
+ if (first)
+ {
+ List *partexprs = NIL;
+ List *partexprs_all = NIL;
+ ListCell *l2,
+ *l3;
+ AppendRelInfo **appinfos;
+ int i,
+ nappinfos;
+ bool found;
+ List *texprs = NIL;
+
+ /*
+ * Put all partition expressions into two lists --- one for
+ * non-nullable expressions, one for nullable.
+ */
+ for (i = 0; i < rel->part_scheme->partnatts; i++)
+ {
+ List *sublist;
+
+ sublist = rel->partexprs[i];
+ if (sublist != NIL)
+ {
+ partexprs = list_union(partexprs, sublist);
+ partexprs_all = list_union(partexprs_all,
+ sublist);
+ }
+
+ /*
+ * The nullable expressions should only appear in
+ * partexprs_all.
+ */
+ sublist = rel->nullable_partexprs[i];
+ if (sublist != NIL)
+ partexprs_all = list_union(partexprs_all,
+ sublist);
+ }
+ Assert(partexprs != NIL);
+
+ /*
+ * Translate the partitioning expressions so that they can match
+ * the subpath target.
+ */
+ appinfos =
+ find_appinfos_by_relids(root, subpath->parent->relids,
+ &nappinfos);
+ partexprs = (List *)
+ adjust_appendrel_attrs(root, (Node *) partexprs,
+ nappinfos, appinfos);
+ partexprs_all = (List *)
+ adjust_appendrel_attrs(root, (Node *) partexprs_all,
+ nappinfos, appinfos);
+ pfree(appinfos);
+
+ /*
+ * Since equivalence classes can be used to derive target
+ * expressions, the following checks have to consider ECs too. The
+ * targetlist we construct here contains the original expressions
+ * plus those generated from the appropriate ECs.
+ */
+ foreach(l2, subpath->pathtarget->exprs)
+ {
+ Expr *texpr = (Expr *) lfirst(l2);
+
+ if (IsA(texpr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, texpr);
+
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Aggregate should not appear in any EC.
+ */
+ texprs = lappend(texprs, texpr);
+ continue;
+ }
+
+ /*
+ * The contained expression (i.e. generic grouping
+ * expression) is what we'll search for in the ECs.
+ */
+ texpr = gvar->gvexpr;
+ }
+
+ /*
+ * Search for matching EC.
+ */
+ foreach(l3, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l3);
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *l4;
+ EquivalenceMember *em;
+
+ if (ec->ec_below_outer_join)
+ continue;
+
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(l4, ec->ec_members)
+ {
+ em = lfirst_node(EquivalenceMember, l4);
+
+ if (em->em_nullable_relids)
+ continue;
+
+ if (!bms_is_subset(em->em_relids,
+ subpath->parent->relids))
+ continue;
+
+ if (equal(em->em_expr, texpr))
+ break;
+ }
+
+ /*
+ * texpr not found in the current EC, try the next one.
+ */
+ if (l4 == NULL)
+ continue;
+
+ /*
+ * The EC does match, so add all its members to texprs.
+ */
+ foreach(l4, ec->ec_members)
+ {
+ em = lfirst_node(EquivalenceMember, l4);
+ texprs = lappend(texprs, em->em_expr);
+ }
+
+ /*
+ * No expression is supposed to appear in multiple EC.
+ */
+ continue;
+ }
+ }
+
+ /*
+ * To ensure that no output row can be produced by multiple
+ * partitions, check that:
+ *
+ * (a) at least one output column is a partitioning expression.
+ * Thus no output row of this path can appear in multiple
+ * partitions.
+ *
+ * Only search in the non-nullable partitioning expressions
+ * because NULL value can be generated by any partition.
+ */
+ found = false;
+ foreach(l2, texprs)
+ {
+ Expr *texpr = (Expr *) lfirst(l2);
+
+ /*
+ * Try to find the matching partitioning expression.
+ */
+ foreach(l3, partexprs)
+ {
+ Expr *pexpr = lfirst(l3);
+
+ if (equal(texpr, pexpr))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ if (found)
+ break;
+ }
+ if (!found)
+ {
+ list_free(uniquekeys_common);
+ return;
+ }
+
+ /*
+ * (b) there's no partitioning expression not contained in the
+ * target. If there was some, then the partitioning expression
+ * found above could appear in multiple partitions.
+ *
+ * Even nullable partition key makes harm here, so consider all
+ * partition keys this time.
+ */
+ foreach(l2, partexprs_all)
+ {
+ Expr *pexpr = lfirst(l2);
+
+ found = false;
+ foreach(l3, texprs)
+ {
+ Expr *texpr = (Expr *) lfirst(l3);
+
+ if (equal(pexpr, texpr))
+ {
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ {
+ list_free(uniquekeys_common);
+ return;
+ }
+ }
+ list_free(texprs);
+
+ /* Do not repeat the checks for the other subpaths. */
+ first = false;
+ }
+ }
+
+ /* All the checks passed. */
+ path->uniquekeys = uniquekeys_common;
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 3445c98..557d934
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** apply_pathtarget_labeling_to_tlist(List
*** 872,884 ****
/*
* Replace each "grouped var" in the source targetlist with the original
! * expression.
*
* TODO Think of more suitable name undo_grouped_var_substitutions? Also note
* that the partial aggregate is retrieved, not the original one.
*/
List *
! restore_grouping_expressions(PlannerInfo *root, List *src)
{
List *result = NIL;
ListCell *l;
--- 872,885 ----
/*
* Replace each "grouped var" in the source targetlist with the original
! * expression. If agg_partial is true, restore the partial aggregate as
! * opposed to the original "simple" one.
*
* TODO Think of more suitable name undo_grouped_var_substitutions? Also note
* that the partial aggregate is retrieved, not the original one.
*/
List *
! restore_grouping_expressions(PlannerInfo *root, List *src, bool agg_partial)
{
List *result = NIL;
ListCell *l;
*************** restore_grouping_expressions(PlannerInfo
*** 898,913 ****
gvar = castNode(GroupedVar, te->expr);
if (IsA(gvar->gvexpr, Aggref))
{
! /*
! * Partial aggregate should appear in the targetlist so that
! * it looks as if convert_combining_aggrefs arranged it.
! */
! expr_new = (Expr *) gvar->agg_partial;
}
else
expr_new = gvar->gvexpr;
}
if (expr_new != NULL)
{
te_new = flatCopyTargetEntry(te);
--- 899,938 ----
gvar = castNode(GroupedVar, te->expr);
if (IsA(gvar->gvexpr, Aggref))
{
! if (agg_partial)
! {
! /*
! * Partial aggregate should appear in the targetlist so
! * that it looks as if convert_combining_aggrefs arranged
! * it.
! */
! expr_new = (Expr *) gvar->agg_partial;
! }
! else
! expr_new = gvar->gvexpr;
}
else
expr_new = gvar->gvexpr;
}
+ /*
+ * Alternatively --- if the query generates an unique set of grouping
+ * keys --- the targetlist may contain aggfinalfn referencing the
+ * partial aggregate. We replace this with the original
+ * (AGGSPLIT_SIMPLE) aggregate so that set_upper_references find a
+ * match.
+ */
+ else if (IS_AGGFINALFN_STANDALONE(te->expr))
+ {
+ FuncExpr *fexpr = castNode(FuncExpr, te->expr);
+ GroupedVar *gvar = linitial_node(GroupedVar, fexpr->args);
+ Aggref *aggref = castNode(Aggref, gvar->gvexpr);
+
+ Assert(fexpr->funcid == aggref->aggfinalfn);
+
+ expr_new = (Expr *) aggref;
+ }
+
if (expr_new != NULL)
{
te_new = flatCopyTargetEntry(te);
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
new file mode 100644
index 071681a..21ce684
*** a/src/backend/parser/parse_func.c
--- b/src/backend/parser/parse_func.c
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 99,104 ****
--- 99,105 ----
FuncDetailCode fdresult;
char aggkind = 0;
Oid aggcombinefn = InvalidOid;
+ Oid aggfinalfn = InvalidOid;
ParseCallbackState pcbstate;
/*
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 323,328 ****
--- 324,330 ----
classForm = (Form_pg_aggregate) GETSTRUCT(tup);
aggkind = classForm->aggkind;
aggcombinefn = classForm->aggcombinefn;
+ aggfinalfn = classForm->aggfinalfn;
catDirectArgs = classForm->aggnumdirectargs;
ReleaseSysCache(tup);
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 669,674 ****
--- 671,677 ----
aggref->aggvariadic = func_variadic;
aggref->aggkind = aggkind;
aggref->aggcombinefn = aggcombinefn;
+ aggref->aggfinalfn = aggfinalfn;
/* agglevelsup will be set by transformAggregateCall */
aggref->aggsplit = AGGSPLIT_SIMPLE; /* planner might change this */
aggref->location = location;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index 5251f98..5076e01
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct Aggref
*** 297,302 ****
--- 297,303 ----
Oid inputcollid; /* OID of collation that function should use */
Oid aggtranstype; /* type Oid of aggregate's transition value */
Oid aggcombinefn; /* combine function (see pg_aggregate.h) */
+ Oid aggfinalfn; /* final function (see pg_aggregate.h) */
List *aggargtypes; /* type Oids of direct and aggregated args */
List *aggdirectargs; /* direct arguments, if an ordered-set agg */
List *args; /* aggregated arguments and sort expressions */
*************** typedef struct FuncExpr
*** 461,466 ****
--- 462,475 ----
} FuncExpr;
/*
+ * Is the expression an aggfinalfn function that can replace the final
+ * aggregation in some cases?
+ */
+ #define IS_AGGFINALFN_STANDALONE(e) \
+ (IsA((e), FuncExpr) && list_length(((FuncExpr *) (e))->args) == 1 &&\
+ IsA(linitial(((FuncExpr *) (e))->args), GroupedVar))
+
+ /*
* NamedArgExpr - a named argument of a function
*
* This node type can only appear in the args list of a FuncCall or FuncExpr
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index e270020..49b1fb4
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct ParamPathInfo
*** 1043,1048 ****
--- 1043,1051 ----
* are only added to path target of a path which should generate input for
* partial aggregation.
*
+ * "sortgroupclauses" is a list of grouping clauses that the relation does
+ * have in its targetlist but the query does not.
+ *
* (Two grouped paths cannot be joined in general because grouping of one side
* of the join essentially reduces occurrence of groups of the other side in
* the input of the final aggregation.)
*************** typedef struct GroupedPathInfo
*** 1086,1091 ****
--- 1089,1098 ----
*
* "pathkeys" is a List of PathKey nodes (see above), describing the sort
* ordering of the path's output rows.
+ *
+ * "groupkeys" is a List of Bitmapset objects, each pointing at a set of
+ * expressions of "pathtarget" whose values within the path output are
+ * distinct.
*/
typedef struct Path
{
*************** typedef struct Path
*** 1109,1114 ****
--- 1116,1125 ----
List *pathkeys; /* sort ordering of path's output */
/* pathkeys is a List of PathKey nodes; see above */
+
+ List *uniquekeys; /* list of bitmapsets where each set contains
+ * positions of unique expressions within
+ * pathtarget. */
} Path;
/* Macro for extracting a path's parameterization relids; beware double eval */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index c825d7c..6c2c0f5
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern AggPath *create_partial_agg_sorte
*** 206,219 ****
List **group_clauses,
List **group_exprs,
List **agg_exprs,
! double input_rows);
extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
Path *subpath,
bool first_call,
List **group_clauses,
List **group_exprs,
List **agg_exprs,
! double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
--- 206,221 ----
List **group_clauses,
List **group_exprs,
List **agg_exprs,
! double input_rows,
! bool parallel);
extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
Path *subpath,
bool first_call,
List **group_clauses,
List **group_exprs,
List **agg_exprs,
! double input_rows,
! bool parallel);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern Path *reparameterize_path(Planner
*** 274,279 ****
--- 276,283 ----
double loop_count);
extern Path *reparameterize_path_by_child(PlannerInfo *root, Path *path,
RelOptInfo *child_rel);
+ extern void make_uniquekeys(PlannerInfo *root, Path *path);
+ extern void make_uniquekeys_for_agg_path(Path *path);
/*
* prototypes for relnode.c
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 60e6ecd..15e94f9
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern bool has_useful_pathkeys(PlannerI
*** 239,243 ****
extern PathKey *make_canonical_pathkey(PlannerInfo *root,
EquivalenceClass *eclass, Oid opfamily,
int strategy, bool nulls_first);
!
#endif /* PATHS_H */
--- 239,244 ----
extern PathKey *make_canonical_pathkey(PlannerInfo *root,
EquivalenceClass *eclass, Oid opfamily,
int strategy, bool nulls_first);
! extern void add_uniquekeys_to_path(Path *path, Bitmapset *new_set);
! extern bool match_path_to_group_pathkeys(PlannerInfo *root, Path *path);
#endif /* PATHS_H */
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 29d1b46..60b8182
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern void split_pathtarget_at_srfs(Pla
*** 71,77 ****
/* TODO Find the best location (position and in some cases even file) for the
* following ones. */
! extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
--- 71,78 ----
/* TODO Find the best location (position and in some cases even file) for the
* following ones. */
! extern List *restore_grouping_expressions(PlannerInfo *root, List *src,
! bool partial);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index cac70a0..72e3a80
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
*************** SET cpu_tuple_cost TO 0.1;
*** 28,44 ****
EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
! QUERY PLAN
! -------------------------------------------------------------------------------------
! Finalize HashAggregate
! Group Key: p.i
! -> Nested Loop
! -> Partial HashAggregate
! Group Key: c1.parent
! -> Seq Scan on agg_pushdown_child1 c1
! -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
! Index Cond: (i = c1.parent)
! (8 rows)
-- Scan index on agg_pushdown_child1(parent) column and partially aggregate
-- the result using AGG_SORTED strategy.
--- 28,42 ----
EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
! QUERY PLAN
! -------------------------------------------------------------------------------
! Nested Loop
! -> Partial HashAggregate
! Group Key: c1.parent
! -> Seq Scan on agg_pushdown_child1 c1
! -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
! Index Cond: (i = c1.parent)
! (6 rows)
-- Scan index on agg_pushdown_child1(parent) column and partially aggregate
-- the result using AGG_SORTED strategy.
*************** SET enable_seqscan TO off;
*** 46,62 ****
EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
! QUERY PLAN
! ---------------------------------------------------------------------------------------------
! Finalize GroupAggregate
! Group Key: p.i
! -> Nested Loop
! -> Partial GroupAggregate
! Group Key: c1.parent
! -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
! -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
! Index Cond: (i = c1.parent)
! (8 rows)
SET enable_seqscan TO on;
-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
--- 44,58 ----
EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
! QUERY PLAN
! ---------------------------------------------------------------------------------------
! Nested Loop
! -> Partial GroupAggregate
! Group Key: c1.parent
! -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
! -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
! Index Cond: (i = c1.parent)
! (6 rows)
SET enable_seqscan TO on;
-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
agg_pushdown_v4/08_fdw.diff
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
new file mode 100644
index 9552376..493c7c3
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
*************** PG_FUNCTION_INFO_V1(file_fdw_validator);
*** 117,126 ****
*/
static void fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static void fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static ForeignScan *fileGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
--- 117,128 ----
*/
static void fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static void fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static ForeignScan *fileGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
*************** get_file_fdw_attribute_options(Oid relid
*** 487,497 ****
static void
fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
FileFdwPlanState *fdw_private;
/*
* Fetch options. We only need filename (or program) at this point, but
* we might as well get everything and not need to re-fetch it later in
* planning.
--- 489,507 ----
static void
fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
FileFdwPlanState *fdw_private;
/*
+ * XXX Grouping at relation level is possible but this FDW does not
+ * implement it yet.
+ */
+ if (grouped)
+ return;
+
+ /*
* Fetch options. We only need filename (or program) at this point, but
* we might as well get everything and not need to re-fetch it later in
* planning.
*************** fileGetForeignRelSize(PlannerInfo *root,
*** 518,524 ****
static void
fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
FileFdwPlanState *fdw_private = (FileFdwPlanState *) baserel->fdw_private;
Cost startup_cost;
--- 528,535 ----
static void
fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
FileFdwPlanState *fdw_private = (FileFdwPlanState *) baserel->fdw_private;
Cost startup_cost;
*************** fileGetForeignPaths(PlannerInfo *root,
*** 526,531 ****
--- 537,549 ----
List *columns;
List *coptions = NIL;
+ /*
+ * XXX Grouping at relation level is possible but this FDW does not
+ * implement it yet.
+ */
+ if (grouped)
+ return;
+
/* Decide whether to selectively perform binary conversion */
if (check_selective_binary_conversion(baserel,
foreigntableid,
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
new file mode 100644
index 0876589..b67f6fd
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
*************** typedef struct deparse_expr_cxt
*** 103,108 ****
--- 103,109 ----
* a base relation. */
StringInfo buf; /* output buffer to append to */
List **params_list; /* exprs that will become remote Params */
+ List *tlist;
} deparse_expr_cxt;
#define REL_ALIAS_PREFIX "r"
*************** static void deparseTargetList(StringInfo
*** 133,138 ****
--- 134,140 ----
bool qualify_col,
List **retrieved_attrs);
static void deparseExplicitTargetList(List *tlist, List **retrieved_attrs,
+ bool grouped,
deparse_expr_cxt *context);
static void deparseSubqueryTargetList(deparse_expr_cxt *context);
static void deparseReturningList(StringInfo buf, PlannerInfo *root,
*************** static void printRemoteParam(int paramin
*** 163,171 ****
static void printRemotePlaceholder(Oid paramtype, int32 paramtypmod,
deparse_expr_cxt *context);
static void deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context);
static void deparseLockingClause(deparse_expr_cxt *context);
! static void appendOrderByClause(List *pathkeys, deparse_expr_cxt *context);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
static void deparseFromExprForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *joinrel, bool use_alias, List **params_list);
--- 165,174 ----
static void printRemotePlaceholder(Oid paramtype, int32 paramtypmod,
deparse_expr_cxt *context);
static void deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context, bool grouped);
static void deparseLockingClause(deparse_expr_cxt *context);
! static void appendOrderByClause(List *pathkeys, deparse_expr_cxt *context,
! bool grouped);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
static void deparseFromExprForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *joinrel, bool use_alias, List **params_list);
*************** static void deparseAggref(Aggref *node,
*** 177,185 ****
static void appendGroupByClause(List *tlist, deparse_expr_cxt *context);
static void appendAggOrderBy(List *orderList, List *targetList,
deparse_expr_cxt *context);
! static void appendFunctionName(Oid funcid, deparse_expr_cxt *context);
static Node *deparseSortGroupClause(Index ref, List *tlist,
deparse_expr_cxt *context);
/*
* Helper functions
--- 180,189 ----
static void appendGroupByClause(List *tlist, deparse_expr_cxt *context);
static void appendAggOrderBy(List *orderList, List *targetList,
deparse_expr_cxt *context);
! static void appendFunctionName(Oid funcid, StringInfo buf, char *schemaname);
static Node *deparseSortGroupClause(Index ref, List *tlist,
deparse_expr_cxt *context);
+ static void deparseSortGroupExpr(Expr *expr, deparse_expr_cxt *context);
/*
* Helper functions
*************** foreign_expr_walker(Node *node,
*** 365,370 ****
--- 369,382 ----
}
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ Assert(!IsA(gvar->gvexpr, Aggref));
+ return foreign_expr_walker((Node *) gvar->gvexpr, glob_cxt,
+ outer_cxt);
+ }
case T_Const:
{
Const *c = (Const *) node;
*************** foreign_expr_walker(Node *node,
*** 676,687 ****
Aggref *agg = (Aggref *) node;
ListCell *lc;
! /* Not safe to pushdown when not in grouping context */
! if (!IS_UPPER_REL(glob_cxt->foreignrel))
! return false;
!
! /* Only non-split aggregates are pushable. */
! if (agg->aggsplit != AGGSPLIT_SIMPLE)
return false;
/* As usual, it must be shippable. */
--- 688,702 ----
Aggref *agg = (Aggref *) node;
ListCell *lc;
! /*
! * Only non-split aggregates are pushable.
! *
! * XXX Like above, AGGSPLIT_SIMPLE shouldn't appear here if
! * the aggregation of the whole upper relation gets abandoned.
! * (Check all occurrences of AGGSPLIT_SIMPLE.)
! */
! if (agg->aggsplit != AGGSPLIT_SIMPLE &&
! agg->aggsplit != AGGSPLIT_INITIAL_SERIAL)
return false;
/* As usual, it must be shippable. */
*************** extern void
*** 925,935 ****
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
List *tlist, List *remote_conds, List *pathkeys,
bool is_subquery, List **retrieved_attrs,
! List **params_list)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
List *quals;
/*
* We handle relations for foreign tables, joins between those and upper
--- 940,951 ----
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
List *tlist, List *remote_conds, List *pathkeys,
bool is_subquery, List **retrieved_attrs,
! List **params_list, bool grouping)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
List *quals;
+ bool grouped;
/*
* We handle relations for foreign tables, joins between those and upper
*************** deparseSelectStmtForRel(StringInfo buf,
*** 943,951 ****
context.foreignrel = rel;
context.scanrel = IS_UPPER_REL(rel) ? fpinfo->outerrel : rel;
context.params_list = params_list;
/* Construct SELECT clause */
! deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context);
/*
* For upper relations, the WHERE clause is built from the remote
--- 959,968 ----
context.foreignrel = rel;
context.scanrel = IS_UPPER_REL(rel) ? fpinfo->outerrel : rel;
context.params_list = params_list;
+ context.tlist = tlist;
/* Construct SELECT clause */
! deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context, grouping);
/*
* For upper relations, the WHERE clause is built from the remote
*************** deparseSelectStmtForRel(StringInfo buf,
*** 965,977 ****
/* Construct FROM and WHERE clauses */
deparseFromExpr(quals, &context);
! if (IS_UPPER_REL(rel))
{
/* Append GROUP BY clause */
appendGroupByClause(tlist, &context);
! /* Append HAVING clause */
! if (remote_conds)
{
appendStringInfoString(buf, " HAVING ");
appendConditions(remote_conds, &context);
--- 982,1011 ----
/* Construct FROM and WHERE clauses */
deparseFromExpr(quals, &context);
! grouped = IS_UPPER_REL(rel) || grouping;
! if (grouped)
{
/* Append GROUP BY clause */
appendGroupByClause(tlist, &context);
! /*
! * Append HAVING clause.
! *
! * XXX As the "partition-wise join" patch will probably be committed
! * to PG core earlier than the "aggregate push-down" patch, we don't
! * add HAVING clause to query that represents remote simple relation
! * or join. The point is that the HAVING clause can't be evaluated
! * below the final aggregation node, and the final aggregation is
! * essentially local.
! *
! * To surpass this limitation, the FDW would have to know whether the
! * relation / join is a partition or the whole table. In case it's
! * partition, HAVING is only legal if the query does not use any other
! * partition. If we eventually can add the HAVING clause to scan /
! * join remote queries, we also have to change the API so that WHERE /
! * ON conditions are passed separate from the HAVING expression.
! */
! if (IS_UPPER_REL(rel) && remote_conds)
{
appendStringInfoString(buf, " HAVING ");
appendConditions(remote_conds, &context);
*************** deparseSelectStmtForRel(StringInfo buf,
*** 980,986 ****
/* Add ORDER BY clause if we found any useful pathkeys */
if (pathkeys)
! appendOrderByClause(pathkeys, &context);
/* Add any necessary FOR UPDATE/SHARE. */
deparseLockingClause(&context);
--- 1014,1020 ----
/* Add ORDER BY clause if we found any useful pathkeys */
if (pathkeys)
! appendOrderByClause(pathkeys, &context, grouped);
/* Add any necessary FOR UPDATE/SHARE. */
deparseLockingClause(&context);
*************** deparseSelectStmtForRel(StringInfo buf,
*** 1001,1007 ****
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
--- 1035,1041 ----
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context, bool grouping)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
*************** deparseSelectSql(List *tlist, bool is_su
*** 1028,1034 ****
* For a join or upper relation the input tlist gives the list of
* columns required to be fetched from the foreign server.
*/
! deparseExplicitTargetList(tlist, retrieved_attrs, context);
}
else
{
--- 1062,1077 ----
* For a join or upper relation the input tlist gives the list of
* columns required to be fetched from the foreign server.
*/
! deparseExplicitTargetList(tlist, retrieved_attrs, grouping, context);
! }
! else if (grouping && IS_SIMPLE_REL(foreignrel) && foreignrel->gpi != NULL)
! {
! /*
! * An explicit targetlist is also passed if aggregation should take
! * place in the remote database.
! */
! deparseExplicitTargetList(fpinfo->grouped_tlist, retrieved_attrs,
! grouping, context);
}
else
{
*************** get_jointype_name(JoinType jointype)
*** 1342,1348 ****
* from 1. It has same number of entries as tlist.
*/
static void
! deparseExplicitTargetList(List *tlist, List **retrieved_attrs,
deparse_expr_cxt *context)
{
ListCell *lc;
--- 1385,1391 ----
* from 1. It has same number of entries as tlist.
*/
static void
! deparseExplicitTargetList(List *tlist, List **retrieved_attrs, bool grouping,
deparse_expr_cxt *context)
{
ListCell *lc;
*************** deparseRangeTblRef(StringInfo buf, Plann
*** 1505,1511 ****
appendStringInfoChar(buf, '(');
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
fpinfo->remote_conds, NIL, true,
! &retrieved_attrs, params_list);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
--- 1548,1554 ----
appendStringInfoChar(buf, '(');
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
fpinfo->remote_conds, NIL, true,
! &retrieved_attrs, params_list, false);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
*************** deparseExpr(Expr *node, deparse_expr_cxt
*** 2126,2131 ****
--- 2169,2177 ----
case T_Var:
deparseVar((Var *) node, context);
break;
+ case T_GroupedVar:
+ deparseExpr((Expr *) ((GroupedVar *) node)->gvexpr, context);
+ break;
case T_Const:
deparseConst((Const *) node, context, 0);
break;
*************** deparseFuncExpr(FuncExpr *node, deparse_
*** 2469,2475 ****
/*
* Normal function: display as proname(args).
*/
! appendFunctionName(node->funcid, context);
appendStringInfoChar(buf, '(');
/* ... and all the arguments */
--- 2515,2521 ----
/*
* Normal function: display as proname(args).
*/
! appendFunctionName(node->funcid, context->buf, NULL);
appendStringInfoChar(buf, '(');
/* ... and all the arguments */
*************** deparseAggref(Aggref *node, deparse_expr
*** 2747,2761 ****
{
StringInfo buf = context->buf;
bool use_variadic;
/* Only basic, non-split aggregation accepted. */
! Assert(node->aggsplit == AGGSPLIT_SIMPLE);
/* Check if need to print VARIADIC (cf. ruleutils.c) */
use_variadic = node->aggvariadic;
/* Find aggregate name from aggfnoid which is a pg_proc entry */
! appendFunctionName(node->aggfnoid, context);
appendStringInfoChar(buf, '(');
/* Add DISTINCT */
--- 2793,2876 ----
{
StringInfo buf = context->buf;
bool use_variadic;
+ ListCell *lc;
+ bool use_core_aggregate = true;
+ char *schemaname = NULL;
/* Only basic, non-split aggregation accepted. */
! Assert(node->aggsplit == AGGSPLIT_SIMPLE ||
! node->aggsplit == AGGSPLIT_INITIAL_SERIAL);
/* Check if need to print VARIADIC (cf. ruleutils.c) */
use_variadic = node->aggvariadic;
+ if (context->root->grouped_var_list != NIL)
+ {
+ /*
+ * The core aggregates (i.e. those in postgres catalog) do or do not
+ * perform finalization on both remote and local side. But sometimes
+ * we need the same aggregate to run w/o finalization on the remote
+ * node and with the finalization locally.
+ *
+ * Since the current Aggref is partial, it essentially returns the
+ * transient type. We need to find the original aggregate.
+ */
+ foreach(lc, context->root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi;
+ Aggref *orig;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ /* GroupedVar currently represents only aggregates. */
+ orig = castNode(Aggref, gvi->gvexpr);
+
+ if (orig->aggfnoid == node->aggfnoid)
+ {
+ /*
+ * The core aggregate can only be useful if it does not
+ * perform finalization.
+ */
+ if (!OidIsValid(orig->aggfinalfn))
+ {
+ use_core_aggregate = true;
+
+ /*
+ * If there's no finalization, the aggregate should return
+ * value of the internal type.
+ */
+ Assert(orig->aggtype == orig->aggtranstype);
+ }
+ else
+ use_core_aggregate = false;
+
+ /*
+ * If there are multiple aggregates of the same aggfnoid in
+ * the query, they should all have the same values of the
+ * fields we're going to check. So just use the first one we
+ * find.
+ */
+ break;
+ }
+ }
+
+ /* Must have found the original aggregate in the list. */
+ Assert(lc != NULL);
+ }
+
+ /*
+ * TODO
+ *
+ * 1. Make the schema name configurable as FDW option.
+ *
+ * 2. Implement the partial aggregates. These don't fit into postgres_fdw
+ * because there's no reason to install postgres_fdw on the remote node.
+ * Separate extension (contrib module?) is probably needed.
+ */
+ if (!use_core_aggregate)
+ schemaname = "partial";
+
/* Find aggregate name from aggfnoid which is a pg_proc entry */
! appendFunctionName(node->aggfnoid, context->buf, schemaname);
appendStringInfoChar(buf, '(');
/* Add DISTINCT */
*************** deparseAggref(Aggref *node, deparse_expr
*** 2829,2834 ****
--- 2944,2965 ----
}
appendStringInfoChar(buf, ')');
+
+ /*
+ * If special (non-core) aggregate is needed on the remote node, it can
+ * return data type for which postgres does not have textual format. cast
+ * to bytea seems to be an universal solution in such a case.
+ *
+ * TODO
+ *
+ * Consider checking if aggserialfn exists --- aggtranstype is used now
+ * because the info is easier to access. Also consider a requirement that
+ * the "remote aggregates" must return bytea, unless they return a
+ * primitive type. Thus we wouldn't have to deal with the cast here at
+ * all.
+ */
+ if (schemaname != NULL && node->aggtranstype == INTERNALOID)
+ appendStringInfoString(buf, "::bytea");
}
/*
*************** appendGroupByClause(List *tlist, deparse
*** 2952,2966 ****
*/
Assert(!query->groupingSets);
! foreach(lc, query->groupClause)
{
! SortGroupClause *grp = (SortGroupClause *) lfirst(lc);
! if (!first)
! appendStringInfoString(buf, ", ");
! first = false;
! deparseSortGroupClause(grp->tleSortGroupRef, tlist, context);
}
}
--- 3083,3110 ----
*/
Assert(!query->groupingSets);
! /*
! * Due to aggregation push-down (which includes join aggregation on the
! * remote server) we should expect multiple TLEs containing different
! * expressions but having the same sortgroupref. The typical case is that
! * an equality join clause receives the input value (var) from each side
! * of a join, and each of these vars is needed above the join for a
! * different reason. That's why we don't use query->groupClause here.
! *
! * TODO Consider using get_grouping_expressions().
! */
! foreach(lc, tlist)
{
! TargetEntry *te = lfirst_node(TargetEntry, lc);
! if (te->ressortgroupref > 0)
! {
! if (!first)
! appendStringInfoString(buf, ", ");
! first = false;
! deparseSortGroupExpr(te->expr, context);
! }
}
}
*************** appendGroupByClause(List *tlist, deparse
*** 2970,2976 ****
* base relation are obtained and deparsed.
*/
static void
! appendOrderByClause(List *pathkeys, deparse_expr_cxt *context)
{
ListCell *lcell;
int nestlevel;
--- 3114,3120 ----
* base relation are obtained and deparsed.
*/
static void
! appendOrderByClause(List *pathkeys, deparse_expr_cxt *context, bool grouped)
{
ListCell *lcell;
int nestlevel;
*************** appendOrderByClause(List *pathkeys, depa
*** 2981,2993 ****
/* Make sure any constants in the exprs are printed portably */
nestlevel = set_transmission_modes();
appendStringInfoString(buf, " ORDER BY");
foreach(lcell, pathkeys)
{
PathKey *pathkey = lfirst(lcell);
! Expr *em_expr;
! em_expr = find_em_expr_for_rel(pathkey->pk_eclass, baserel);
Assert(em_expr != NULL);
appendStringInfoString(buf, delim);
--- 3125,3215 ----
/* Make sure any constants in the exprs are printed portably */
nestlevel = set_transmission_modes();
+ /*
+ * If there's no GROUP BY clause, the query cannot be sorted by aggregates.
+ */
+ if (!grouped)
+ {
+ ListCell *l1;
+
+ foreach(l1, pathkeys)
+ {
+ PathKey *pathkey = lfirst_node(PathKey, l1);
+ EquivalenceClass *ec = pathkey->pk_eclass;
+ ListCell *l2;
+
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+
+ if (IsA(em->em_expr, Aggref))
+ return;
+ }
+ }
+ }
+
appendStringInfoString(buf, " ORDER BY");
foreach(lcell, pathkeys)
{
PathKey *pathkey = lfirst(lcell);
! Expr *em_expr = NULL;
! if (grouped)
! {
! ListCell *l1;
!
! /*
! * Since the targetlist can contain EC-derived expressions, extra
! * effort is needed to ensure that ORDER BY clause references
! * expression actually contained in the targetlist.
! */
! foreach(l1, pathkey->pk_eclass->ec_members)
! {
! EquivalenceMember *em = lfirst_node(EquivalenceMember, l1);
!
! if (bms_is_subset(em->em_relids, baserel->relids))
! {
! ListCell *l2;
!
! /*
! * If there is more than one equivalence member whose Vars
! * are taken entirely from this relation, we only accept
! * one that exists in the targetlist.
! */
! Assert(context->tlist != NIL);
! foreach(l2, context->tlist)
! {
! TargetEntry *te = lfirst_node(TargetEntry, l2);
! Expr *expr = te->expr;
!
! if (IsA(expr, Aggref) && IsA(em->em_expr, Aggref))
! {
! Aggref *aggref = (Aggref *) copyObject(expr);
! Aggref *aggref_em = (Aggref *) em->em_expr;
!
! /*
! * The EC members are simple aggregates, so adjust
! * the target expression to make match possible.
! */
! Assert(aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL);
! aggref->aggsplit = AGGSPLIT_SIMPLE;
! aggref->aggtype = aggref_em->aggtype;
! expr = (Expr *) aggref;
! }
!
! if (equal(expr, em->em_expr))
! {
! em_expr = em->em_expr;
! break;
! }
! }
! if (em_expr != NULL)
! break;
! }
! }
! }
! else
! em_expr = find_em_expr_for_rel(pathkey->pk_eclass, baserel);
Assert(em_expr != NULL);
appendStringInfoString(buf, delim);
*************** appendOrderByClause(List *pathkeys, depa
*** 3012,3020 ****
* Deparses function name from given function oid.
*/
static void
! appendFunctionName(Oid funcid, deparse_expr_cxt *context)
{
- StringInfo buf = context->buf;
HeapTuple proctup;
Form_pg_proc procform;
const char *proname;
--- 3234,3241 ----
* Deparses function name from given function oid.
*/
static void
! appendFunctionName(Oid funcid, StringInfo buf, char *schemaname)
{
HeapTuple proctup;
Form_pg_proc procform;
const char *proname;
*************** appendFunctionName(Oid funcid, deparse_e
*** 3025,3035 ****
procform = (Form_pg_proc) GETSTRUCT(proctup);
/* Print schema name only if it's not pg_catalog */
! if (procform->pronamespace != PG_CATALOG_NAMESPACE)
{
! const char *schemaname;
!
! schemaname = get_namespace_name(procform->pronamespace);
appendStringInfo(buf, "%s.", quote_identifier(schemaname));
}
--- 3246,3256 ----
procform = (Form_pg_proc) GETSTRUCT(proctup);
/* Print schema name only if it's not pg_catalog */
! if (procform->pronamespace != PG_CATALOG_NAMESPACE ||
! schemaname != NULL)
{
! if (schemaname == NULL)
! schemaname = get_namespace_name(procform->pronamespace);
appendStringInfo(buf, "%s.", quote_identifier(schemaname));
}
*************** appendFunctionName(Oid funcid, deparse_e
*** 3049,3061 ****
static Node *
deparseSortGroupClause(Index ref, List *tlist, deparse_expr_cxt *context)
{
- StringInfo buf = context->buf;
TargetEntry *tle;
Expr *expr;
tle = get_sortgroupref_tle(ref, tlist);
expr = tle->expr;
if (expr && IsA(expr, Const))
{
/*
--- 3270,3291 ----
static Node *
deparseSortGroupClause(Index ref, List *tlist, deparse_expr_cxt *context)
{
TargetEntry *tle;
Expr *expr;
tle = get_sortgroupref_tle(ref, tlist);
expr = tle->expr;
+ deparseSortGroupExpr(expr, context);
+
+ return (Node *) expr;
+ }
+
+ static void
+ deparseSortGroupExpr(Expr *expr, deparse_expr_cxt *context)
+ {
+ StringInfo buf = context->buf;
+
if (expr && IsA(expr, Const))
{
/*
*************** deparseSortGroupClause(Index ref, List *
*** 3074,3081 ****
deparseExpr(expr, context);
appendStringInfoChar(buf, ')');
}
-
- return (Node *) expr;
}
--- 3304,3309 ----
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
new file mode 100644
index 459c6b9..a434fe9
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
*************** PG_FUNCTION_INFO_V1(postgres_fdw_handler
*** 273,282 ****
*/
static void postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static void postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static ForeignScan *postgresGetForeignPlan(PlannerInfo *root,
RelOptInfo *foreignrel,
Oid foreigntableid,
--- 273,283 ----
*/
static void postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid, bool grouped);
static void postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static ForeignScan *postgresGetForeignPlan(PlannerInfo *root,
RelOptInfo *foreignrel,
Oid foreigntableid,
*************** static void postgresGetForeignJoinPaths(
*** 341,347 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static bool postgresRecheckForeignScan(ForeignScanState *node,
TupleTableSlot *slot);
static void postgresGetForeignUpperPaths(PlannerInfo *root,
--- 342,349 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped);
static bool postgresRecheckForeignScan(ForeignScanState *node,
TupleTableSlot *slot);
static void postgresGetForeignUpperPaths(PlannerInfo *root,
*************** static void postgresGetForeignUpperPaths
*** 352,363 ****
/*
* Helper functions
*/
static void estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *baserel,
List *join_conds,
List *pathkeys,
! double *p_rows, int *p_width,
! Cost *p_startup_cost, Cost *p_total_cost);
static void get_remote_estimate(const char *sql,
PGconn *conn,
double *rows,
--- 354,367 ----
/*
* Helper functions
*/
+ static void init_pgfdw_baserel_info(PlannerInfo *root, RelOptInfo *rel,
+ Oid foreigntableid, bool grouped);
static void estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *baserel,
List *join_conds,
List *pathkeys,
! EstimateInfo * estimates,
! bool grouped);
static void get_remote_estimate(const char *sql,
PGconn *conn,
double *rows,
*************** static HeapTuple make_tuple_from_result_
*** 404,416 ****
static void conversion_error_callback(void *arg);
static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
! RelOptInfo *rel);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
static void add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path);
static void add_foreign_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
RelOptInfo *grouped_rel);
--- 408,423 ----
static void conversion_error_callback(void *arg);
static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra, bool grouped);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel);
+ static List *create_remote_agg_tlist(PlannerInfo *root,
+ RelOptInfo *grouped_rel,
+ PathTarget *grouping_target);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
! RelOptInfo *rel, bool grouped);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
static void add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path, bool grouped);
static void add_foreign_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
RelOptInfo *grouped_rel);
*************** postgres_fdw_handler(PG_FUNCTION_ARGS)
*** 485,567 ****
static void
postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
PgFdwRelationInfo *fpinfo;
! ListCell *lc;
! RangeTblEntry *rte = planner_rt_fetch(baserel->relid, root);
! const char *namespace;
! const char *relname;
! const char *refname;
/*
* We use PgFdwRelationInfo to pass various information to subsequent
* functions.
*/
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! baserel->fdw_private = (void *) fpinfo;
!
! /* Base foreign tables need to be pushed down always. */
! fpinfo->pushdown_safe = true;
!
! /* Look up foreign-table catalog info. */
! fpinfo->table = GetForeignTable(foreigntableid);
! fpinfo->server = GetForeignServer(fpinfo->table->serverid);
!
! /*
! * Extract user-settable option values. Note that per-table setting of
! * use_remote_estimate overrides per-server setting.
! */
! fpinfo->use_remote_estimate = false;
! fpinfo->fdw_startup_cost = DEFAULT_FDW_STARTUP_COST;
! fpinfo->fdw_tuple_cost = DEFAULT_FDW_TUPLE_COST;
! fpinfo->shippable_extensions = NIL;
! fpinfo->fetch_size = 100;
!
! apply_server_options(fpinfo);
! apply_table_options(fpinfo);
!
! /*
! * If the table or the server is configured to use remote estimates,
! * identify which user to do remote access as during planning. This
! * should match what ExecCheckRTEPerms() does. If we fail due to lack of
! * permissions, the query would have failed at runtime anyway.
! */
! if (fpinfo->use_remote_estimate)
{
! Oid userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
!
! fpinfo->user = GetUserMapping(userid, fpinfo->server->serverid);
}
- else
- fpinfo->user = NULL;
! /*
! * Identify which baserestrictinfo clauses can be sent to the remote
! * server and which can't.
! */
! classifyConditions(root, baserel, baserel->baserestrictinfo,
! &fpinfo->remote_conds, &fpinfo->local_conds);
! /*
! * Identify which attributes will need to be retrieved from the remote
! * server. These include all attrs needed for joins or final output, plus
! * all attrs used in the local_conds. (Note: if we end up using a
! * parameterized scan, it's possible that some of the join clauses will be
! * sent to the remote and thus we wouldn't really need to retrieve the
! * columns used in them. Doesn't seem worth detecting that case though.)
! */
! fpinfo->attrs_used = NULL;
! pull_varattnos((Node *) baserel->reltarget->exprs, baserel->relid,
! &fpinfo->attrs_used);
! foreach(lc, fpinfo->local_conds)
{
! RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
! pull_varattnos((Node *) rinfo->clause, baserel->relid,
! &fpinfo->attrs_used);
}
/*
* Compute the selectivity and cost of the local_conds, so we don't have
* to do it over again for each path. The best we can do for these
--- 492,542 ----
static void
postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
PgFdwRelationInfo *fpinfo;
! EstimateInfo *estimates;
/*
* We use PgFdwRelationInfo to pass various information to subsequent
* functions.
+ *
+ * This function can be called twice: first time for plain scan, second
+ * time for the grouped one. fpinfo should only be initialized once.
*/
! if (baserel->fdw_private == NULL)
{
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! baserel->fdw_private = fpinfo;
! init_pgfdw_baserel_info(root, baserel, foreigntableid, grouped);
}
! fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
! if (grouped)
{
! /*
! * baserestrictinfo must be evaluated below the aggregation, so all
! * its expressions must be remote.
! */
! if (fpinfo->local_conds != NIL)
! return;
! /*
! * Given that there are no local_conds and remote_conds are evaluated
! * on the foreign server, baserel->gpi->target is the only thing left
! * to check.
! *
! * XXX Do we need to verify that the target contains no
! * PlaceHolderVars?
! */
! if (!foreign_grouping_ok(root, baserel))
! return;
}
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+
/*
* Compute the selectivity and cost of the local_conds, so we don't have
* to do it over again for each path. The best we can do for these
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 580,587 ****
* when they are set to some sensible costs during one (usually the first)
* of the calls to estimate_path_cost_size().
*/
! fpinfo->rel_startup_cost = -1;
! fpinfo->rel_total_cost = -1;
/*
* If the table or the server is configured to use remote estimates,
--- 555,562 ----
* when they are set to some sensible costs during one (usually the first)
* of the calls to estimate_path_cost_size().
*/
! estimates->rel_startup_cost = -1;
! estimates->rel_total_cost = -1;
/*
* If the table or the server is configured to use remote estimates,
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 597,609 ****
* values in fpinfo so we don't need to do it again to generate the
* basic foreign path.
*/
! estimate_path_cost_size(root, baserel, NIL, NIL,
! &fpinfo->rows, &fpinfo->width,
! &fpinfo->startup_cost, &fpinfo->total_cost);
/* Report estimated baserel size to planner. */
! baserel->rows = fpinfo->rows;
! baserel->reltarget->width = fpinfo->width;
}
else
{
--- 572,587 ----
* values in fpinfo so we don't need to do it again to generate the
* basic foreign path.
*/
! estimate_path_cost_size(root, baserel, NIL, NIL, estimates, grouped);
/* Report estimated baserel size to planner. */
!
! /*
! * TODO If grouped, make sure baserel->gpi exists and store the values
! * there.
! */
! baserel->rows = estimates->rows;
! baserel->reltarget->width = estimates->width;
}
else
{
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 628,661 ****
set_baserel_size_estimates(root, baserel);
/* Fill in basically-bogus cost estimates for use later. */
! estimate_path_cost_size(root, baserel, NIL, NIL,
! &fpinfo->rows, &fpinfo->width,
! &fpinfo->startup_cost, &fpinfo->total_cost);
}
-
- /*
- * Set the name of relation in fpinfo, while we are constructing it here.
- * It will be used to build the string describing the join relation in
- * EXPLAIN output. We can't know whether VERBOSE option is specified or
- * not, so always schema-qualify the foreign table name.
- */
- fpinfo->relation_name = makeStringInfo();
- namespace = get_namespace_name(get_rel_namespace(foreigntableid));
- relname = get_rel_name(foreigntableid);
- refname = rte->eref->aliasname;
- appendStringInfo(fpinfo->relation_name, "%s.%s",
- quote_identifier(namespace),
- quote_identifier(relname));
- if (*refname && strcmp(refname, relname) != 0)
- appendStringInfo(fpinfo->relation_name, " %s",
- quote_identifier(rte->eref->aliasname));
-
- /* No outer and inner relations. */
- fpinfo->make_outerrel_subquery = false;
- fpinfo->make_innerrel_subquery = false;
- fpinfo->lower_subquery_rels = NULL;
- /* Set the relation index. */
- fpinfo->relation_index = baserel->relid;
}
/*
--- 606,613 ----
set_baserel_size_estimates(root, baserel);
/* Fill in basically-bogus cost estimates for use later. */
! estimate_path_cost_size(root, baserel, NIL, NIL, estimates, grouped);
}
}
/*
*************** get_useful_ecs_for_relation(PlannerInfo
*** 775,781 ****
* to figure out which pathkeys to consider.
*/
static List *
! get_useful_pathkeys_for_relation(PlannerInfo *root, RelOptInfo *rel)
{
List *useful_pathkeys_list = NIL;
List *useful_eclass_list;
--- 727,734 ----
* to figure out which pathkeys to consider.
*/
static List *
! get_useful_pathkeys_for_relation(PlannerInfo *root, RelOptInfo *rel,
! bool grouped)
{
List *useful_pathkeys_list = NIL;
List *useful_eclass_list;
*************** get_useful_pathkeys_for_relation(Planner
*** 790,801 ****
if (root->query_pathkeys)
{
bool query_pathkeys_ok = true;
foreach(lc, root->query_pathkeys)
{
! PathKey *pathkey = (PathKey *) lfirst(lc);
! EquivalenceClass *pathkey_ec = pathkey->pk_eclass;
! Expr *em_expr;
/*
* The planner and executor don't have any clever strategy for
--- 743,757 ----
if (root->query_pathkeys)
{
bool query_pathkeys_ok = true;
+ bool sort_pathkeys_ok = true;
+ PathKey *pathkey;
+ EquivalenceClass *pathkey_ec;
+ Expr *em_expr;
foreach(lc, root->query_pathkeys)
{
! pathkey = lfirst_node(PathKey, lc);
! pathkey_ec = pathkey->pk_eclass;
/*
* The planner and executor don't have any clever strategy for
*************** get_useful_pathkeys_for_relation(Planner
*** 815,823 ****
break;
}
}
-
if (query_pathkeys_ok)
useful_pathkeys_list = list_make1(list_copy(root->query_pathkeys));
}
/*
--- 771,801 ----
break;
}
}
if (query_pathkeys_ok)
useful_pathkeys_list = list_make1(list_copy(root->query_pathkeys));
+
+ /*
+ * If the path is grouped, then it can be sorted by aggregates. These
+ * are only present in sort_pathkeys.
+ */
+ if (grouped)
+ {
+ foreach(lc, root->sort_pathkeys)
+ {
+ pathkey = lfirst_node(PathKey, lc);
+ pathkey_ec = pathkey->pk_eclass;
+
+ if (pathkey_ec->ec_has_volatile ||
+ !(em_expr = find_em_expr_for_rel(pathkey_ec, rel)) ||
+ !is_foreign_expr(root, rel, em_expr))
+ {
+ sort_pathkeys_ok = false;
+ break;
+ }
+ }
+ if (sort_pathkeys_ok)
+ useful_pathkeys_list = list_make1(list_copy(root->sort_pathkeys));
+ }
}
/*
*************** get_useful_pathkeys_for_relation(Planner
*** 884,895 ****
static void
postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
ForeignPath *path;
List *ppi_list;
ListCell *lc;
/*
* Create simplest ForeignScan path node and add it to baserel. This path
--- 862,905 ----
static void
postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
ForeignPath *path;
+ PathTarget *target = NULL;
List *ppi_list;
ListCell *lc;
+ List *fdw_private;
+ EstimateInfo *estimates;
+
+ if (grouped)
+ {
+ /*
+ * See postgresGetForeignRelSize().
+ */
+ if (fpinfo->local_conds != NIL)
+ return;
+
+ /*
+ * Wasn't it possible to construct remote query?
+ */
+ if (fpinfo->grouped_tlist == NIL)
+ return;
+
+ /*
+ * The target must contain aggregates.
+ */
+ Assert(baserel->gpi != NULL);
+ target = baserel->gpi->target;
+ }
+
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+
+ /*
+ * Add information to the paths whether they are grouped or not.
+ */
+ fdw_private = list_make1(makeInteger(grouped ? 1 : 0));
/*
* Create simplest ForeignScan path node and add it to baserel. This path
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 899,916 ****
* to estimate cost and size of this path.
*/
path = create_foreignscan_path(root, baserel,
! NULL, /* default pathtarget */
! fpinfo->rows,
! fpinfo->startup_cost,
! fpinfo->total_cost,
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
/* Add paths with pathkeys */
! add_paths_with_pathkeys_for_rel(root, baserel, NULL);
/*
* If we're not using remote estimates, stop here. We have no way to
--- 909,934 ----
* to estimate cost and size of this path.
*/
path = create_foreignscan_path(root, baserel,
! target,
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! fdw_private);
!
! /*
! * Grouped path essentially produces an unique set of grouping
! * keys.
! */
! if (grouped)
! make_uniquekeys_for_agg_path((Path *) path);
!
! add_path(baserel, (Path *) path, grouped);
/* Add paths with pathkeys */
! add_paths_with_pathkeys_for_rel(root, baserel, NULL, grouped);
/*
* If we're not using remote estimates, stop here. We have no way to
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 921,926 ****
--- 939,952 ----
return;
/*
+ * Grouped parameterized path would lead to repeated aggregation, which is
+ * not too interesting. Since the rest deals only with parameterized
+ * paths, return now.
+ */
+ if (grouped)
+ return;
+
+ /*
* Thumb through all join clauses for the rel to identify which outer
* relations could supply one or more safe-to-send-to-remote join clauses.
* We'll build a parameterized path for each such outer relation.
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 1052,1085 ****
foreach(lc, ppi_list)
{
ParamPathInfo *param_info = (ParamPathInfo *) lfirst(lc);
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
/* Get a cost estimate from the remote */
estimate_path_cost_size(root, baserel,
! param_info->ppi_clauses, NIL,
! &rows, &width,
! &startup_cost, &total_cost);
/*
* ppi_rows currently won't get looked at by anything, but still we
* may as well ensure that it matches our idea of the rowcount.
*/
! param_info->ppi_rows = rows;
/* Make the path */
path = create_foreignscan_path(root, baserel,
NULL, /* default pathtarget */
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
param_info->ppi_req_outer,
NULL,
! NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
}
}
--- 1078,1109 ----
foreach(lc, ppi_list)
{
ParamPathInfo *param_info = (ParamPathInfo *) lfirst(lc);
! EstimateInfo est_param;
/* Get a cost estimate from the remote */
+ memset(&est_param, 0, sizeof(EstimateInfo));
estimate_path_cost_size(root, baserel,
! param_info->ppi_clauses, NIL, &est_param,
! grouped);
/*
* ppi_rows currently won't get looked at by anything, but still we
* may as well ensure that it matches our idea of the rowcount.
*/
! param_info->ppi_rows = est_param.rows;
/* Make the path */
path = create_foreignscan_path(root, baserel,
NULL, /* default pathtarget */
! est_param.rows,
! est_param.startup_cost,
! est_param.total_cost,
NIL, /* no pathkeys */
param_info->ppi_req_outer,
NULL,
! fdw_private);
!
! add_path(baserel, (Path *) path, grouped);
}
}
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1107,1119 ****
List *retrieved_attrs;
StringInfoData sql;
ListCell *lc;
if (IS_SIMPLE_REL(foreignrel))
{
! /*
! * For base relations, set scan_relid as the relid of the relation.
! */
! scan_relid = foreignrel->relid;
/*
* In a base-relation scan, we must apply the given scan_clauses.
--- 1131,1169 ----
List *retrieved_attrs;
StringInfoData sql;
ListCell *lc;
+ Value *grouped_val;
+ bool grouped;
+
+ Assert(best_path->fdw_private != NIL);
+ grouped_val = (Value *) linitial(best_path->fdw_private);
+ Assert(IsA(grouped_val, Integer));
+ grouped = intVal(grouped_val) == 1;
if (IS_SIMPLE_REL(foreignrel))
{
! if (!grouped)
! {
! /*
! * For base relations, set scan_relid as the relid of the
! * relation.
! */
! scan_relid = foreignrel->relid;
! }
! else
! {
! /*
! * XXX Something seems to be inconsistent here: deparseSelectSql()
! * would create the targetlist from foreignrel->gpi->target, but
! * not appendGroupByClause(). Try to refactor.
! */
! fdw_scan_tlist = fpinfo->grouped_tlist;
!
! /*
! * The scan tuple descriptor should be constructed from
! * fdw_scan_tlist rather than from that of the remote relation.
! */
! scan_relid = 0;
! }
/*
* In a base-relation scan, we must apply the given scan_clauses.
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1191,1197 ****
*/
/* Build the list of columns to be fetched from the foreign server. */
! fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
/*
* Ensure that the outer plan produces a tuple whose descriptor
--- 1241,1250 ----
*/
/* Build the list of columns to be fetched from the foreign server. */
! if (!grouped)
! fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
! else
! fdw_scan_tlist = fpinfo->grouped_tlist;
/*
* Ensure that the outer plan produces a tuple whose descriptor
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1239,1245 ****
initStringInfo(&sql);
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_exprs, best_path->path.pathkeys,
! false, &retrieved_attrs, ¶ms_list);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
--- 1292,1299 ----
initStringInfo(&sql);
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_exprs, best_path->path.pathkeys,
! false, &retrieved_attrs, ¶ms_list,
! grouped);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
*************** postgresExplainDirectModify(ForeignScanS
*** 2481,2486 ****
--- 2535,2658 ----
}
}
+ /*
+ * init_pgfdw_rel_info
+ * Initialize PgFdwRelationInfo and assign it to the base relation. That
+ * includes estimates, as well as separation of local expressions from
+ * remote ones.
+ */
+ static void
+ init_pgfdw_baserel_info(PlannerInfo *root, RelOptInfo *rel,
+ Oid foreigntableid, bool grouped)
+
+ {
+ const char *namespace;
+ const char *relname;
+ const char *refname;
+ PgFdwRelationInfo *fpinfo;
+ ListCell *lc;
+ RangeTblEntry *rte;
+
+ rte = planner_rt_fetch(rel->relid, root);
+ fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
+
+ /*
+ * Identify which baserestrictinfo clauses can be sent to the remote
+ * server and which can't.
+ */
+ classifyConditions(root, rel, rel->baserestrictinfo,
+ &fpinfo->remote_conds, &fpinfo->local_conds);
+
+ /*
+ * Identify which attributes will need to be retrieved from the remote
+ * server. These include all attrs needed for joins or final output, plus
+ * all attrs used in the local_conds. (Note: if we end up using a
+ * parameterized scan, it's possible that some of the join clauses will be
+ * sent to the remote and thus we wouldn't really need to retrieve the
+ * columns used in them. Doesn't seem worth detecting that case though.)
+ */
+ fpinfo->attrs_used = NULL;
+ if (!grouped)
+ {
+ pull_varattnos((Node *) rel->reltarget->exprs, rel->relid,
+ &fpinfo->attrs_used);
+ foreach(lc, fpinfo->local_conds)
+ {
+ RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
+
+ pull_varattnos((Node *) rinfo->clause, rel->relid,
+ &fpinfo->attrs_used);
+ }
+ }
+ else
+ {
+ /*
+ * deparseExplicitTargetList() is used for the grouped relation, thus
+ * no need to retrieve attrs_used.
+ */
+ }
+
+ /* Base foreign tables need to be pushed down always. */
+ fpinfo->pushdown_safe = true;
+
+ /* Look up foreign-table catalog info. */
+ fpinfo->table = GetForeignTable(foreigntableid);
+ fpinfo->server = GetForeignServer(fpinfo->table->serverid);
+
+ /*
+ * Extract user-settable option values. Note that per-table setting of
+ * use_remote_estimate overrides per-server setting.
+ */
+ fpinfo->use_remote_estimate = false;
+ fpinfo->fdw_startup_cost = DEFAULT_FDW_STARTUP_COST;
+ fpinfo->fdw_tuple_cost = DEFAULT_FDW_TUPLE_COST;
+ fpinfo->shippable_extensions = NIL;
+ fpinfo->fetch_size = 100;
+
+ apply_server_options(fpinfo);
+ apply_table_options(fpinfo);
+
+ /*
+ * If the table or the server is configured to use remote estimates,
+ * identify which user to do remote access as during planning. This
+ * should match what ExecCheckRTEPerms() does. If we fail due to lack of
+ * permissions, the query would have failed at runtime anyway.
+ */
+ if (fpinfo->use_remote_estimate)
+ {
+ Oid userid = rte->checkAsUser ? rte->checkAsUser :
+ GetUserId();
+
+ fpinfo->user = GetUserMapping(userid, fpinfo->server->serverid);
+ }
+ else
+ fpinfo->user = NULL;
+
+ /*
+ * Set the name of relation in fpinfo, while we are constructing it here.
+ * It will be used to build the string describing the join relation in
+ * EXPLAIN output. We can't know whether VERBOSE option is specified or
+ * not, so always schema-qualify the foreign table name.
+ */
+ fpinfo->relation_name = makeStringInfo();
+ namespace = get_namespace_name(get_rel_namespace(foreigntableid));
+ relname = get_rel_name(foreigntableid);
+ refname = rte->eref->aliasname;
+ appendStringInfo(fpinfo->relation_name, "%s.%s",
+ quote_identifier(namespace),
+ quote_identifier(relname));
+ if (*refname && strcmp(refname, relname) != 0)
+ appendStringInfo(fpinfo->relation_name, " %s",
+ quote_identifier(rte->eref->aliasname));
+
+ /* No outer and inner relations. */
+ fpinfo->make_outerrel_subquery = false;
+ fpinfo->make_innerrel_subquery = false;
+ fpinfo->lower_subquery_rels = NULL;
+ /* Set the relation index. */
+ fpinfo->relation_index = rel->relid;
+ }
+
/*
* estimate_path_cost_size
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2499,2506 ****
RelOptInfo *foreignrel,
List *param_join_conds,
List *pathkeys,
! double *p_rows, int *p_width,
! Cost *p_startup_cost, Cost *p_total_cost)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
double rows;
--- 2671,2678 ----
RelOptInfo *foreignrel,
List *param_join_conds,
List *pathkeys,
! EstimateInfo * estimates,
! bool grouped)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
double rows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2539,2546 ****
&remote_param_join_conds, &local_param_join_conds);
/* Build the list of columns to be fetched from the foreign server. */
! if (IS_JOIN_REL(foreignrel) || IS_UPPER_REL(foreignrel))
fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
else
fdw_scan_tlist = NIL;
--- 2711,2723 ----
&remote_param_join_conds, &local_param_join_conds);
/* Build the list of columns to be fetched from the foreign server. */
! if ((IS_JOIN_REL(foreignrel) && !grouped) || IS_UPPER_REL(foreignrel))
fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
+ else if (grouped)
+ {
+ Assert(fpinfo->grouped_tlist != NIL);
+ fdw_scan_tlist = fpinfo->grouped_tlist;
+ }
else
fdw_scan_tlist = NIL;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2561,2567 ****
appendStringInfoString(&sql, "EXPLAIN ");
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_conds, pathkeys, false,
! &retrieved_attrs, NULL);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
--- 2738,2744 ----
appendStringInfoString(&sql, "EXPLAIN ");
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_conds, pathkeys, false,
! &retrieved_attrs, NULL, grouped);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2615,2629 ****
* bare scan each time. Instead, use the costs if we have cached them
* already.
*/
! if (fpinfo->rel_startup_cost > 0 && fpinfo->rel_total_cost > 0)
{
! startup_cost = fpinfo->rel_startup_cost;
! run_cost = fpinfo->rel_total_cost - fpinfo->rel_startup_cost;
}
else if (IS_JOIN_REL(foreignrel))
{
! PgFdwRelationInfo *fpinfo_i;
! PgFdwRelationInfo *fpinfo_o;
QualCost join_cost;
QualCost remote_conds_cost;
double nrows;
--- 2792,2808 ----
* bare scan each time. Instead, use the costs if we have cached them
* already.
*/
! if (estimates->rel_startup_cost > 0 && estimates->rel_total_cost > 0)
{
! startup_cost = estimates->rel_startup_cost;
! run_cost = estimates->rel_total_cost - estimates->rel_startup_cost;
}
else if (IS_JOIN_REL(foreignrel))
{
! PgFdwRelationInfo *fpinfo_i,
! *fpinfo_o;
! EstimateInfo *est_i,
! *est_o;
QualCost join_cost;
QualCost remote_conds_cost;
double nrows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2632,2641 ****
Assert(fpinfo->innerrel && fpinfo->outerrel);
fpinfo_i = (PgFdwRelationInfo *) fpinfo->innerrel->fdw_private;
fpinfo_o = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/* Estimate of number of rows in cross product */
! nrows = fpinfo_i->rows * fpinfo_o->rows;
/* Clamp retrieved rows estimate to at most size of cross product */
retrieved_rows = Min(retrieved_rows, nrows);
--- 2811,2822 ----
Assert(fpinfo->innerrel && fpinfo->outerrel);
fpinfo_i = (PgFdwRelationInfo *) fpinfo->innerrel->fdw_private;
+ est_i = !grouped ? &fpinfo_i->est_plain : &fpinfo_i->est_grouped;
fpinfo_o = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
+ est_o = !grouped ? &fpinfo_o->est_plain : &fpinfo_o->est_grouped;
/* Estimate of number of rows in cross product */
! nrows = est_i->rows * est_o->rows;
/* Clamp retrieved rows estimate to at most size of cross product */
retrieved_rows = Min(retrieved_rows, nrows);
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2659,2665 ****
* tables) since we do not know what strategy the foreign server
* is going to use.
*/
! startup_cost = fpinfo_i->rel_startup_cost + fpinfo_o->rel_startup_cost;
startup_cost += join_cost.startup;
startup_cost += remote_conds_cost.startup;
startup_cost += fpinfo->local_conds_cost.startup;
--- 2840,2846 ----
* tables) since we do not know what strategy the foreign server
* is going to use.
*/
! startup_cost = est_i->rel_startup_cost + est_o->rel_startup_cost;
startup_cost += join_cost.startup;
startup_cost += remote_conds_cost.startup;
startup_cost += fpinfo->local_conds_cost.startup;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2679,2695 ****
* 4. Run time cost of applying nonpushable other clauses locally
* on the result fetched from the foreign server.
*/
! run_cost = fpinfo_i->rel_total_cost - fpinfo_i->rel_startup_cost;
! run_cost += fpinfo_o->rel_total_cost - fpinfo_o->rel_startup_cost;
run_cost += nrows * join_cost.per_tuple;
nrows = clamp_row_est(nrows * fpinfo->joinclause_sel);
run_cost += nrows * remote_conds_cost.per_tuple;
run_cost += fpinfo->local_conds_cost.per_tuple * retrieved_rows;
}
! else if (IS_UPPER_REL(foreignrel))
{
PgFdwRelationInfo *ofpinfo;
! PathTarget *ptarget = root->upper_targets[UPPERREL_GROUP_AGG];
AggClauseCosts aggcosts;
double input_rows;
int numGroupCols;
--- 2860,2888 ----
* 4. Run time cost of applying nonpushable other clauses locally
* on the result fetched from the foreign server.
*/
! run_cost = est_i->rel_total_cost - est_i->rel_startup_cost;
! run_cost += est_o->rel_total_cost - est_o->rel_startup_cost;
run_cost += nrows * join_cost.per_tuple;
nrows = clamp_row_est(nrows * fpinfo->joinclause_sel);
run_cost += nrows * remote_conds_cost.per_tuple;
run_cost += fpinfo->local_conds_cost.per_tuple * retrieved_rows;
}
!
! /*
! * TODO Consider a separate branch for
! *
! * (IS_SIMPLE_REL(foreignrel) && * grouped)
! *
! * and use foreignrel->gpi->target in it instead of
! * fpinfo->grouped_tlist.
! */
! else if (IS_UPPER_REL(foreignrel) ||
! (IS_SIMPLE_REL(foreignrel) && grouped)
! )
{
PgFdwRelationInfo *ofpinfo;
! EstimateInfo *est;
! PathTarget *ptarget;
AggClauseCosts aggcosts;
double input_rows;
int numGroupCols;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2707,2717 ****
* considering remote and local conditions for costing.
*/
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/* Get rows and width from input rel */
! input_rows = ofpinfo->rows;
! width = ofpinfo->width;
/* Collect statistics about aggregates for estimating costs. */
MemSet(&aggcosts, 0, sizeof(AggClauseCosts));
--- 2900,2912 ----
* considering remote and local conditions for costing.
*/
! ofpinfo = IS_UPPER_REL(foreignrel) ?
! (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private : fpinfo;
! est = !grouped ? &ofpinfo->est_plain : &ofpinfo->est_grouped;
/* Get rows and width from input rel */
! input_rows = est->rows;
! width = est->width;
/* Collect statistics about aggregates for estimating costs. */
MemSet(&aggcosts, 0, sizeof(AggClauseCosts));
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2736,2741 ****
--- 2931,2944 ----
*/
rows = retrieved_rows = numGroups;
+ if (IS_UPPER_REL(foreignrel))
+ ptarget = root->upper_targets[UPPERREL_GROUP_AGG];
+ else
+ {
+ Assert(foreignrel->gpi != NULL);
+ ptarget = foreignrel->gpi->target;
+ }
+
/*-----
* Startup cost includes:
* 1. Startup cost for underneath input * relation
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2743,2749 ****
* 3. Startup cost for PathTarget eval
*-----
*/
! startup_cost = ofpinfo->rel_startup_cost;
startup_cost += aggcosts.transCost.startup;
startup_cost += aggcosts.transCost.per_tuple * input_rows;
startup_cost += (cpu_operator_cost * numGroupCols) * input_rows;
--- 2946,2952 ----
* 3. Startup cost for PathTarget eval
*-----
*/
! startup_cost = est->rel_startup_cost;
startup_cost += aggcosts.transCost.startup;
startup_cost += aggcosts.transCost.per_tuple * input_rows;
startup_cost += (cpu_operator_cost * numGroupCols) * input_rows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2756,2762 ****
* 3. PathTarget eval cost for each output row
*-----
*/
! run_cost = ofpinfo->rel_total_cost - ofpinfo->rel_startup_cost;
run_cost += aggcosts.finalCost * numGroups;
run_cost += cpu_tuple_cost * numGroups;
run_cost += ptarget->cost.per_tuple * numGroups;
--- 2959,2965 ----
* 3. PathTarget eval cost for each output row
*-----
*/
! run_cost = est->rel_total_cost - est->rel_startup_cost;
run_cost += aggcosts.finalCost * numGroups;
run_cost += cpu_tuple_cost * numGroups;
run_cost += ptarget->cost.per_tuple * numGroups;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2809,2816 ****
*/
if (pathkeys == NIL && param_join_conds == NIL)
{
! fpinfo->rel_startup_cost = startup_cost;
! fpinfo->rel_total_cost = total_cost;
}
/*
--- 3012,3019 ----
*/
if (pathkeys == NIL && param_join_conds == NIL)
{
! estimates->rel_startup_cost = startup_cost;
! estimates->rel_total_cost = total_cost;
}
/*
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2825,2834 ****
total_cost += cpu_tuple_cost * retrieved_rows;
/* Return results. */
! *p_rows = rows;
! *p_width = width;
! *p_startup_cost = startup_cost;
! *p_total_cost = total_cost;
}
/*
--- 3028,3037 ----
total_cost += cpu_tuple_cost * retrieved_rows;
/* Return results. */
! estimates->rows = rows;
! estimates->width = width;
! estimates->startup_cost = startup_cost;
! estimates->total_cost = total_cost;
}
/*
*************** postgresImportForeignSchema(ImportForeig
*** 4057,4075 ****
* Assess whether the join between inner and outer relations can be pushed down
* to the foreign server. As a side effect, save information we obtain in this
* function to PgFdwRelationInfo passed in.
*/
static bool
foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra)
{
! PgFdwRelationInfo *fpinfo;
! PgFdwRelationInfo *fpinfo_o;
! PgFdwRelationInfo *fpinfo_i;
ListCell *lc;
List *joinclauses;
/*
* We support pushing down INNER, LEFT, RIGHT and FULL OUTER joins.
* Constructing queries representing SEMI and ANTI joins is hard, hence
* not considered right now.
--- 4260,4289 ----
* Assess whether the join between inner and outer relations can be pushed down
* to the foreign server. As a side effect, save information we obtain in this
* function to PgFdwRelationInfo passed in.
+ *
+ * Caller expects that the "grouped" argument does not affect the result, so
+ * it's enough to call the function only once per relation. XXX As that
+ * argument is only used in Assert() statement, shouldn't it be removed?
*/
static bool
foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra, bool grouped)
{
! PgFdwRelationInfo *fpinfo,
! *fpinfo_o,
! *fpinfo_i;
ListCell *lc;
List *joinclauses;
/*
+ * TODO
+ *
+ * Put the (join-relevant) initial checks of foreign_grouping_ok into a
+ * function and call it here.
+ */
+
+ /*
* We support pushing down INNER, LEFT, RIGHT and FULL OUTER joins.
* Constructing queries representing SEMI and ANTI joins is hard, hence
* not considered right now.
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4139,4145 ****
--- 4353,4369 ----
if (is_remote_clause)
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
else
+ {
+ /*
+ * We could return in the grouped case because all clauses
+ * must be evaluated below the grouping plan (Thus if the
+ * grouping is remote, those clauses must be remote too.)
+ * However the same PgFdwRelationInfo is used for both grouped
+ * and non-grouped case and we don't want to force caller to
+ * process the non-grouped case first.
+ */
fpinfo->local_conds = lappend(fpinfo->local_conds, rinfo);
+ }
}
}
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4158,4166 ****
PlaceHolderInfo *phinfo = lfirst(lc);
Relids relids = joinrel->relids;
! if (bms_is_subset(phinfo->ph_eval_at, relids) &&
! bms_nonempty_difference(relids, phinfo->ph_eval_at))
! return false;
}
/* Save the join clauses, for later use. */
--- 4382,4407 ----
PlaceHolderInfo *phinfo = lfirst(lc);
Relids relids = joinrel->relids;
! if (bms_is_subset(phinfo->ph_eval_at, relids))
! {
! if (bms_nonempty_difference(relids, phinfo->ph_eval_at))
! {
! #ifdef USE_ASSERT_CHECKING
! if (grouped)
! {
! /*
! * Grouped join shouldn't have PlaceHolderVar even in its
! * own targetlist. It'd mean that the output of partial
! * aggregation can be set to NULL before it gets to the
! * input of the final aggregation.
! */
! Assert(false);
! }
! #endif /* USE_ASSERT_CHECKING */
!
! return false;
! }
! }
}
/* Save the join clauses, for later use. */
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4283,4296 ****
fpinfo->user = NULL;
/*
- * Set cached relation costs to some negative value, so that we can detect
- * when they are set to some sensible costs, during one (usually the
- * first) of the calls to estimate_path_cost_size().
- */
- fpinfo->rel_startup_cost = -1;
- fpinfo->rel_total_cost = -1;
-
- /*
* Set the string describing this join relation to be used in EXPLAIN
* output of corresponding ForeignScan.
*/
--- 4524,4529 ----
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4315,4350 ****
static void
add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path)
{
List *useful_pathkeys_list = NIL; /* List of all pathkeys */
ListCell *lc;
! useful_pathkeys_list = get_useful_pathkeys_for_relation(root, rel);
/* Create one path for each set of pathkeys we found above. */
foreach(lc, useful_pathkeys_list)
{
- double rows;
- int width;
- Cost startup_cost;
- Cost total_cost;
List *useful_pathkeys = lfirst(lc);
estimate_path_cost_size(root, rel, NIL, useful_pathkeys,
! &rows, &width, &startup_cost, &total_cost);
! add_path(rel, (Path *)
! create_foreignscan_path(root, rel,
! NULL,
! rows,
! startup_cost,
! total_cost,
! useful_pathkeys,
! NULL,
! epq_path,
! NIL),
! false);
}
}
--- 4548,4607 ----
static void
add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path, bool grouped)
{
List *useful_pathkeys_list = NIL; /* List of all pathkeys */
ListCell *lc;
+ List *fdw_private;
+ PathTarget *target = NULL;
! useful_pathkeys_list = get_useful_pathkeys_for_relation(root, rel,
! grouped);
!
! /*
! * Add information to the paths whether they are grouped or not.
! */
! fdw_private = list_make1(makeInteger(grouped ? 1 : 0));
!
! /*
! * Grouped foreign scan should emit tuples according to the grouped
! * target, i.e. the one containing GroupedVars.
! */
! if (grouped)
! {
! Assert(rel->gpi != NULL);
! target = rel->gpi->target;
! }
/* Create one path for each set of pathkeys we found above. */
foreach(lc, useful_pathkeys_list)
{
List *useful_pathkeys = lfirst(lc);
+ EstimateInfo estimates;
+ Path *path;
estimate_path_cost_size(root, rel, NIL, useful_pathkeys,
! &estimates, grouped);
!
! path = (Path *) create_foreignscan_path(root, rel,
! target,
! estimates.rows,
! estimates.startup_cost,
! estimates.total_cost,
! useful_pathkeys,
! NULL,
! epq_path,
! fdw_private);
!
! /*
! * Grouped path essentially produces an unique set of grouping
! * keys.
! */
! if (grouped)
! make_uniquekeys_for_agg_path(path);
!
! add_path(rel, path, grouped);
}
}
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4462,4496 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
PgFdwRelationInfo *fpinfo;
ForeignPath *joinpath;
- double rows;
- int width;
- Cost startup_cost;
- Cost total_cost;
Path *epq_path; /* Path to create plan to be executed when
* EvalPlanQual gets triggered. */
! /*
! * Skip if this join combination has been considered already.
! */
! if (joinrel->fdw_private)
! return;
! /*
! * Create unfinished PgFdwRelationInfo entry which is used to indicate
! * that the join relation is already considered, so that we won't waste
! * time in judging safety of join pushdown and adding the same paths again
! * if found safe. Once we know that this join can be pushed down, we fill
! * the entry.
! */
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! fpinfo->pushdown_safe = false;
! joinrel->fdw_private = fpinfo;
! /* attrs_used is only for base relations. */
! fpinfo->attrs_used = NULL;
/*
* If there is a possibility that EvalPlanQual will be executed, we need
--- 4719,4769 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
PgFdwRelationInfo *fpinfo;
+ EstimateInfo *estimates;
ForeignPath *joinpath;
Path *epq_path; /* Path to create plan to be executed when
* EvalPlanQual gets triggered. */
+ bool first_time;
+ PathTarget *target = NULL;
+ List *fdw_private;
! if (joinrel->fdw_private == NULL)
! {
! /*
! * First time through.
! *
! * Create unfinished PgFdwRelationInfo entry which is used to indicate
! * that the join relation is already considered, so that we won't
! * waste time in judging safety of join pushdown and adding the same
! * paths again if found safe. Once we know that this join can be
! * pushed down, we fill the entry.
! */
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! fpinfo->pushdown_safe = false;
! joinrel->fdw_private = fpinfo;
! /* attrs_used is only for base relations. */
! fpinfo->attrs_used = NULL;
! first_time = true;
! }
! else
! {
! fpinfo = (PgFdwRelationInfo *) joinrel->fdw_private;
! first_time = false;
!
! if (!fpinfo->pushdown_safe)
! {
! /*
! * The function was already called but some test failed. No reason
! * to see that failure again.
! */
! return;
! }
! }
/*
* If there is a possibility that EvalPlanQual will be executed, we need
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4501,4527 ****
* dominate the only suitable local path available. We also do it before
* calling foreign_join_ok(), since that function updates fpinfo and marks
* it as pushable if the join is found to be pushable.
*/
! if (root->parse->commandType == CMD_DELETE ||
! root->parse->commandType == CMD_UPDATE ||
! root->rowMarks)
{
! epq_path = GetExistingLocalJoinPath(joinrel);
! if (!epq_path)
{
! elog(DEBUG3, "could not push down foreign join because a local path suitable for EPQ checks was not found");
return;
}
}
- else
- epq_path = NULL;
! if (!foreign_join_ok(root, joinrel, jointype, outerrel, innerrel, extra))
! {
! /* Free path required for EPQ if we copied one; we don't need it now */
! if (epq_path)
! pfree(epq_path);
return;
}
/*
--- 4774,4851 ----
* dominate the only suitable local path available. We also do it before
* calling foreign_join_ok(), since that function updates fpinfo and marks
* it as pushable if the join is found to be pushable.
+ *
+ * The following tests should only be performed once --- repeated, the are
+ * supposed to yield the same result. (If anything failed initially,
+ * pushdown_safe should have caused return above.)
*/
! if (first_time)
{
! if (root->parse->commandType == CMD_DELETE ||
! root->parse->commandType == CMD_UPDATE ||
! root->rowMarks)
{
! epq_path = GetExistingLocalJoinPath(joinrel);
! if (!epq_path)
! {
! elog(DEBUG3, "could not push down foreign join because a local path suitable for EPQ checks was not found");
! return;
! }
! }
! else
! epq_path = NULL;
!
! if (!foreign_join_ok(root, joinrel, jointype, outerrel, innerrel, extra,
! grouped))
! {
! /*
! * Free path required for EPQ if we copied one; we don't need it
! * now
! */
! if (epq_path)
! pfree(epq_path);
return;
}
}
! /*
! * Local expressions would be evaluated above the grouping plan, which
! * will be remote.
! */
! if (grouped && fpinfo->local_conds != NIL)
return;
+
+ /*
+ * This function should not be called repeatedly wit the same value of
+ * "grouped", but if it happened, make sure the "upper conditions" do not
+ * get duplicated or used at all if !grouped.
+ */
+ if (fpinfo->remote_conds_upper)
+ {
+ list_free(fpinfo->remote_conds_upper);
+ fpinfo->remote_conds_upper = NIL;
+ }
+ if (fpinfo->local_conds_upper)
+ {
+ list_free(fpinfo->local_conds_upper);
+ fpinfo->local_conds_upper = NIL;
+ }
+
+ if (grouped)
+ {
+ /*
+ * Create targelist to be used for the remote grouping.
+ */
+ Assert(joinrel->gpi != NULL);
+ fpinfo->grouped_tlist = create_remote_agg_tlist(root, joinrel,
+ joinrel->gpi->target);
+
+ /*
+ * Couldn't push any GROUP BY expression or aggregate to the remote
+ * server?
+ */
+ if (fpinfo->grouped_tlist == NIL)
+ return;
}
/*
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4548,4563 ****
0, fpinfo->jointype,
extra->sjinfo);
/* Estimate costs for bare join relation */
! estimate_path_cost_size(root, joinrel, NIL, NIL, &rows,
! &width, &startup_cost, &total_cost);
/* Now update this information in the joinrel */
! joinrel->rows = rows;
! joinrel->reltarget->width = width;
! fpinfo->rows = rows;
! fpinfo->width = width;
! fpinfo->startup_cost = startup_cost;
! fpinfo->total_cost = total_cost;
/*
* Create a new join path and add it to the joinrel which represents a
--- 4872,4915 ----
0, fpinfo->jointype,
extra->sjinfo);
+ /*
+ * Set cached relation costs to some negative value, so that we can detect
+ * when they are set to some sensible costs, during one (usually the
+ * first) of the calls to estimate_path_cost_size().
+ */
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+ estimates->rel_startup_cost = -1;
+ estimates->rel_total_cost = -1;
+
/* Estimate costs for bare join relation */
!
! /*
! * TODO Include remote_conds_upper and local_conds_upper into the
! * estimates.
! */
! estimate_path_cost_size(root, joinrel, NIL, NIL, estimates, grouped);
/* Now update this information in the joinrel */
!
! /*
! * TODO If grouped, make sure baserel->gpi exists and store the values
! * there.
! */
! joinrel->rows = estimates->rows;
! joinrel->reltarget->width = estimates->width;
!
! if (grouped)
! {
! /*
! * The target must contain aggregates.
! */
! Assert(joinrel->gpi != NULL);
! target = joinrel->gpi->target;
! }
!
! /*
! * Add information to the paths whether they are grouped or not.
! */
! fdw_private = list_make1(makeInteger(grouped ? 1 : 0));
/*
* Create a new join path and add it to the joinrel which represents a
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4565,4584 ****
*/
joinpath = create_foreignscan_path(root,
joinrel,
! NULL, /* default pathtarget */
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
epq_path,
! NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, false);
/* Consider pathkeys for the join relation */
! add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
/* XXX Consider parameterized paths for the join relation */
}
--- 4917,4943 ----
*/
joinpath = create_foreignscan_path(root,
joinrel,
! target, /* default pathtarget */
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
epq_path,
! fdw_private); /* no fdw_private */
!
! /*
! * Grouped path essentially produces an unique set of grouping
! * keys.
! */
! if (grouped)
! make_uniquekeys_for_agg_path((Path *) joinpath);
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, grouped);
/* Consider pathkeys for the join relation */
! add_paths_with_pathkeys_for_rel(root, joinrel, epq_path, grouped);
/* XXX Consider parameterized paths for the join relation */
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4594,4618 ****
Query *query = root->parse;
PathTarget *grouping_target;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
! PgFdwRelationInfo *ofpinfo;
! List *aggvars;
! ListCell *lc;
! int i;
! List *tlist = NIL;
/* Grouping Sets are not pushable */
if (query->groupingSets)
return false;
/* Get the fpinfo of the underlying scan relation. */
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/*
* If underneath input relation has any local conditions, those conditions
* are required to be applied before performing aggregation. Hence the
* aggregate cannot be pushed down.
*/
! if (ofpinfo->local_conds)
return false;
/*
--- 4953,4975 ----
Query *query = root->parse;
PathTarget *grouping_target;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
! PgFdwRelationInfo *ofpinfo = NULL;
/* Grouping Sets are not pushable */
if (query->groupingSets)
return false;
/* Get the fpinfo of the underlying scan relation. */
! if (IS_UPPER_REL(grouped_rel))
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/*
* If underneath input relation has any local conditions, those conditions
* are required to be applied before performing aggregation. Hence the
* aggregate cannot be pushed down.
*/
! if ((IS_UPPER_REL(grouped_rel) && ofpinfo->local_conds) ||
! fpinfo->local_conds)
return false;
/*
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4623,4637 ****
* different from those in the plan's targetlist. Use a copy of path
* target to record the new sortgrouprefs.
*/
! grouping_target = copy_pathtarget(root->upper_targets[UPPERREL_GROUP_AGG]);
/*
! * Evaluate grouping targets and check whether they are safe to push down
! * to the foreign side. All GROUP BY expressions will be part of the
! * grouping target and thus there is no need to evaluate it separately.
! * While doing so, add required expressions into target list which can
! * then be used to pass to foreign server.
*/
i = 0;
foreach(lc, grouping_target->exprs)
{
--- 4980,5048 ----
* different from those in the plan's targetlist. Use a copy of path
* target to record the new sortgrouprefs.
*/
! if (IS_UPPER_REL(grouped_rel))
! grouping_target = root->upper_targets[UPPERREL_GROUP_AGG];
! else
! {
! /*
! * FDW should not be asked to generate grouped paths if the simple or
! * join relation has no relevant target.
! */
! Assert(grouped_rel->gpi != NULL);
! Assert(grouped_rel->gpi->target != NULL);
!
! grouping_target = grouped_rel->gpi->target;
! }
/*
! * Create targelist to be sent to the remote server.
! */
! fpinfo->grouped_tlist = create_remote_agg_tlist(root, grouped_rel,
! grouping_target);
! if (fpinfo->grouped_tlist == NIL)
! return false;
!
! /* Safe to pushdown */
! fpinfo->pushdown_safe = true;
!
! /*
! * Set the string describing this grouped relation to be used in EXPLAIN
! * output of corresponding ForeignScan.
*/
+ if (ofpinfo != NULL)
+ {
+ fpinfo->relation_name = makeStringInfo();
+ appendStringInfo(fpinfo->relation_name, "Aggregate on (%s)",
+ ofpinfo->relation_name->data);
+ }
+
+ return true;
+ }
+
+ /*
+ * Evaluate grouping targets and check whether they are safe to push down to
+ * the foreign side. All GROUP BY expressions will be part of the grouping
+ * target and thus there is no need to evaluate it separately. While doing
+ * so, add required expressions into target list which can then be used to
+ * pass to foreign server.
+ *
+ * NIL is returned if the remote aggregation appears to be impossible.
+ *
+ * Note that the function can add items to both remote_conds_upper and
+ * local_conds_upper lists of PgFdwRelationInfo.
+ */
+ static List *
+ create_remote_agg_tlist(PlannerInfo *root, RelOptInfo *grouped_rel,
+ PathTarget *grouping_target)
+ {
+ List *tlist = NIL;
+ ListCell *lc;
+ int i;
+ Query *query = root->parse;
+ PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
+
+ grouping_target = copy_pathtarget(grouping_target);
+
i = 0;
foreach(lc, grouping_target->exprs)
{
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4647,4659 ****
* push down aggregation to the foreign server.
*/
if (!is_foreign_expr(root, grouped_rel, expr))
! return false;
/* Pushable, add to tlist */
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
else
{
/* Check entire expression whether it is pushable or not */
if (is_foreign_expr(root, grouped_rel, expr))
{
--- 5058,5091 ----
* push down aggregation to the foreign server.
*/
if (!is_foreign_expr(root, grouped_rel, expr))
! return NIL;
/* Pushable, add to tlist */
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
else
{
+ /*
+ * Aggregate can sometimes be wrapped by GroupedVar.
+ *
+ */
+ if ((IS_SIMPLE_REL(grouped_rel) || IS_JOIN_REL(grouped_rel)) &&
+ IsA(expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, expr);
+
+ /*
+ * GroupedVar is currently used only to handle Aggrefs.
+ *
+ * XXX These may include aggregates that appear in the HAVING
+ * clause. Although the HAVING clause is currently not sent to
+ * the remote server (see deparseSelectStmtForRel), we still
+ * need to send the aggregates the HAVING clause contains.
+ */
+ Assert(IsA(gvar->agg_partial, Aggref));
+ expr = (Expr *) gvar->agg_partial;
+ }
+
/* Check entire expression whether it is pushable or not */
if (is_foreign_expr(root, grouped_rel, expr))
{
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4662,4667 ****
--- 5094,5101 ----
}
else
{
+ List *aggvars;
+
/*
* If we have sortgroupref set, then it means that we have an
* ORDER BY entry pointing to this expression. Since we are
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4670,4681 ****
if (sgref)
grouping_target->sortgrouprefs[i] = 0;
! /* Not matched exactly, pull the var with aggregates then */
aggvars = pull_var_clause((Node *) expr,
PVC_INCLUDE_AGGREGATES);
if (!is_foreign_expr(root, grouped_rel, (Expr *) aggvars))
! return false;
/*
* Add aggregates, if any, into the targetlist. Plain var
--- 5104,5117 ----
if (sgref)
grouping_target->sortgrouprefs[i] = 0;
! /*
! * Not matched exactly, pull the var with aggregates then
! */
aggvars = pull_var_clause((Node *) expr,
PVC_INCLUDE_AGGREGATES);
if (!is_foreign_expr(root, grouped_rel, (Expr *) aggvars))
! return NIL;
/*
* Add aggregates, if any, into the targetlist. Plain var
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4703,4714 ****
/*
* Classify the pushable and non-pushable having clauses and save them in
! * remote_conds and local_conds of the grouped rel's fpinfo.
*/
if (root->hasHavingQual && query->havingQual)
{
- ListCell *lc;
-
foreach(lc, (List *) query->havingQual)
{
Expr *expr = (Expr *) lfirst(lc);
--- 5139,5155 ----
/*
* Classify the pushable and non-pushable having clauses and save them in
! * remote_conds_upper and local_conds_upper of the grouped rel's fpinfo.
! */
!
! /*
! * TODO For non-upper grouped_rel, only pick those expressions for which
! * grouped_rel provides all input vars. (And Assert() that grouped_rel
! * does not emit just some of them --- in such a case grouped_rel
! * shouldn't be subject to partial aggregation at all.)
*/
if (root->hasHavingQual && query->havingQual)
{
foreach(lc, (List *) query->havingQual)
{
Expr *expr = (Expr *) lfirst(lc);
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4728,4736 ****
NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
! fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
else
! fpinfo->local_conds = lappend(fpinfo->local_conds, rinfo);
}
}
--- 5169,5179 ----
NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
! fpinfo->remote_conds_upper =
! lappend(fpinfo->remote_conds_upper, rinfo);
else
! fpinfo->local_conds_upper =
! lappend(fpinfo->local_conds_upper, rinfo);
}
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4738,4749 ****
* If there are any local conditions, pull Vars and aggregates from it and
* check whether they are safe to pushdown or not.
*/
! if (fpinfo->local_conds)
{
List *aggvars = NIL;
- ListCell *lc;
! foreach(lc, fpinfo->local_conds)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
--- 5181,5191 ----
* If there are any local conditions, pull Vars and aggregates from it and
* check whether they are safe to pushdown or not.
*/
! if (fpinfo->local_conds_upper)
{
List *aggvars = NIL;
! foreach(lc, fpinfo->local_conds_upper)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4765,4771 ****
if (IsA(expr, Aggref))
{
if (!is_foreign_expr(root, grouped_rel, expr))
! return false;
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
--- 5207,5213 ----
if (IsA(expr, Aggref))
{
if (!is_foreign_expr(root, grouped_rel, expr))
! return NIL;
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4775,4803 ****
/* Transfer any sortgroupref data to the replacement tlist */
apply_pathtarget_labeling_to_tlist(tlist, grouping_target);
! /* Store generated targetlist */
! fpinfo->grouped_tlist = tlist;
!
! /* Safe to pushdown */
! fpinfo->pushdown_safe = true;
!
! /*
! * Set cached relation costs to some negative value, so that we can detect
! * when they are set to some sensible costs, during one (usually the
! * first) of the calls to estimate_path_cost_size().
! */
! fpinfo->rel_startup_cost = -1;
! fpinfo->rel_total_cost = -1;
!
! /*
! * Set the string describing this grouped relation to be used in EXPLAIN
! * output of corresponding ForeignScan.
! */
! fpinfo->relation_name = makeStringInfo();
! appendStringInfo(fpinfo->relation_name, "Aggregate on (%s)",
! ofpinfo->relation_name->data);
!
! return true;
}
/*
--- 5217,5223 ----
/* Transfer any sortgroupref data to the replacement tlist */
apply_pathtarget_labeling_to_tlist(tlist, grouping_target);
! return tlist;
}
/*
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4846,4857 ****
Query *parse = root->parse;
PgFdwRelationInfo *ifpinfo = input_rel->fdw_private;
PgFdwRelationInfo *fpinfo = grouped_rel->fdw_private;
ForeignPath *grouppath;
PathTarget *grouping_target;
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
/* Nothing to be done, if there is no grouping or aggregation required. */
if (!parse->groupClause && !parse->groupingSets && !parse->hasAggs &&
--- 5266,5275 ----
Query *parse = root->parse;
PgFdwRelationInfo *ifpinfo = input_rel->fdw_private;
PgFdwRelationInfo *fpinfo = grouped_rel->fdw_private;
+ EstimateInfo *estimates;
ForeignPath *grouppath;
PathTarget *grouping_target;
! List *fdw_private;
/* Nothing to be done, if there is no grouping or aggregation required. */
if (!parse->groupClause && !parse->groupingSets && !parse->hasAggs &&
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4877,4902 ****
return;
/* Estimate the cost of push down */
! estimate_path_cost_size(root, grouped_rel, NIL, NIL, &rows,
! &width, &startup_cost, &total_cost);
! /* Now update this information in the fpinfo */
! fpinfo->rows = rows;
! fpinfo->width = width;
! fpinfo->startup_cost = startup_cost;
! fpinfo->total_cost = total_cost;
/* Create and add foreign path to the grouping relation. */
grouppath = create_foreignscan_path(root,
grouped_rel,
grouping_target,
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
NULL,
! NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
add_path(grouped_rel, (Path *) grouppath, false);
--- 5295,5320 ----
return;
/* Estimate the cost of push down */
! estimates = &fpinfo->est_grouped;
! estimate_path_cost_size(root, grouped_rel, NIL, NIL, estimates, true);
! /*
! * The path is considered grouped when it comes to plan creation and
! * deparsing.
! */
! fdw_private = list_make1(makeInteger(1));
/* Create and add foreign path to the grouping relation. */
grouppath = create_foreignscan_path(root,
grouped_rel,
grouping_target,
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
NULL,
! fdw_private);
/* Add generated path into grouped_rel by add_path(). */
add_path(grouped_rel, (Path *) grouppath, false);
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
new file mode 100644
index 788b003..e796236
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 20,25 ****
--- 20,37 ----
#include "libpq-fe.h"
+ typedef struct EstimateInfo
+ {
+ /* Estimated size and cost for a scan or join. */
+ double rows;
+ int width;
+ Cost startup_cost;
+ Cost total_cost;
+ /* Costs excluding costs for transferring data from the foreign server */
+ Cost rel_startup_cost;
+ Cost rel_total_cost;
+ } EstimateInfo;
+
/*
* FDW-specific planner information kept in RelOptInfo.fdw_private for a
* postgres_fdw foreign table. For a baserel, this struct is created by
*************** typedef struct PgFdwRelationInfo
*** 43,48 ****
--- 55,68 ----
List *remote_conds;
List *local_conds;
+ /*
+ * HAVING clauses.
+ *
+ * TODO Use these where appropriate.
+ */
+ List *remote_conds_upper;
+ List *local_conds_upper;
+
/* Actual remote restriction clauses for scan (sans RestrictInfos) */
List *final_remote_exprs;
*************** typedef struct PgFdwRelationInfo
*** 56,69 ****
/* Selectivity of join conditions */
Selectivity joinclause_sel;
! /* Estimated size and cost for a scan or join. */
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
! /* Costs excluding costs for transferring data from the foreign server */
! Cost rel_startup_cost;
! Cost rel_total_cost;
/* Options extracted from catalogs. */
bool use_remote_estimate;
--- 76,84 ----
/* Selectivity of join conditions */
Selectivity joinclause_sel;
! /* Estimates for both plain and grouped relation. */
! EstimateInfo est_plain;
! EstimateInfo est_grouped;
/* Options extracted from catalogs. */
bool use_remote_estimate;
*************** typedef struct PgFdwRelationInfo
*** 92,98 ****
/* joinclauses contains only JOIN/ON conditions for an outer join */
List *joinclauses; /* List of RestrictInfo */
! /* Grouping information */
List *grouped_tlist;
/* Subquery information */
--- 107,115 ----
/* joinclauses contains only JOIN/ON conditions for an outer join */
List *joinclauses; /* List of RestrictInfo */
! /*
! * The targetlist used to construct the remote query.
! */
List *grouped_tlist;
/* Subquery information */
*************** extern unsigned int GetCursorNumber(PGco
*** 121,126 ****
--- 138,144 ----
extern unsigned int GetPrepStmtNumber(PGconn *conn);
extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
+ extern void pgfdw_send_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
*************** extern List *build_tlist_to_deparse(RelO
*** 175,181 ****
extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
List *remote_conds, List *pathkeys, bool is_subquery,
! List **retrieved_attrs, List **params_list);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
--- 193,200 ----
extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
List *remote_conds, List *pathkeys, bool is_subquery,
! List **retrieved_attrs, List **params_list,
! bool grouping);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index c278acb..a5c1b68
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 978,988 ****
static void
set_foreign_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
/* Mark rel with estimated output rows, width, etc */
! set_foreign_size_estimates(root, rel);
! /* Let FDW adjust the size estimates, if it can */
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid);
/* ... but do not let it set the rows estimate to zero */
rel->rows = clamp_row_est(rel->rows);
--- 978,1006 ----
static void
set_foreign_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
+ Relids required_outer = rel->lateral_relids;
+
/* Mark rel with estimated output rows, width, etc */
! set_foreign_size_estimates(root, rel, false);
! /*
! * Let FDW adjust the size estimates, if it can.
! *
! * For the grouped relation it only makes sense if set_foreign_pathlist is
! * supposed to handle it, see below.
! */
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid, false);
! if (rel->gpi != NULL && rel->gpi->target != NULL &&
! required_outer == NULL)
! {
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid, true);
!
! /*
! * XXX Should cost_qual_eval be called separate so we don't repeat it
! * here for essentially identical baserestrictinfo?
! */
! set_foreign_size_estimates(root, rel, true);
! }
/* ... but do not let it set the rows estimate to zero */
rel->rows = clamp_row_est(rel->rows);
*************** set_foreign_size(PlannerInfo *root, RelO
*** 995,1002 ****
static void
set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
/* Call the FDW's GetForeignPaths function to generate path(s) */
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid);
}
/*
--- 1013,1030 ----
static void
set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
+ Relids required_outer = rel->lateral_relids;
+
/* Call the FDW's GetForeignPaths function to generate path(s) */
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid, false);
!
! /*
! * Create grouped paths if grouped target exists. Do nothing if outer
! * relations are needed --- that could cause parameterized (and therefore
! * repeated) grouping.
! */
! if (rel->gpi != NULL && rel->gpi->target && required_outer == NULL)
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid, true);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1574,1579 ****
--- 1602,1608 ----
foreach(l, live_childrels)
{
RelOptInfo *childrel = lfirst(l);
+ List *childpaths;
ListCell *lcp;
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1636,1642 ****
* heuristic to indicate which sort orderings and parameterizations we
* should build Append and MergeAppend paths for.
*/
! foreach(lcp, childrel->pathlist)
{
Path *childpath = (Path *) lfirst(lcp);
List *childkeys = childpath->pathkeys;
--- 1665,1681 ----
* heuristic to indicate which sort orderings and parameterizations we
* should build Append and MergeAppend paths for.
*/
! childpaths = childrel->pathlist;
!
! /*
! * Extra orderings may be available for grouped paths, i.e. ordered by
! * aggregate. (At least ForeignPath can generate these.)
! */
! if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
! childpaths = list_concat(list_copy(childpaths),
! childrel->gpi->pathlist);
!
! foreach(lcp, childpaths)
{
Path *childpath = (Path *) lfirst(lcp);
List *childkeys = childpath->pathkeys;
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index 2f1d8fc..eea377f
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
*************** set_namedtuplestore_size_estimates(Plann
*** 4808,4814 ****
* already.
*/
void
! set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel)
{
/* Should only be applied to base relations */
Assert(rel->relid > 0);
--- 4808,4814 ----
* already.
*/
void
! set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
/* Should only be applied to base relations */
Assert(rel->relid > 0);
*************** set_foreign_size_estimates(PlannerInfo *
*** 4817,4823 ****
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel, rel->reltarget);
}
--- 4817,4835 ----
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! if (!grouped)
! set_rel_width(root, rel, rel->reltarget);
! else
! {
! Assert(rel->gpi != NULL);
!
! /*
! * If the relation can produce grouped path, estimate width and costs
! * of the corresponding target.
! */
! if (rel->gpi->target)
! set_rel_width(root, rel, rel->gpi->target);
! }
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index fdcd6d1..439904d
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 334,342 ****
*/
if (joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
! jointype, &extra);
/*
* 6. Finally, give extensions a chance to manipulate the path list.
--- 334,348 ----
*/
if (joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
+ {
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
! jointype, &extra, false);
! if (joinrel->gpi != NULL && joinrel->gpi->target != NULL)
! joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
! outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 6. Finally, give extensions a chance to manipulate the path list.
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index 6cd9c7c..1a22226
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** static Plan *prepare_sort_from_pathkeys(
*** 250,255 ****
--- 250,256 ----
static EquivalenceMember *find_ec_member_for_tle(EquivalenceClass *ec,
TargetEntry *tle,
Relids relids);
+ static TargetEntry *get_aggref_from_tle(TargetEntry *tle);
static Sort *make_sort_from_pathkeys(Plan *lefttree, List *pathkeys,
Relids relids);
static Sort *make_sort_from_groupcols(List *groupcls,
*************** prepare_sort_from_pathkeys(Plan *lefttre
*** 5640,5645 ****
--- 5641,5649 ----
tle = get_tle_by_resno(tlist, reqColIdx[numsortkeys]);
if (tle)
{
+ /* Handle special cases of sorting by aggregate. */
+ tle = get_aggref_from_tle(tle);
+
em = find_ec_member_for_tle(ec, tle, relids);
if (em)
{
*************** prepare_sort_from_pathkeys(Plan *lefttre
*** 5672,5708 ****
{
tle = (TargetEntry *) lfirst(j);
! /*
! * Does this happen to be an aggregate final function
! * referencing the partial aggregate? This can replace the
! * final aggregation if the path generates an unique set of
! * grouping keys.
! */
! if (IS_AGGFINALFN_STANDALONE(tle->expr))
! {
! FuncExpr *fexpr = castNode(FuncExpr, tle->expr);
! GroupedVar *gvar = linitial_node(GroupedVar, fexpr->args);
! Aggref *aggref = castNode(Aggref, gvar->gvexpr);
!
! Assert(fexpr->funcid == aggref->aggfinalfn);
!
! tle = flatCopyTargetEntry(tle);
! tle->expr = (Expr *) aggref;
! }
!
! /*
! * Aggregate w/o aggfinalfn?
! */
! else if (IsA(tle->expr, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, tle->expr);
!
! if (IsA(gvar->gvexpr, Aggref))
! {
! tle = flatCopyTargetEntry(tle);
! tle->expr = gvar->gvexpr;
! }
! }
em = find_ec_member_for_tle(ec, tle, relids);
if (em)
--- 5676,5683 ----
{
tle = (TargetEntry *) lfirst(j);
! /* Handle special cases of sorting by aggregate. */
! tle = get_aggref_from_tle(tle);
em = find_ec_member_for_tle(ec, tle, relids);
if (em)
*************** find_ec_member_for_tle(EquivalenceClass
*** 5877,5882 ****
--- 5852,5900 ----
}
/*
+ * Get Aggref from target entry if it's wrapped in GroupedVar.
+ */
+ static TargetEntry *
+ get_aggref_from_tle(TargetEntry *tle)
+ {
+ /*
+ * Does this happen to be an aggregate final function referencing the
+ * partial aggregate? This can replace the final aggregation if the path
+ * generates an unique set of grouping keys.
+ */
+ if (IS_AGGFINALFN_STANDALONE(tle->expr))
+ {
+ FuncExpr *fexpr = castNode(FuncExpr, tle->expr);
+ GroupedVar *gvar = linitial_node(GroupedVar, fexpr->args);
+ Aggref *aggref = castNode(Aggref, gvar->gvexpr);
+
+ Assert(fexpr->funcid == aggref->aggfinalfn);
+
+ tle = flatCopyTargetEntry(tle);
+ tle->expr = (Expr *) aggref;
+ }
+
+ /*
+ * Aggregate w/o aggfinalfn?
+ */
+ else if (IsA(tle->expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, tle->expr);
+
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, gvar->gvexpr);
+
+ Assert(aggref->aggfinalfn == InvalidOid);
+ tle = flatCopyTargetEntry(tle);
+ tle->expr = (Expr *) aggref;
+ }
+ }
+
+ return tle;
+ }
+
+ /*
* make_sort_from_pathkeys
* Create sort plan to sort according to given pathkeys
*
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index 99aabd5..8d0c453
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** set_foreignscan_references(PlannerInfo *
*** 1193,1225 ****
if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0)
{
/*
* Adjust tlist, qual, fdw_exprs, fdw_recheck_quals to reference
! * foreign scan tuple
*/
! indexed_tlist *itlist = build_tlist_index(fscan->fdw_scan_tlist);
!
fscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
! (Node *) fscan->scan.plan.targetlist,
itlist,
INDEX_VAR,
rtoffset);
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
(Node *) fscan->scan.plan.qual,
itlist,
INDEX_VAR,
rtoffset);
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
(Node *) fscan->fdw_exprs,
itlist,
INDEX_VAR,
rtoffset);
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
! (Node *) fscan->fdw_recheck_quals,
itlist,
INDEX_VAR,
rtoffset);
--- 1193,1242 ----
if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0)
{
+ List *tlist_tmp;
+ indexed_tlist *itlist = build_tlist_index(fscan->fdw_scan_tlist);
+
/*
* Adjust tlist, qual, fdw_exprs, fdw_recheck_quals to reference
! * foreign scan tuple.
! *
! * Since fdw_scan_tlist contains Aggrefs instead of GroupVars (this is
! * convenient for deparsing), the Aggrefs also need to be restored in
! * the referencing lists.
! *
! * XXX Consider thoroughly where tlist_tmp really needs to be used,
! * i.e. which of the lists can / cannot contain aggregates.
*/
! tlist_tmp = restore_grouping_expressions(root,
! fscan->scan.plan.targetlist,
! true);
fscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
! (Node *) tlist_tmp,
itlist,
INDEX_VAR,
rtoffset);
+
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
(Node *) fscan->scan.plan.qual,
itlist,
INDEX_VAR,
rtoffset);
+
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
(Node *) fscan->fdw_exprs,
itlist,
INDEX_VAR,
rtoffset);
+
+ tlist_tmp = restore_grouping_expressions(root,
+ fscan->fdw_recheck_quals,
+ true);
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
! (Node *) tlist_tmp,
itlist,
INDEX_VAR,
rtoffset);
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
new file mode 100644
index 04e43cc..16d6082
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
*************** struct ExplainState;
*** 26,36 ****
typedef void (*GetForeignRelSize_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
typedef void (*GetForeignPaths_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
typedef ForeignScan *(*GetForeignPlan_function) (PlannerInfo *root,
RelOptInfo *baserel,
--- 26,38 ----
typedef void (*GetForeignRelSize_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
typedef void (*GetForeignPaths_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
typedef ForeignScan *(*GetForeignPlan_function) (PlannerInfo *root,
RelOptInfo *baserel,
*************** typedef void (*GetForeignJoinPaths_funct
*** 57,63 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
UpperRelationKind stage,
--- 59,66 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped);
typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
UpperRelationKind stage,
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
new file mode 100644
index 8424d3a..8419c40
*** a/src/include/optimizer/cost.h
--- b/src/include/optimizer/cost.h
*************** extern void set_cte_size_estimates(Plann
*** 188,194 ****
double cte_rows);
extern void set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern void set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel);
! extern void set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern PathTarget *set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target);
extern double compute_bitmap_pages(PlannerInfo *root, RelOptInfo *baserel,
Path *bitmapqual, int loop_count, Cost *cost, double *tuple);
--- 188,195 ----
double cte_rows);
extern void set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern void set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel);
! extern void set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel,
! bool groupe);
extern PathTarget *set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target);
extern double compute_bitmap_pages(PlannerInfo *root, RelOptInfo *baserel,
Path *bitmapqual, int loop_count, Cost *cost, double *tuple);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 60b8182..6b7667b
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern void split_pathtarget_at_srfs(Pla
*** 72,78 ****
/* TODO Find the best location (position and in some cases even file) for the
* following ones. */
extern List *restore_grouping_expressions(PlannerInfo *root, List *src,
! bool partial);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
--- 72,78 ----
/* TODO Find the best location (position and in some cases even file) for the
* following ones. */
extern List *restore_grouping_expressions(PlannerInfo *root, List *src,
! bool agg_partial);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
agg_pushdown_v4/09_readme.diff
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
new file mode 100644
index 1e4084d..3994394
*** a/src/backend/optimizer/README
--- b/src/backend/optimizer/README
*************** plan as possible. Expanding the range o
*** 1076,1081 ****
--- 1076,1116 ----
pushed below the Gather (and costing them accurately) is likely to keep us
busy for a long time to come.
+ Grouped Paths
+ -------------
+
+ If both query semantics and path type allow, an existing path can be subject
+ to partial aggregation (ie aggregation without calling the aggfinalfn
+ function) and stored as a "grouped path" in a separate pathlist. Both base
+ relation and join paths can be turned into a grouped path this way. If the
+ source path was partial (in terms of parallel processing, see above), then the
+ grouped path is called "partial grouped path".
+
+ If a grouped path appears at the top of the join tree, the transient state
+ values are aggregated and the aggfinalfn function is called for each
+ aggregate. Likewise, if the top-level join produces a partial grouped path, we
+ first apply Gather plan on it and then we finalize the aggregation in the same
+ way.
+
+ Besides being a result of aggregation, (partial) grouped path can be created
+ by joining an existing grouped path to an ordinary (non-grouped) path. This
+ technique can result in duplication of grouping keys, which are essentially
+ unique in the output of aggregation. The final aggregation takes these
+ duplicities into account.
+
+ In contrast, join of two grouped paths is not supported as there doesn't seem
+ to be an interesting use case for it. If we want to support this kind of join
+ in the future, additional attention needs to be paid to the duplication of
+ grouping keys mentioned above. For example, if scan of table A is joined to a
+ grouped scan of table B, then the cardinality of grouping keys of table B
+ present in (non-grouped) table A determines how many times does each
+ aggregation transient state value of B appear on the input of the final
+ aggregation path. However if A is grouped too, the number of those key values
+ on the A side can get lower and thus affect the input of the final aggregation
+ path. When compensating for this effect, we'd have to count input values of
+ each group during the aggregation of A and "multiply" the corresponding
+ transient state values of B accordingly.
+
Partition-wise joins
--------------------
A join between two similarly partitioned tables can be broken down into joins
[application/x-gzip] shard.tgz (1.4K, ../../14577.1509723225@localhost/3-shard.tgz)
download
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2017-11-30 04:15 Michael Paquier <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Michael Paquier @ 2017-11-30 04:15 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <[email protected]> wrote:
> I'm not about to add any other features now. Implementation of the missing
> parts (see the TODO comments in the code) is the next step. But what I'd
> appreciate most is a feedback on the design. Thanks.
I am getting a conflict after applying patch 5 but this did not get
any reviews so moved to next CF with waiting on author as status.
--
Michael
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2017-12-22 15:43 Antonin Houska <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2017-12-22 15:43 UTC (permalink / raw)
To: pgsql-hackers
Michael Paquier <[email protected]> wrote:
> On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <[email protected]> wrote:
> > I'm not about to add any other features now. Implementation of the missing
> > parts (see the TODO comments in the code) is the next step. But what I'd
> > appreciate most is a feedback on the design. Thanks.
>
> I am getting a conflict after applying patch 5 but this did not get
> any reviews so moved to next CF with waiting on author as status.
Attached is the next version.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
Attachments:
[application/x-gzip] agg_pushdown_v5.tgz (118.4K, ../../18007.1513957437@localhost/2-agg_pushdown_v5.tgz)
download | 11 patch file(s) in archive:
agg_pushdown_v5/01_new_types.diff
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 96d1fc3..85e8ea3
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyAggref(const Aggref *from)
*** 1361,1366 ****
--- 1361,1367 ----
COPY_SCALAR_FIELD(aggcollid);
COPY_SCALAR_FIELD(inputcollid);
COPY_SCALAR_FIELD(aggtranstype);
+ COPY_SCALAR_FIELD(aggcombinefn);
COPY_NODE_FIELD(aggargtypes);
COPY_NODE_FIELD(aggdirectargs);
COPY_NODE_FIELD(args);
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2213,2218 ****
--- 2214,2235 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** _copyPlaceHolderInfo(const PlaceHolderIn
*** 2285,2290 ****
--- 2302,2322 ----
return newnode;
}
+ static GroupedVarInfo *
+ _copyGroupedVarInfo(const GroupedVarInfo *from)
+ {
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_SCALAR_FIELD(gvid);
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(gv_width);
+
+ return newnode;
+ }
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
*************** copyObjectImpl(const void *from)
*** 5020,5025 ****
--- 5052,5060 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
*************** copyObjectImpl(const void *from)
*** 5032,5037 ****
--- 5067,5075 ----
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index 2e869a9..3d735a2
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 873,878 ****
--- 873,886 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3171,3176 ****
--- 3179,3187 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index c2a93b2..9a53aa7
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 259,264 ****
--- 259,270 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ type = exprType((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprTypmod(const Node *expr)
*** 492,497 ****
--- 498,508 ----
return ((const SetToDefault *) expr)->typeMod;
case T_PlaceHolderVar:
return exprTypmod((Node *) ((const PlaceHolderVar *) expr)->phexpr);
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->gvexpr);
default:
break;
}
*************** exprCollation(const Node *expr)
*** 903,908 ****
--- 914,925 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2176,2181 ****
--- 2193,2200 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2968,2973 ****
--- 2987,3002 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index e468d7c..d70d210
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outAggref(StringInfo str, const Aggref
*** 1136,1141 ****
--- 1136,1142 ----
WRITE_OID_FIELD(aggcollid);
WRITE_OID_FIELD(inputcollid);
WRITE_OID_FIELD(aggtranstype);
+ WRITE_OID_FIELD(aggcombinefn);
WRITE_NODE_FIELD(aggargtypes);
WRITE_NODE_FIELD(aggdirectargs);
WRITE_NODE_FIELD(args);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2225,2230 ****
--- 2226,2232 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2232,2237 ****
--- 2234,2240 ----
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
*************** _outRelOptInfo(StringInfo str, const Rel
*** 2271,2276 ****
--- 2274,2280 ----
WRITE_NODE_FIELD(cheapest_parameterized_paths);
WRITE_BITMAPSET_FIELD(direct_lateral_relids);
WRITE_BITMAPSET_FIELD(lateral_relids);
+ WRITE_NODE_FIELD(gpi);
WRITE_UINT_FIELD(relid);
WRITE_OID_FIELD(reltablespace);
WRITE_ENUM_FIELD(rtekind, RTEKind);
*************** _outParamPathInfo(StringInfo str, const
*** 2447,2452 ****
--- 2451,2468 ----
}
static void
+ _outGroupedPathInfo(StringInfo str, const GroupedPathInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDPATHINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(pathlist);
+ WRITE_NODE_FIELD(partial_pathlist);
+ WRITE_NODE_FIELD(sortgroupclauses);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2490,2495 ****
--- 2506,2522 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** _outPlaceHolderInfo(StringInfo str, cons
*** 2543,2548 ****
--- 2570,2588 ----
}
static void
+ _outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_UINT_FIELD(gvid);
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_INT_FIELD(gv_width);
+ }
+
+ static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
*************** outNode(StringInfo str, const void *obj)
*** 4045,4056 ****
--- 4085,4102 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_GroupedPathInfo:
+ _outGroupedPathInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
*************** outNode(StringInfo str, const void *obj)
*** 4063,4068 ****
--- 4109,4117 ----
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index 1133c70..f02a963
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 529,534 ****
--- 529,550 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(sortgroupref);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** _readAggref(void)
*** 584,589 ****
--- 600,606 ----
READ_OID_FIELD(aggcollid);
READ_OID_FIELD(inputcollid);
READ_OID_FIELD(aggtranstype);
+ READ_OID_FIELD(aggcombinefn);
READ_NODE_FIELD(aggargtypes);
READ_NODE_FIELD(aggdirectargs);
READ_NODE_FIELD(args);
*************** parseNodeString(void)
*** 2472,2477 ****
--- 2489,2496 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index f4e0a6e..2e6a60e
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 114,120 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 382791f..5a3efbd
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** subquery_planner(PlannerGlobal *glob, Qu
*** 544,549 ****
--- 544,550 ----
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
new file mode 100644
index 1d7e499..710f028
*** a/src/backend/optimizer/prep/prepjointree.c
--- b/src/backend/optimizer/prep/prepjointree.c
*************** pull_up_simple_subquery(PlannerInfo *roo
*** 911,916 ****
--- 911,917 ----
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ subroot->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
new file mode 100644
index e6b0856..7a98753
*** a/src/backend/parser/parse_func.c
--- b/src/backend/parser/parse_func.c
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 98,103 ****
--- 98,104 ----
Oid vatype;
FuncDetailCode fdresult;
char aggkind = 0;
+ Oid aggcombinefn = InvalidOid;
ParseCallbackState pcbstate;
/*
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 350,355 ****
--- 351,357 ----
elog(ERROR, "cache lookup failed for aggregate %u", funcid);
classForm = (Form_pg_aggregate) GETSTRUCT(tup);
aggkind = classForm->aggkind;
+ aggcombinefn = classForm->aggcombinefn;
catDirectArgs = classForm->aggnumdirectargs;
ReleaseSysCache(tup);
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 695,700 ****
--- 697,703 ----
aggref->aggstar = agg_star;
aggref->aggvariadic = func_variadic;
aggref->aggkind = aggkind;
+ aggref->aggcombinefn = aggcombinefn;
/* agglevelsup will be set by transformAggregateCall */
aggref->aggsplit = AGGSPLIT_SIMPLE; /* planner might change this */
aggref->location = location;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index c5b5115..86693d1
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 218,223 ****
--- 218,224 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_GroupedPathInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 258,267 ****
--- 259,270 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index 074ae0a..97fd887
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct Aggref
*** 296,301 ****
--- 296,302 ----
Oid aggcollid; /* OID of collation of result */
Oid inputcollid; /* OID of collation that function should use */
Oid aggtranstype; /* type Oid of aggregate's transition value */
+ Oid aggcombinefn; /* combine function (see pg_aggregate.h) */
List *aggargtypes; /* type Oids of direct and aggregated args */
List *aggdirectargs; /* direct arguments, if an ordered-set agg */
List *args; /* aggregated arguments and sort expressions */
*************** typedef struct Aggref
*** 306,311 ****
--- 307,313 ----
bool aggvariadic; /* true if variadic arguments have been
* combined into an array last argument */
char aggkind; /* aggregate kind (see pg_aggregate.h) */
+
Index agglevelsup; /* > 0 if agg belongs to outer query */
AggSplit aggsplit; /* expected agg-splitting mode of parent Agg */
int location; /* token location, or -1 if unknown */
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index 3b9d303..1c93688
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 257,262 ****
--- 257,264 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 283,288 ****
--- 285,296 ----
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
*************** typedef struct PartitionSchemeData *Part
*** 438,443 ****
--- 446,453 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 609,614 ****
--- 619,627 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to produce grouped paths. */
+ struct GroupedPathInfo *gpi;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 1003,1008 ****
--- 1016,1065 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * GroupedPathInfo
+ *
+ * If RelOptInfo points to this structure, grouped paths can be created for
+ * it.
+ *
+ * "target" will be used as pathtarget of grouped paths produced either by
+ * "explicit aggregation" of the relation that owns this structure, or --- if
+ * the relation is a join --- by joining grouped path to a non-grouped
+ * one.
+ *
+ * The target contains plain-Var grouping expressions, generic grouping
+ * expressions wrapped in GroupedVar structure, or Aggrefs which are also
+ * wrapped in GroupedVar. Once GroupedVar is evaluated, its value is passed to
+ * the upper paths w/o being evaluated again. If final aggregation appears to
+ * be necessary above the final join, the contained Aggrefs are supposed to
+ * provide the final aggregation plan with input values, i.e. the aggregate
+ * transient state.
+ *
+ * Note: There's a convention that GroupedVars that contain Aggref expressions
+ * are supposed to follow the other expressions of the target. Iterations of
+ * target->exprs may rely on this arrangement.
+ *
+ * "group_exprs" contains grouping expressions which are not plain vars. These
+ * are only added to path target of a path which should generate input for
+ * partial aggregation.
+ *
+ * "sortgroupclauses" is a list of grouping clauses that the relation does
+ * have in its targetlist but the query does not.
+ *
+ * (Two grouped paths cannot be joined in general because grouping of one side
+ * of the join essentially reduces occurrence of groups of the other side in
+ * the input of the final aggregation.)
+ */
+ typedef struct GroupedPathInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* target of grouped paths aggregation. */
+ PathTarget *group_exprs; /* non-Var grouping expressions. */
+ List *pathlist; /* List of grouped paths. */
+ List *partial_pathlist; /* List of partial grouped paths. */
+ List *sortgroupclauses; /* Relation-specific grouping clauses. */
+ } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct PlaceHolderVar
*** 1950,1955 ****
--- 2007,2047 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. The reason is that aggregates having start as the argument can be
+ * evaluated at various places in the join tree (i.e. cannot be assigned to
+ * target list of exactly one relation). Also this concept seems to be less
+ * invasive than adding the grouped vars to reltarget (in which case
+ * attr_needed and attr_widths arrays of RelOptInfo) would also need
+ * additional changes.
+ *
+ * gvexpr is a pointer to gvexpr field of the corresponding instance
+ * GroupedVarInfo. It's there for the sake of exprType(), exprCollation(),
+ * etc.
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if the
+ * GroupedVar is in the target of a parent relation (RELOPT_BASEREL). However
+ * within a child relation's (RELOPT_OTHER_MEMBER_REL) target it points to a
+ * copy which has argument expressions translated, so they no longer reference
+ * the parent.
+ *
+ * XXX Currently we only create GroupedVar for aggregates, but sometime we can
+ * do it for grouping keys as well. That would allow grouping below the
+ * top-level join by keys other than plain Var.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index sortgroupref; /* SortGroupClause.tleSortGroupRef if gvexpr
+ * is grouping expression. */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PlaceHolderInfo
*** 2165,2170 ****
--- 2257,2281 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
agg_pushdown_v5/02_preprocess.diff
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
new file mode 100644
index 45a6889..bd22449
*** a/src/backend/optimizer/path/equivclass.c
--- b/src/backend/optimizer/path/equivclass.c
*************** static bool reconsider_outer_join_clause
*** 65,70 ****
--- 65,83 ----
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
+ typedef struct translate_expr_context
+ {
+ Var **keys; /* Dictionary keys. */
+ Var **values; /* Dictionary values */
+ int nitems; /* Number of dictionary items. */
+ Relids *gv_eval_at_p; /* See GroupedVarInfo. */
+ Relids relids; /* Translate into these relids. */
+ } translate_expr_context;
+
+ static Node *translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context *context);
+ static int var_dictionary_comparator(const void *a, const void *b);
+
/*
* process_equivalence
*************** is_redundant_derived_clause(RestrictInfo
*** 2510,2512 ****
--- 2523,2844 ----
return false;
}
+
+ /*
+ * translate_expression_to_rels
+ *
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is contained in relids.
+ */
+ GroupedVarInfo *
+ translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Relids relids)
+ {
+ List *vars;
+ ListCell *l1;
+ int i,
+ j;
+ int nkeys,
+ nkeys_resolved;
+ Var **keys,
+ **values,
+ **keys_tmp;
+ Var *key,
+ *key_prev;
+ translate_expr_context context;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ /*
+ * Before actually trying to modify the expression tree, find out if all
+ * vars can be translated.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+
+ /* No vars to translate? */
+ if (vars == NIL)
+ return NULL;
+
+ /*
+ * Search for individual replacement vars as well as the actual expression
+ * translation will be more efficient if we use a dictionary with the keys
+ * (i.e. the "source vars") unique and sorted.
+ */
+ nkeys = list_length(vars);
+ keys = (Var **) palloc(nkeys * sizeof(Var *));
+ i = 0;
+ foreach(l1, vars)
+ {
+ key = lfirst_node(Var, l1);
+ keys[i++] = key;
+ }
+
+ /*
+ * Sort the keys by varno. varattno decides where varnos are equal.
+ */
+ pg_qsort(keys, nkeys, sizeof(Var *), var_dictionary_comparator);
+
+ /*
+ * Pick unique values and get rid of the vars that need no translation.
+ */
+ keys_tmp = (Var **) palloc(nkeys * sizeof(Var *));
+ key_prev = NULL;
+ j = 0;
+ for (i = 0; i < nkeys; i++)
+ {
+ key = keys[i];
+
+ if ((key_prev == NULL || (key->varno != key_prev->varno &&
+ key->varattno != key_prev->varattno)) &&
+ !bms_is_member(key->varno, relids))
+ keys_tmp[j++] = key;
+
+ key_prev = key;
+ }
+ pfree(keys);
+ keys = keys_tmp;
+ nkeys = j;
+
+ /*
+ * Is there actually nothing to be translated?
+ */
+ if (nkeys == 0)
+ {
+ pfree(keys);
+ return NULL;
+ }
+
+ nkeys_resolved = 0;
+
+ /*
+ * Find the replacement vars.
+ */
+ values = (Var **) palloc0(nkeys * sizeof(Var *));
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ Relids ec_var_relids;
+ Var **ec_vars;
+ int ec_nvars;
+ ListCell *l2;
+
+ /* TODO Re-check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ ec_vars = (Var **) palloc(list_length(ec->ec_members) * sizeof(Var *));
+ ec_nvars = 0;
+ ec_var_relids = NULL;
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ ec_vars[ec_nvars++] = ec_var;
+ ec_var_relids = bms_add_member(ec_var_relids, ec_var->varno);
+ }
+
+ /*
+ * At least two vars are needed so that the EC is usable for
+ * translation.
+ */
+ if (ec_nvars <= 1)
+ {
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+ continue;
+ }
+
+ /*
+ * Now check where this EC can help.
+ */
+ for (i = 0; i < nkeys; i++)
+ {
+ Relids ec_rest;
+ bool relids_ok,
+ key_found;
+ Var *key = keys[i];
+ Var *value = values[i];
+
+ /* Skip this item if it's already resolved. */
+ if (value != NULL)
+ continue;
+
+ /*
+ * Can't translate if the EC does not mention key->varno.
+ */
+ if (!bms_is_member(key->varno, ec_var_relids))
+ continue;
+
+ /*
+ * Besides key, at least one EC member must belong to the relation
+ * we're translating our expression to.
+ */
+ ec_rest = bms_copy(ec_var_relids);
+ ec_rest = bms_del_member(ec_rest, key->varno);
+ relids_ok = bms_overlap(ec_rest, relids);
+ bms_free(ec_rest);
+ if (!relids_ok)
+ continue;
+
+ /*
+ * The preliminary checks passed, so try to find the exact vars.
+ */
+ key_found = false;
+ for (j = 0; j < ec_nvars; j++)
+ {
+ Var *ec_var = ec_vars[j];
+
+ if (!key_found && key->varno == ec_var->varno &&
+ key->varattno == ec_var->varattno)
+ key_found = true;
+
+ /*
+ * If relids contains multiple members, it shouldn't matter to
+ * which one we translate our key. Simply use the first one.
+ *
+ * XXX Shouldn't ec_var be copied?
+ */
+ if (value == NULL && bms_is_member(ec_var->varno, relids))
+ value = ec_var;
+
+ if (key_found && value != NULL)
+ break;
+ }
+
+ if (key_found && value != NULL)
+ {
+ values[i] = value;
+ nkeys_resolved++;
+
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+ }
+
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+
+ /* Don't need to check the remaining ECs? */
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+
+ /* Couldn't compose usable dictionary? */
+ if (nkeys_resolved < nkeys)
+ {
+ pfree(keys);
+ pfree(values);
+ return NULL;
+ }
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_copy(result->gv_eval_at);
+
+ /* The dictionary is ready, so perform the translation. */
+ context.keys = keys;
+ context.values = values;
+ context.nitems = nkeys;
+ context.gv_eval_at_p = &result->gv_eval_at;
+ context.relids = relids;
+ result->gvexpr = (Expr *)
+ translate_expression_to_rels_mutator((Node *) gvi->gvexpr, &context);
+
+ pfree(keys);
+ pfree(values);
+ return result;
+ }
+
+ static Node *
+ translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context *context)
+ {
+ if (node == NULL)
+ return NULL;
+
+ if (IsA(node, Var))
+ {
+ Var *var = castNode(Var, node);
+ Var **key_p;
+ Var *value;
+ int index;
+
+ /*
+ * Simply return the existing variable if already belongs to the
+ * relation we're adjusting the expression to.
+ */
+ if (bms_is_member(var->varno, context->relids))
+ return (Node *) var;
+
+ key_p = bsearch(&var, context->keys, context->nitems, sizeof(Var *),
+ var_dictionary_comparator);
+
+ /* We shouldn't have omitted any var from the dictionary. */
+ Assert(key_p != NULL);
+
+ index = key_p - context->keys;
+ Assert(index >= 0 && index < context->nitems);
+ value = context->values[index];
+
+ /* All values should be present in the dictionary. */
+ Assert(value != NULL);
+
+ /* Update gv_eval_at accordingly. */
+ bms_del_member(*context->gv_eval_at_p, var->varno);
+ *context->gv_eval_at_p = bms_add_member(*context->gv_eval_at_p,
+ value->varno);
+
+ return (Node *) value;
+ }
+
+ return expression_tree_mutator(node, translate_expression_to_rels_mutator,
+ (void *) context);
+ }
+
+ static int
+ var_dictionary_comparator(const void *a, const void *b)
+ {
+ Var **var1_p,
+ **var2_p;
+ Var *var1,
+ *var2;
+
+ var1_p = (Var **) a;
+ var1 = castNode(Var, *var1_p);
+ var2_p = (Var **) b;
+ var2 = castNode(Var, *var2_p);
+
+ if (var1->varno < var2->varno)
+ return -1;
+ else if (var1->varno > var2->varno)
+ return 1;
+
+ if (var1->varattno < var2->varattno)
+ return -1;
+ else if (var1->varattno > var2->varattno)
+ return 1;
+
+ return 0;
+ }
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 448cb73..8de59fc
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,21 ****
--- 14,23 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "catalog/pg_class.h"
+ #include "catalog/pg_constraint_fn.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
***************
*** 27,32 ****
--- 29,35 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 46,51 ****
--- 49,56 ----
} PostponedQual;
+ static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+ static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** add_vars_to_targetlist(PlannerInfo *root
*** 241,246 ****
--- 246,839 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression and setup GroupedPathInfo for each base
+ * relation that can product grouped paths.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouping_info_to_base_rels(PlannerInfo *root)
+ {
+ int i;
+ ListCell *lc;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * TODO Consider if this is a real limitation.
+ */
+ if (root->parse->hasWindowFuncs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Is no grouping is possible below the top-level join? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+
+ /*
+ * "other rels" will have their targets built later, by translation of
+ * the target of the parent rel - see set_append_rel_size. If we
+ * wanted to prepare the child rels here, we'd need another iteration
+ * of simple_rel_array_size.
+ */
+ if (rel != NULL && rel->reloptkind == RELOPT_BASEREL)
+ prepare_rel_for_grouping(root, rel);
+ }
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+ static void
+ create_aggregate_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /*
+ * Aggregation push-down does not work w/o aggcombinefn. This field is
+ * not mandatory, so check if this particular aggregate can handle
+ * partial aggregation.
+ */
+ if (!OidIsValid(aggref->aggcombinefn))
+ {
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ /*
+ * The transient state is what appears in the target.
+ */
+ gvi->gv_width =
+ get_typavgwidth(exprType((Node *) gvi->agg_partial),
+ exprTypmod((Node *) gvi->agg_partial));
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+ static void
+ create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+ {
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVar exists for each expression usable as grouping key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Try to derive additional grouping expressions from group_pathkeys. This
+ * increases the chance that relation can be grouped even if its columns
+ * are not mentioned in the grouping clause.
+ *
+ * It's important that we have processed the explicit grouping columns
+ * first. If the grouping clause contains multiple expressions belonging
+ * to the same EC, the original (i.e. not derived) one should be preferred
+ * when we build grouping target for a relation. Otherwise we have a
+ * problem when trying to match target entries to grouping clauses during
+ * plan creation, see extract_grouping_cols.
+ */
+ foreach(l1, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l1);
+ EquivalenceClass *ec = pk->pk_eclass;
+ EquivalenceMember *em;
+ Index sortgroupref = 0;
+
+ /* We need equality anywhere in the join tree. */
+ if (ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * TODO Reconsider this restriction. As the grouping expression is
+ * only evaluated at the relation level (and only the result will be
+ * propagated to the final targetlist), volatile function might be
+ * o.k. Need to think what volatile EC exactly means.
+ */
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(l2, ec->ec_members)
+ {
+ ListCell *l3;
+
+ em = lfirst_node(EquivalenceMember, l2);
+
+ /*
+ * We search for grouping expressions now.
+ */
+ if (IsA(em->em_expr, Aggref))
+ continue;
+
+ if (em->em_nullable_relids)
+ continue;
+
+ /*
+ * If targetlist contains this expression, it should provide us
+ * with the sortgroupref.
+ */
+ foreach(l3, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, l3);
+
+ if (equal(em->em_expr, te->expr))
+ {
+ /*
+ * XXX Not sure if the same expression exist in the
+ * targetlist multiple times and if some occurrences can
+ * miss the ressortgroupref. Maybe we just need to
+ * Assert() here that ressortgroupref is non-zero.
+ */
+ if (te->ressortgroupref > 0)
+ {
+ sortgroupref = te->ressortgroupref;
+ break;
+ }
+ }
+ }
+
+ /*
+ * If a single EC member matches, no need to check the rest of the
+ * EC.
+ */
+ if (sortgroupref > 0)
+ break;
+ }
+
+ if (sortgroupref == 0)
+ /* Go for the next EC. */
+ continue;
+
+ /*
+ * The EC contains a grouping expression, so each member of that EC
+ * should be usable as grouping key. Remember all the expressions of
+ * the EC as well as their (supposedly common) sortgroupref for the
+ * final processing.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ ListCell *l3;
+
+ em = lfirst_node(EquivalenceMember, l2);
+
+ /*
+ * Do not add the expression if it's explicitly mentioned by the
+ * targetlist. We should already have it in that case.
+ */
+ foreach(l3, exprs)
+ {
+ if (equal(lfirst(l3), em->em_expr))
+ break;
+ }
+ if (l3 != NULL)
+ continue;
+
+ exprs = lappend(exprs, em->em_expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+ }
+
+ /*
+ * Finally construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(expr);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = pull_varnos((Node *) expr);
+
+ /*
+ * As a special case, Var can be present in the non-grouped target, so
+ * set_rel_widths() could take care of the width computation. However
+ * the Var might not be propagated high enough in the join tree, so
+ * set_rel_width() might miss it. Handle it manually.
+ */
+ if (IsA(expr, Var))
+ {
+ Var *var = castNode(Var, expr);
+ RelOptInfo *rel = root->simple_rel_array[var->varno];
+ Oid reloid = rel->relid;
+
+ if (reloid != InvalidOid && var->varattno > 0)
+ {
+ int32 width = get_attavgwidth(reloid, var->varattno);
+
+ if (width > 0)
+ {
+ gvi->gv_width += width;
+ continue;
+ }
+ }
+ }
+
+ /*
+ * In general we should not expect any statistics to exist for an
+ * expression just because it's a grouping expression. So use the type
+ * information to get the width estimate.
+ */
+ gvi->gv_width = get_typavgwidth(exprType((Node *) gvi->gvexpr),
+ exprTypmod((Node *) gvi->gvexpr));
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ /*
+ * Check if rel->reltarget allows the relation to be grouped and initialize
+ * the grouping target(s).
+ *
+ * target_agg is a target that we'll eventually used to aggregate the output
+ * of relation paths.
+ *
+ * If we succeed to create the grouping target, also replace rel->reltarget
+ * with a new one that has sortgrouprefs initialized --- this is necessary for
+ * create_agg_plan to match the grouping clauses against the input target
+ * expressions. (Thus the scan / join that provides the AggPath with input
+ * does not have to care whether the source relation does have grouped target
+ * or not.)
+ *
+ * The *group_exprs_extra_p list may receive additional grouping expressions
+ * that the query does not have. These can make the aggregation of base
+ * relation / join less efficient, but can allow for join of the grouped
+ * relation that wouldn't be possible otherwise.
+ */
+ void
+ initialize_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target_agg,
+ List **group_exprs_extra_p)
+ {
+ PathTarget *target_plain;
+ ListCell *lc1;
+ List *vars_unresolved = NIL;
+
+ /* The target to replace rel->reltarget. */
+ target_plain = create_empty_pathtarget();
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ GroupedVar *gvar;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var. (As for generic grouping expressions,
+ * add_grouped_vars_to_target will retrieve them from the query
+ * targetlist and add them to target_agg outside this function.)
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ gvar = get_grouping_expression(root, (Expr *) tvar);
+ if (gvar != NULL)
+ {
+ /*
+ * It's o.k. to use the target expression for grouping.
+ *
+ * The actual Var is added to the target. If we used the
+ * containing GroupedVar, references from various clauses (e.g.
+ * join quals) wouldn't work.
+ */
+ add_column_to_pathtarget(target_agg, (Expr *) gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /*
+ * As for the plain target, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(target_plain, gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the plain-var grouping expressions gathered. So far
+ * only store the var in a list.
+ */
+ vars_unresolved = lappend(vars_unresolved, tvar);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ foreach(lc1, vars_unresolved)
+ {
+ Var *var;
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ var = lfirst_node(Var, lc1);
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Dependent var is almost the same as one that has sortgroupref.
+ */
+ if (check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target_agg->exprs, &deps))
+ {
+
+ Index sortgroupref = 0;
+
+ add_column_to_pathtarget(target_agg, (Expr *) var, sortgroupref);
+
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important. But once we have it ...
+ */
+ add_column_to_pathtarget(target_plain, (Expr *) var,
+ sortgroupref);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) or aggregate arguments, but we already have it in
+ * the targets, so don't care.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon as GroupedVars are installed, the top level
+ * aggregates / grouping expressions no longer need direct reference
+ * to arg_var anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by upper join. This includes one that is
+ * referenced by a generic grouping expression but couldn't be
+ * recognized as grouping expression on its own at the top of the
+ * loop.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ *
+ * Since root->parse->groupClause is not supposed to contain this
+ * expression, we need to construct special SortGroupClause. Its
+ * tleSortGroupRef needs to be unique within target_agg, so
+ * postpone creation of the SortGroupRefs until we're done with
+ * the iteration of rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced either by aggregate argument
+ * or by generic grouping expression. The per-relation aggregation
+ * target should not contain it, as it only provides input for the
+ * final aggregation.
+ */
+ }
+
+ /*
+ * The var is not suitable for grouping, but the plain target ought to
+ * stay complete.
+ */
+ add_column_to_pathtarget(target_plain, (Expr *) var, 0);
+ }
+
+ /*
+ * Apply the adjusted input target as the replacement is complete now.
+ */
+ /* TODO Free the old one. */
+ rel->reltarget = target_plain;
+ }
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 2e6a60e..9a15e5a
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 178,183 ****
--- 178,191 ----
(*qp_callback) (root, qp_extra);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, Initialize GroupedPathInfo
+ * of base relations capable to do the grouping and setup
+ * root->grouped_var_list.
+ */
+ add_grouping_info_to_base_rels(root);
+
+ /*
* Examine any "placeholder" expressions generated during subquery pullup.
* Make sure that the Vars they need are marked as needed at the relevant
* join level. This must be done before join removal because it might
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 674cfc6..08686f6
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 27,32 ****
--- 27,33 ----
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "parser/parse_oper.h"
#include "utils/hsearch.h"
*************** build_joinrel_partition_info(RelOptInfo
*** 1748,1750 ****
--- 1749,2083 ----
joinrel->nullable_partexprs[cnt] = nullable_partexpr;
}
}
+
+ /*
+ * If the relation can produce grouped paths, create GroupedPathInfo for it
+ * and create target for aggregation of the relation output.
+ */
+ void
+ prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *gvis;
+ List *aggregates = NIL;
+ List *grp_exprs = NIL;
+ bool found_higher_agg;
+ ListCell *lc;
+ GroupedPathInfo *gpi;
+ PathTarget *target;
+ List *grp_exprs_extra = NIL;
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some grouping expression or Aggref
+ * argument), we'd just let initialize_grouped_targets create GroupedVar
+ * for the corresponding expression (phexpr). On the other hand, if we
+ * knew that the PHV is evaluated below the current rel, we'd ignore it
+ * because the referencing GroupedVar would take care of propagation of
+ * the value to upper joins. (PHV whose ph_eval_at is above the current
+ * rel make the aggregation push-down impossible in any case because the
+ * partial aggregation would receive wrong input if we ignored the
+ * ph_eval_at.)
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return;
+ }
+
+ /*
+ * target_agg will be used to aggregate the output of the current
+ * relation's paths.
+ */
+ target = create_empty_pathtarget();
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and initialize_grouped_targets_target
+ * take care?) of this rel to NULL are provably below rel. (It's ok if rel
+ * is one of these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * While create_grouping_expr_grouped_var_infos generates equivalence
+ * class implied plain-Var grouping expressions at once, doing so for
+ * generic expressions would be complex and might result in very long list
+ * to search in. So generate the EC-related grouping expressions for each
+ * particular set of relids.
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ /*
+ * Should have been processed by
+ * create_grouping_expr_grouped_var_infos.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ gvi = translate_expression_to_rels(root, gvi, rel->relids);
+ if (gvi != NULL)
+ gvis = lappend(gvis, gvi);
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_higher_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * initialize_grouped_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ *
+ * XXX A plain Var could actually be handled w/o GroupedVar, but
+ * thus initialize_grouped_targets would have to spend extra
+ * effort looking for the EC-related vars, instead of relying on
+ * create_grouping_expr_grouped_var_infos. (Processing of
+ * particular expression would look different, so we could hardly
+ * reuse the same piece of code.)
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Accept the aggregate / grouping expression.
+ *
+ * (GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.)
+ */
+ if (IsA(gvi->gvexpr, Aggref))
+ aggregates = lappend(aggregates, gvi);
+ else
+ grp_exprs = lappend(grp_exprs, gvi);
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids) &&
+ IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate / grouping
+ * expression that needs more than this rel.
+ */
+ found_higher_agg = true;
+ }
+ }
+ list_free(gvis);
+
+ /*
+ * Give up if some other aggregate(s) need multiple relations including
+ * the current one. The problem is that grouping of the current relation
+ * could make some input variables unavailable for the "higher aggregate",
+ * and it'd also decrease the number of input rows the "higher aggregate"
+ * receives.
+ *
+ * In contrast, grp_exprs is only supposed to contain generic grouping
+ * expression, so it can be NIL so far. If all the grouping keys are just
+ * plain Vars, initialize_grouped_targets will take care of them.
+ */
+ if (found_higher_agg)
+ return;
+
+ /*
+ * Add plain-var grouping expressions to the target.
+ */
+ initialize_grouped_target(root, rel, target, &grp_exprs_extra);
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ *
+ * This test was postponed because initialize_grouped_targets initializes
+ * sortgrouprefs of rel->reltarget. This information is useful to avoid
+ * final aggregation. In particular, even non-grouped relation (e.g.
+ * unique index scan) can generate unique values of grouping keys. See
+ * check_group_key_uniqueness for details.
+ */
+ if (aggregates == NIL)
+ return;
+
+ /*
+ * Add (non-Var) grouping expressions (in the form of GroupedVar) to
+ * target_agg.
+ *
+ * Follow the convention that the grouping expressions should precede the
+ * aggregates.
+ */
+ add_grouped_vars_to_target(root, target, grp_exprs);
+
+ /*
+ * Initialize GroupedPathInfo.
+ */
+ Assert(rel->gpi == NULL);
+ gpi = makeNode(GroupedPathInfo);
+ gpi->pathlist = NIL;
+ gpi->partial_pathlist = NIL;
+ rel->gpi = gpi;
+
+ /*
+ * Partial aggregation makes no sense w/o grouping expressions.
+ */
+ if (list_length(target->exprs) == 0)
+ {
+ pfree(rel->gpi);
+ rel->gpi = NULL;
+ return;
+ }
+
+ /*
+ * If the aggregation target should have extra grouping expressions, add
+ * them now. This step includes assignment of tleSortGroupRef's which we
+ * can generate now (the "ordinary" grouping expression are present in the
+ * target by now).
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+ PathTarget *target_plain;
+ int i = 0;
+ ListCell *lc2;
+
+ /*
+ * TODO Verify that these fields are sufficient for this special
+ * SortGroupClause.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ NULL, &cl->eqop, NULL,
+ &cl->hashable);
+ gpi->sortgroupclauses = lappend(gpi->sortgroupclauses, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too. It can
+ * already be there, so avoid adding it again.
+ */
+ target_plain = rel->reltarget;
+ foreach(lc2, target_plain->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc2);
+
+ if (equal(expr, var))
+ {
+ /*
+ * The fact that the var is there does not imply that it
+ * has sortgroupref set. For example, the reason that it's
+ * there can be that a generic grouping expression
+ * references it, so grouping by the var alone hasn't been
+ * considered so far.
+ */
+ if (target_plain->sortgrouprefs == NULL)
+ {
+ target_plain->sortgrouprefs = (Index *)
+ palloc0(list_length(target_plain->exprs) *
+ sizeof(Index));
+ }
+ if (target_plain->sortgrouprefs[i] == 0)
+ target_plain->sortgrouprefs[i] = cl->tleSortGroupRef;
+
+ break;
+ }
+
+ i++;
+ }
+ if (lc2 != NULL)
+ continue;
+
+ /*
+ * Add the var if it's not in the target yet.
+ */
+ add_column_to_pathtarget(target_plain, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the target(s).
+ */
+ add_grouped_vars_to_target(root, target, aggregates);
+
+ /* TODO Check if copy is needed here. */
+ gpi->target = copy_pathtarget(target);
+
+ /*
+ * Add the non-Var group expressions to a separate target. If
+ * rel->reltarget should be used to generate input for partial
+ * aggregation, the corresponding path should include these expressions.
+ */
+ if (list_length(grp_exprs) > 0)
+ {
+ rel->gpi->group_exprs = create_empty_pathtarget();
+ add_grouped_vars_to_target(root, rel->gpi->group_exprs, grp_exprs);
+ }
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 9345891..abdc5d9
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 783,853 ----
}
/*
+ * For each aggregate add GroupedVar to the grouped target.
+ *
+ * Caller passes the aggregates in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+ void
+ add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+ {
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+ GroupedVar *gvar;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_column_to_pathtarget(target, (Expr *) gvar, gvi->sortgroupref);
+ }
+ }
+
+ /*
+ * Return GroupedVar containing the passed-in expression if one exists, or
+ * NULL if the expression cannot be used as grouping key.
+ *
+ * create_grouping_expr_grouped_var_infos should have been called before this
+ * function gets called the first time.
+ */
+ GroupedVar *
+ get_grouping_expression(PlannerInfo *root, Expr *expr)
+ {
+ ListCell *lc;
+
+ /* Caller should have noticed the empty list much earlier. */
+ Assert(root->grouped_var_list != NIL);
+
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ GroupedVar *result = makeNode(GroupedVar);
+
+ Assert(gvi->sortgroupref > 0);
+ result->gvexpr = gvi->gvexpr;
+ result->gvid = gvi->gvid;
+ result->sortgroupref = gvi->sortgroupref;
+ return result;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return NULL;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
new file mode 100644
index e367221..7090d02
*** a/src/include/optimizer/clauses.h
--- b/src/include/optimizer/clauses.h
*************** extern Node *estimate_expression_value(P
*** 84,88 ****
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
!
#endif /* CLAUSES_H */
--- 84,90 ----
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
! extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
! GroupedVarInfo *gvi,
! Relids relids);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 3ef12b3..08111d2
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern RelOptInfo *build_child_join_rel(
*** 300,304 ****
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
!
#endif /* PATHNODE_H */
--- 300,304 ----
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
! extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index d613322..a4c6f5d
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** extern void add_base_rels_to_query(Plann
*** 76,81 ****
--- 76,83 ----
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+ extern void add_grouping_info_to_base_rels(PlannerInfo *root);
+ extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 0d3ec92..a725818
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern Node *get_sortgroupclause_expr(So
*** 41,46 ****
--- 41,50 ----
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
+ extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List *target_group_clauses,
+ List **grouping_clauses,
+ List **grouping_exprs, List **agg_exprs);
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 69,82 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+ extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
+ extern void initialize_grouped_target(PlannerInfo *root,
+ RelOptInfo *rel,
+ PathTarget *target_agg,
+ List **group_exprs_extra_p);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
agg_pushdown_v5/03_add_path.diff
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
new file mode 100644
index 370cc36..0690bd4
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
*************** fileGetForeignPaths(PlannerInfo *root,
*** 551,557 ****
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! coptions));
/*
* If data file was sorted, and we knew it somehow, we could insert
--- 551,557 ----
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! coptions), false);
/*
* If data file was sorted, and we knew it somehow, we could insert
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
new file mode 100644
index fb65e2e..459c6b9
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 907,913 ****
NULL, /* no outer rel either */
NULL, /* no extra plan */
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path);
/* Add paths with pathkeys */
add_paths_with_pathkeys_for_rel(root, baserel, NULL);
--- 907,913 ----
NULL, /* no outer rel either */
NULL, /* no extra plan */
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
/* Add paths with pathkeys */
add_paths_with_pathkeys_for_rel(root, baserel, NULL);
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 1079,1085 ****
param_info->ppi_req_outer,
NULL,
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path);
}
}
--- 1079,1085 ----
param_info->ppi_req_outer,
NULL,
NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
}
}
*************** add_paths_with_pathkeys_for_rel(PlannerI
*** 4343,4349 ****
useful_pathkeys,
NULL,
epq_path,
! NIL));
}
}
--- 4343,4350 ----
useful_pathkeys,
NULL,
epq_path,
! NIL),
! false);
}
}
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4574,4580 ****
NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath);
/* Consider pathkeys for the join relation */
add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
--- 4575,4581 ----
NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, false);
/* Consider pathkeys for the join relation */
add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4898,4904 ****
NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
! add_path(grouped_rel, (Path *) grouppath);
}
/*
--- 4899,4905 ----
NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
! add_path(grouped_rel, (Path *) grouppath, false);
}
/*
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 0e8463e..6872e14
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 698,704 ****
required_outer = rel->lateral_relids;
/* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
/* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
--- 698,704 ----
required_outer = rel->lateral_relids;
/* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0), false);
/* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
*************** create_plain_partial_paths(PlannerInfo *
*** 727,733 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 727,734 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers),
! false);
}
/*
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 813,819 ****
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path);
/* For the moment, at least, there are no other paths to consider */
}
--- 814,820 ----
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path, false);
/* For the moment, at least, there are no other paths to consider */
}
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1550,1556 ****
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NIL,
NULL, 0, false,
! partitioned_rels, -1));
/*
* Consider an append of unordered, unparameterized partial paths. Make
--- 1551,1558 ----
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NIL,
NULL, 0, false,
! partitioned_rels, -1),
! false);
/*
* Consider an append of unordered, unparameterized partial paths. Make
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1602,1608 ****
partial_rows = appendpath->path.rows;
/* Add the path. */
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1604,1610 ----
partial_rows = appendpath->path.rows;
/* Add the path. */
! add_partial_path(rel, (Path *) appendpath, false);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1643,1649 ****
pa_partial_subpaths,
NULL, parallel_workers, true,
partitioned_rels, partial_rows);
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1645,1651 ----
pa_partial_subpaths,
NULL, parallel_workers, true,
partitioned_rels, partial_rows);
! add_partial_path(rel, (Path *) appendpath, false);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1697,1703 ****
add_path(rel, (Path *)
create_append_path(rel, subpaths, NIL,
required_outer, 0, false,
! partitioned_rels, -1));
}
}
--- 1699,1705 ----
add_path(rel, (Path *)
create_append_path(rel, subpaths, NIL,
required_outer, 0, false,
! partitioned_rels, -1), false);
}
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1793,1806 ****
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels));
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels));
}
}
--- 1795,1810 ----
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
}
}
*************** set_dummy_rel_pathlist(RelOptInfo *rel)
*** 1961,1967 ****
rel->partial_pathlist = NIL;
add_path(rel, (Path *) create_append_path(rel, NIL, NIL, NULL,
! 0, false, NIL, -1));
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
--- 1965,1972 ----
rel->partial_pathlist = NIL;
add_path(rel, (Path *) create_append_path(rel, NIL, NIL, NULL,
! 0, false, NIL, -1),
! false);
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
*************** set_subquery_pathlist(PlannerInfo *root,
*** 2175,2181 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer));
}
}
--- 2180,2187 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer),
! false);
}
}
*************** set_function_pathlist(PlannerInfo *root,
*** 2244,2250 ****
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer));
}
/*
--- 2250,2256 ----
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer), false);
}
/*
*************** set_values_pathlist(PlannerInfo *root, R
*** 2264,2270 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer));
}
/*
--- 2270,2276 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer), false);
}
/*
*************** set_tablefunc_pathlist(PlannerInfo *root
*** 2285,2291 ****
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer));
}
/*
--- 2291,2297 ----
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer), false);
}
/*
*************** set_cte_pathlist(PlannerInfo *root, RelO
*** 2351,2357 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer));
}
/*
--- 2357,2363 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer), false);
}
/*
*************** set_namedtuplestore_pathlist(PlannerInfo
*** 2378,2384 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
--- 2384,2391 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
*************** set_worktable_pathlist(PlannerInfo *root
*** 2431,2437 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer));
}
/*
--- 2438,2445 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer),
! false);
}
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2463,2469 ****
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path);
/*
* For each useful ordering, we can consider an order-preserving Gather
--- 2471,2477 ----
simple_gather_path = (Path *)
create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path, false);
/*
* For each useful ordering, we can consider an order-preserving Gather
*************** generate_gather_paths(PlannerInfo *root,
*** 2479,2485 ****
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path);
}
}
--- 2487,2493 ----
path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, false);
}
}
*************** create_partial_bitmap_paths(PlannerInfo
*** 3304,3310 ****
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3312,3319 ----
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers),
! false);
}
/*
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index c3daacd..b0c7a0c
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
***************
*** 91,96 ****
--- 91,97 ----
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
*************** static Selectivity get_foreign_key_join_
*** 164,170 ****
List **restrictlist);
static Cost append_nonpartial_cost(List *subpaths, int numpaths,
int parallel_workers);
! static void set_rel_width(PlannerInfo *root, RelOptInfo *rel);
static double relation_byte_size(double tuples, int width);
static double page_size(double tuples, int width);
static double get_parallel_divisor(Path *path);
--- 165,172 ----
List **restrictlist);
static Cost append_nonpartial_cost(List *subpaths, int numpaths,
int parallel_workers);
! static void set_rel_width(PlannerInfo *root, RelOptInfo *rel,
! PathTarget *reltarget, bool grouped);
static double relation_byte_size(double tuples, int width);
static double page_size(double tuples, int width);
static double get_parallel_divisor(Path *path);
*************** set_baserel_size_estimates(PlannerInfo *
*** 4285,4291 ****
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel);
}
/*
--- 4287,4300 ----
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel, rel->reltarget, false);
!
! /*
! * If the relation can produce grouped path, estimate width and costs of
! * the corresponding target.
! */
! if (rel->gpi)
! set_rel_width(root, rel, rel->gpi->target, true);
}
/*
*************** set_foreign_size_estimates(PlannerInfo *
*** 5036,5042 ****
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel);
}
--- 5045,5051 ----
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel, rel->reltarget, false);
}
*************** set_foreign_size_estimates(PlannerInfo *
*** 5060,5068 ****
*
* The per-attribute width estimates are cached for possible re-use while
* building join relations or post-scan/join pathtargets.
*/
static void
! set_rel_width(PlannerInfo *root, RelOptInfo *rel)
{
Oid reloid = planner_rt_fetch(rel->relid, root)->relid;
int32 tuple_width = 0;
--- 5069,5081 ----
*
* The per-attribute width estimates are cached for possible re-use while
* building join relations or post-scan/join pathtargets.
+ *
+ * If called for grouped target, it's assumed that it the function was already
+ * called for the non-grouped one.
*/
static void
! set_rel_width(PlannerInfo *root, RelOptInfo *rel, PathTarget *reltarget,
! bool grouped)
{
Oid reloid = planner_rt_fetch(rel->relid, root)->relid;
int32 tuple_width = 0;
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 5070,5079 ****
ListCell *lc;
/* Vars are assumed to have cost zero, but other exprs do not */
! rel->reltarget->cost.startup = 0;
! rel->reltarget->cost.per_tuple = 0;
! foreach(lc, rel->reltarget->exprs)
{
Node *node = (Node *) lfirst(lc);
--- 5083,5092 ----
ListCell *lc;
/* Vars are assumed to have cost zero, but other exprs do not */
! reltarget->cost.startup = 0;
! reltarget->cost.per_tuple = 0;
! foreach(lc, reltarget->exprs)
{
Node *node = (Node *) lfirst(lc);
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 5115,5120 ****
--- 5128,5140 ----
continue;
}
+ /*
+ * The non-grouped target of this rel should already have been
+ * processed, so do not waste any effort to get zero width again.
+ */
+ if (grouped && rel->attr_widths[ndx] == 0)
+ continue;
+
/* Try to get column width from statistics */
if (reloid != InvalidOid && var->varattno > 0)
{
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 5148,5155 ****
tuple_width += phinfo->ph_width;
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
! rel->reltarget->cost.startup += cost.startup;
! rel->reltarget->cost.per_tuple += cost.per_tuple;
}
else
{
--- 5168,5195 ----
tuple_width += phinfo->ph_width;
cost_qual_eval_node(&cost, (Node *) phv->phexpr, root);
! reltarget->cost.startup += cost.startup;
! reltarget->cost.per_tuple += cost.per_tuple;
! }
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = (GroupedVar *) node;
! GroupedVarInfo *gvinfo;
!
! /*
! * GroupedVar should not appear in the "ordinary" target.
! */
! Assert(grouped);
!
! gvinfo = find_grouped_var_info(root, gvar);
! tuple_width += gvinfo->gv_width;
!
! /*
! * Only AggPath can evaluate GroupedVar, whether it's an aggregate
! * or generic grouping expression. In the other cases the
! * GroupedVar we see here only bubbled up from a lower AggPath, so
! * it does not add any cost to the path that owns this target.
! */
}
else
{
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 5166,5173 ****
tuple_width += item_width;
/* Not entirely clear if we need to account for cost, but do so */
cost_qual_eval_node(&cost, node, root);
! rel->reltarget->cost.startup += cost.startup;
! rel->reltarget->cost.per_tuple += cost.per_tuple;
}
}
--- 5206,5213 ----
tuple_width += item_width;
/* Not entirely clear if we need to account for cost, but do so */
cost_qual_eval_node(&cost, node, root);
! reltarget->cost.startup += cost.startup;
! reltarget->cost.per_tuple += cost.per_tuple;
}
}
*************** set_rel_width(PlannerInfo *root, RelOptI
*** 5204,5210 ****
}
Assert(tuple_width >= 0);
! rel->reltarget->width = tuple_width;
}
/*
--- 5244,5250 ----
}
Assert(tuple_width >= 0);
! reltarget->width = tuple_width;
}
/*
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 18f6baf..c8bd820
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
*************** create_index_paths(PlannerInfo *root, Re
*** 338,344 ****
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
--- 338,344 ----
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath, false);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
*************** create_index_paths(PlannerInfo *root, Re
*** 415,421 ****
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath);
}
}
}
--- 415,421 ----
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath, false);
}
}
}
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,795 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath);
if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
--- 789,795 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, false);
if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1077,1083 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1129,1135 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath, false);
else
pfree(ipath);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index e774130..2a931af
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** try_nestloop_path(PlannerInfo *root,
*** 422,428 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
/*
* If the inner path is parameterized, it is parameterized by the
--- 422,428 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
/*
* If the inner path is parameterized, it is parameterized by the
*************** try_nestloop_path(PlannerInfo *root,
*** 455,461 ****
inner_path,
extra->restrictlist,
pathkeys,
! required_outer));
}
else
{
--- 455,461 ----
inner_path,
extra->restrictlist,
pathkeys,
! required_outer), false);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 513,519 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/*
--- 513,520 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/*
*************** try_partial_nestloop_path(PlannerInfo *r
*** 543,549 ****
inner_path,
extra->restrictlist,
pathkeys,
! NULL));
}
/*
--- 544,550 ----
inner_path,
extra->restrictlist,
pathkeys,
! NULL), false);
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 617,623 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
--- 618,624 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
*************** try_mergejoin_path(PlannerInfo *root,
*** 632,638 ****
required_outer,
mergeclauses,
outersortkeys,
! innersortkeys));
}
else
{
--- 633,639 ----
required_outer,
mergeclauses,
outersortkeys,
! innersortkeys), false);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 691,697 ****
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/* Might be good enough to be worth trying, so let's try it. */
--- 692,699 ----
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/* Might be good enough to be worth trying, so let's try it. */
*************** try_partial_mergejoin_path(PlannerInfo *
*** 708,714 ****
NULL,
mergeclauses,
outersortkeys,
! innersortkeys));
}
/*
--- 710,716 ----
NULL,
mergeclauses,
outersortkeys,
! innersortkeys), false);
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 751,757 ****
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
add_path(joinrel, (Path *)
create_hashjoin_path(root,
--- 753,759 ----
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, false))
{
add_path(joinrel, (Path *)
create_hashjoin_path(root,
*************** try_hashjoin_path(PlannerInfo *root,
*** 764,770 ****
false, /* parallel_hash */
extra->restrictlist,
required_outer,
! hashclauses));
}
else
{
--- 766,772 ----
false, /* parallel_hash */
extra->restrictlist,
required_outer,
! hashclauses), false);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 815,821 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, true);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
/* Might be good enough to be worth trying, so let's try it. */
--- 817,823 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, true);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL, false))
return;
/* Might be good enough to be worth trying, so let's try it. */
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 830,836 ****
parallel_hash,
extra->restrictlist,
NULL,
! hashclauses));
}
/*
--- 832,838 ----
parallel_hash,
extra->restrictlist,
NULL,
! hashclauses), false);
}
/*
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 5e03f8b..285f269
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** mark_dummy_rel(RelOptInfo *rel)
*** 1233,1239 ****
/* Set up the dummy path */
add_path(rel, (Path *) create_append_path(rel, NIL, NIL, NULL,
! 0, false, NIL, -1));
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
--- 1233,1240 ----
/* Set up the dummy path */
add_path(rel, (Path *) create_append_path(rel, NIL, NIL, NULL,
! 0, false, NIL, -1),
! false);
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index a2fe661..91d855c
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** create_tidscan_paths(PlannerInfo *root,
*** 266,270 ****
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 266,270 ----
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer), false);
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 8de59fc..faa654b
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
*************** create_grouping_expr_grouped_var_infos(P
*** 653,658 ****
--- 653,659 ----
}
}
+
/*
* Check if rel->reltarget allows the relation to be grouped and initialize
* the grouping target(s).
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 889e8af..351fb93
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,229 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual));
}
/*
--- 223,230 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual),
! false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 9a15e5a..c1939a1
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,89 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,90 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 5a3efbd..8c524ac
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** inheritance_planner(PlannerInfo *root)
*** 1520,1526 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)));
}
/*--------------------
--- 1520,1527 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)),
! false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 2135,2141 ****
}
/* And shove it into final_rel */
! add_path(final_rel, path);
}
/*
--- 2136,2142 ----
}
/* And shove it into final_rel */
! add_path(final_rel, path, false);
}
/*
*************** create_grouping_paths(PlannerInfo *root,
*** 3698,3704 ****
(List *) parse->havingQual);
}
! add_path(grouped_rel, path);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
--- 3699,3705 ----
(List *) parse->havingQual);
}
! add_path(grouped_rel, path, false);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
*************** create_grouping_paths(PlannerInfo *root,
*** 3875,3881 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
--- 3876,3883 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3884,3890 ****
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups));
}
}
}
--- 3886,3893 ----
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3915,3921 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
}
}
}
--- 3918,3924 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3967,3973 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
else if (parse->groupClause)
{
--- 3970,3976 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
else if (parse->groupClause)
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3982,3988 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
else
{
--- 3985,3991 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
else
{
*************** create_grouping_paths(PlannerInfo *root,
*** 4031,4037 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 4034,4040 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4040,4046 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
/*
* The point of using Gather Merge rather than Gather is that it
--- 4043,4049 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
/*
* The point of using Gather Merge rather than Gather is that it
*************** create_grouping_paths(PlannerInfo *root,
*** 4093,4099 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 4096,4102 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4102,4108 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
}
}
--- 4105,4111 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4147,4153 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
}
--- 4150,4156 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4185,4191 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
}
}
}
--- 4188,4194 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
}
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4387,4393 ****
strat,
new_rollups,
agg_costs,
! dNumGroups));
return;
}
--- 4390,4396 ----
strat,
new_rollups,
agg_costs,
! dNumGroups), false);
return;
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4545,4551 ****
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups));
}
}
--- 4548,4554 ----
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups), false);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4562,4568 ****
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups));
}
/*
--- 4565,4571 ----
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups), false);
}
/*
*************** create_one_window_path(PlannerInfo *root
*** 4747,4753 ****
window_pathkeys);
}
! add_path(window_rel, path);
}
/*
--- 4750,4756 ----
window_pathkeys);
}
! add_path(window_rel, path, false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4853,4859 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
}
--- 4856,4862 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4880,4886 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
/*
--- 4883,4889 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4927,4933 ****
parse->distinctClause,
NIL,
NULL,
! numDistinctRows));
}
/* Give a helpful error if we failed to find any implementation */
--- 4930,4936 ----
parse->distinctClause,
NIL,
NULL,
! numDistinctRows), false);
}
/* Give a helpful error if we failed to find any implementation */
*************** create_ordered_paths(PlannerInfo *root,
*** 5025,5031 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5028,5034 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
*************** create_ordered_paths(PlannerInfo *root,
*** 5076,5082 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5079,5085 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
new file mode 100644
index a24e8ac..eb8050f
*** a/src/backend/optimizer/prep/prepunion.c
--- b/src/backend/optimizer/prep/prepunion.c
*************** plan_set_operations(PlannerInfo *root)
*** 221,227 ****
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
--- 221,227 ----
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path, false);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 2aee156..a1bea53
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
*************** set_cheapest(RelOptInfo *parent_rel)
*** 419,426 ****
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
--- 419,427 ----
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
+ List *pathlist;
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 437,442 ****
--- 438,451 ----
/* Pretend parameterized paths have no pathkeys, per comment above */
new_path_pathkeys = new_path->param_info ? NIL : new_path->pathkeys;
+ if (!grouped)
+ pathlist = parent_rel->pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->pathlist;
+ }
+
/*
* Loop to check proposed new path against old paths. Note it is possible
* for more than one old path to be tossed out because new_path dominates
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 446,452 ****
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
--- 455,461 ----
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 592,599 ****
*/
if (remove_old)
{
! parent_rel->pathlist = list_delete_cell(parent_rel->pathlist,
! p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
--- 601,607 ----
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 624,632 ****
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(parent_rel->pathlist, insert_after, new_path);
else
! parent_rel->pathlist = lcons(new_path, parent_rel->pathlist);
}
else
{
--- 632,645 ----
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->pathlist = pathlist;
! else
! parent_rel->gpi->pathlist = pathlist;
}
else
{
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 656,663 ****
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer)
{
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
--- 669,677 ----
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped)
{
+ List *pathlist;
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 666,674 ****
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup : parent_rel->consider_startup;
! foreach(p1, parent_rel->pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 680,697 ----
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup :
! parent_rel->consider_startup;
! if (!grouped)
! pathlist = parent_rel->pathlist;
! else
! {
! Assert(parent_rel->gpi != NULL);
! pathlist = parent_rel->gpi->pathlist;
! }
!
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 759,781 ****
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->partial_pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
--- 782,813 ----
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
+ List *pathlist;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
+
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
*************** add_partial_path(RelOptInfo *parent_rel,
*** 829,840 ****
}
/*
! * Remove current element from partial_pathlist if dominated by new.
*/
if (remove_old)
{
! parent_rel->partial_pathlist =
! list_delete_cell(parent_rel->partial_pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
--- 861,871 ----
}
/*
! * Remove current element from pathlist if dominated by new.
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
*************** add_partial_path(RelOptInfo *parent_rel,
*** 849,856 ****
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the partial_pathlist; we will not add new_path, and we
! * assume new_path cannot dominate any later path.
*/
if (!accept_new)
break;
--- 880,887 ----
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the pathlist; we will not add new_path, and we assume
! * new_path cannot dominate any later path.
*/
if (!accept_new)
break;
*************** add_partial_path(RelOptInfo *parent_rel,
*** 860,869 ****
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(parent_rel->partial_pathlist, insert_after, new_path);
else
! parent_rel->partial_pathlist =
! lcons(new_path, parent_rel->partial_pathlist);
}
else
{
--- 891,904 ----
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->partial_pathlist = pathlist;
! else
! parent_rel->gpi->partial_pathlist = pathlist;
}
else
{
*************** add_partial_path(RelOptInfo *parent_rel,
*** 884,892 ****
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys)
{
ListCell *p1;
/*
* Our goal here is twofold. First, we want to find out whether this path
--- 919,936 ----
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys, bool grouped)
{
ListCell *p1;
+ List *pathlist;
+
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
/*
* Our goal here is twofold. First, we want to find out whether this path
*************** add_partial_path_precheck(RelOptInfo *pa
*** 896,905 ****
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist to be very short, and getting a definitive
! * answer at this stage avoids the need to call add_path_precheck.
*/
! foreach(p1, parent_rel->partial_pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 940,950 ----
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist / grouped_pathlist to be very short, and
! * getting a definitive answer at this stage avoids the need to call
! * add_path_precheck.
*/
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_partial_path_precheck(RelOptInfo *pa
*** 928,934 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL))
return false;
return true;
--- 973,979 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, false))
return false;
return true;
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
new file mode 100644
index 81c60dc..8ecb21d
*** a/src/backend/optimizer/util/var.c
--- b/src/backend/optimizer/util/var.c
*************** alias_relid_set(PlannerInfo *root, Relid
*** 840,842 ****
--- 840,864 ----
}
return result;
}
+
+ /*
+ * Return GroupedVarInfo for given GroupedVar.
+ *
+ * XXX Consider better location of this routine.
+ */
+ GroupedVarInfo *
+ find_grouped_var_info(PlannerInfo *root, GroupedVar *gvar)
+ {
+ ListCell *l;
+
+ foreach(l, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, l);
+
+ if (gvi->gvid == gvar->gvid)
+ return gvi;
+ }
+
+ elog(ERROR, "GroupedVarInfo not found");
+ return NULL; /* keep compiler quiet */
+ }
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 08111d2..9c05492
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern int compare_path_costs(Path *path
*** 26,38 ****
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
--- 26,40 ----
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path,
! bool grouped);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys,
! bool grouped);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
diff --git a/src/include/optimizer/var.h b/src/include/optimizer/var.h
new file mode 100644
index 6186152..22816db
*** a/src/include/optimizer/var.h
--- b/src/include/optimizer/var.h
*************** extern bool contain_vars_of_level(Node *
*** 36,40 ****
--- 36,42 ----
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
extern Node *flatten_join_alias_vars(PlannerInfo *root, Node *node);
+ extern GroupedVarInfo *find_grouped_var_info(PlannerInfo *root,
+ GroupedVar *gvar);
#endif /* VAR_H */
agg_pushdown_v5/04_planner_prepare.diff
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 8c524ac..6371c34
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 131,139 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
--- 131,136 ----
*************** static void consider_groupingsets_paths(
*** 148,153 ****
--- 145,162 ----
grouping_sets_data *gd,
const AggClauseCosts *agg_costs,
double dNumGroups);
+ static void sort_agg_partial_grouped_paths(PlannerInfo *root, List *pathlist,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups,
+ RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target);
+ static void hash_agg_partial_grouped_path(PlannerInfo *root, Path *path,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups,
+ RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target);
static RelOptInfo *create_window_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *input_target,
*************** get_number_of_groups(PlannerInfo *root,
*** 3542,3581 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3551,3556 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 4000,4114 ****
* path. We can do this using either Gather or Gather Merge.
*/
if (grouped_rel->partial_pathlist)
! {
! Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
! double total_groups = path->rows * path->parallel_workers;
!
! path = (Path *) create_gather_path(root,
! grouped_rel,
! path,
! partial_grouping_target,
! NULL,
! &total_groups);
!
! /*
! * Since Gather's output is always unsorted, we'll need to sort,
! * unless there's no GROUP BY clause or a degenerate (constant)
! * one, in which case there will only be a single group.
! */
! if (root->group_pathkeys)
! path = (Path *) create_sort_path(root,
! grouped_rel,
! path,
! root->group_pathkeys,
! -1.0);
!
! if (parse->hasAggs)
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! path,
! target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! else
! add_path(grouped_rel, (Path *)
! create_group_path(root,
! grouped_rel,
! path,
! target,
! parse->groupClause,
! (List *) parse->havingQual,
! dNumGroups), false);
!
! /*
! * The point of using Gather Merge rather than Gather is that it
! * can preserve the ordering of the input path, so there's no
! * reason to try it unless (1) it's possible to produce more than
! * one output row and (2) we want the output path to be ordered.
! */
! if (parse->groupClause != NIL && root->group_pathkeys != NIL)
! {
! foreach(lc, grouped_rel->partial_pathlist)
! {
! Path *subpath = (Path *) lfirst(lc);
! Path *gmpath;
! double total_groups;
!
! /*
! * It's useful to consider paths that are already properly
! * ordered for Gather Merge, because those don't need a
! * sort. It's also useful to consider the cheapest path,
! * because sorting it in parallel and then doing Gather
! * Merge may be better than doing an unordered Gather
! * followed by a sort. But there's no point in
! * considering non-cheapest paths that aren't already
! * sorted correctly.
! */
! if (path != subpath &&
! !pathkeys_contained_in(root->group_pathkeys,
! subpath->pathkeys))
! continue;
!
! total_groups = subpath->rows * subpath->parallel_workers;
!
! gmpath = (Path *)
! create_gather_merge_path(root,
! grouped_rel,
! subpath,
! partial_grouping_target,
! root->group_pathkeys,
! NULL,
! &total_groups);
!
! if (parse->hasAggs)
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! gmpath,
! target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! else
! add_path(grouped_rel, (Path *)
! create_group_path(root,
! grouped_rel,
! gmpath,
! target,
! parse->groupClause,
! (List *) parse->havingQual,
! dNumGroups), false);
! }
! }
! }
}
if (can_hash)
--- 3975,3985 ----
* path. We can do this using either Gather or Gather Merge.
*/
if (grouped_rel->partial_pathlist)
! sort_agg_partial_grouped_paths(root,
! grouped_rel->partial_pathlist,
! &agg_final_costs,
! dNumGroups, grouped_rel,
! partial_grouping_target, target);
}
if (can_hash)
*************** create_grouping_paths(PlannerInfo *root,
*** 4163,4195 ****
{
Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
! hashaggtablesize = estimate_hashagg_tablesize(path,
! &agg_final_costs,
! dNumGroups);
!
! if (hashaggtablesize < work_mem * 1024L)
! {
! double total_groups = path->rows * path->parallel_workers;
!
! path = (Path *) create_gather_path(root,
! grouped_rel,
! path,
! partial_grouping_target,
! NULL,
! &total_groups);
!
! add_path(grouped_rel, (Path *)
! create_agg_path(root,
! grouped_rel,
! path,
! target,
! AGG_HASHED,
! AGGSPLIT_FINAL_DESERIAL,
! parse->groupClause,
! (List *) parse->havingQual,
! &agg_final_costs,
! dNumGroups), false);
! }
}
}
--- 4034,4042 ----
{
Path *path = (Path *) linitial(grouped_rel->partial_pathlist);
! hash_agg_partial_grouped_path(root, path, &agg_final_costs,
! dNumGroups, grouped_rel,
! partial_grouping_target, target);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4569,4574 ****
--- 4416,4588 ----
}
/*
+ * Subroutine of create_grouping_paths() to apply GatherPath or
+ * GatherMergePath and AGG_SORTED AggPath to cases which only differ in the
+ * GatherPath / GatherMergePath target.
+ */
+ static void
+ sort_agg_partial_grouped_paths(PlannerInfo *root, List *pathlist,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups, RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target)
+ {
+ Path *path = (Path *) linitial(pathlist);
+ double total_groups = path->rows * path->parallel_workers;
+ Query *parse = root->parse;
+ ListCell *lc;
+
+ /*
+ * Generate a complete GroupAgg Path atop of the cheapest partial path. We
+ * can do this using either Gather or Gather Merge.
+ */
+ path = (Path *) create_gather_path(root,
+ grouped_rel,
+ path,
+ gather_target,
+ NULL,
+ &total_groups);
+
+ /*
+ * Since Gather's output is always unsorted, we'll need to sort, unless
+ * there's no GROUP BY clause or a degenerate (constant) one, in which
+ * case there will only be a single group.
+ */
+ if (root->group_pathkeys)
+ path = (Path *) create_sort_path(root,
+ grouped_rel,
+ path,
+ root->group_pathkeys,
+ -1.0);
+
+ if (parse->hasAggs)
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ parse->groupClause ? AGG_SORTED : AGG_PLAIN,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ else
+ add_path(grouped_rel, (Path *)
+ create_group_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ dNumGroups), false);
+
+ /*
+ * The point of using Gather Merge rather than Gather is that it can
+ * preserve the ordering of the input path, so there's no reason to try it
+ * unless (1) it's possible to produce more than one output row and (2) we
+ * want the output path to be ordered.
+ */
+ if (parse->groupClause != NIL && root->group_pathkeys != NIL)
+ {
+ foreach(lc, pathlist)
+ {
+ Path *subpath = (Path *) lfirst(lc);
+ Path *gmpath;
+ double total_groups;
+
+ /*
+ * It's useful to consider paths that are already properly ordered
+ * for Gather Merge, because those don't need a sort. It's also
+ * useful to consider the cheapest path, because sorting it in
+ * parallel and then doing Gather Merge may be better than doing
+ * an unordered Gather followed by a sort. But there's no point
+ * in considering non-cheapest paths that aren't already sorted
+ * correctly.
+ */
+ if (path != subpath &&
+ !pathkeys_contained_in(root->group_pathkeys,
+ subpath->pathkeys))
+ continue;
+
+ total_groups = subpath->rows * subpath->parallel_workers;
+
+ gmpath = (Path *)
+ create_gather_merge_path(root,
+ grouped_rel,
+ subpath,
+ gather_target,
+ root->group_pathkeys,
+ NULL,
+ &total_groups);
+
+ if (parse->hasAggs)
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ gmpath,
+ group_target,
+ parse->groupClause ? AGG_SORTED : AGG_PLAIN,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ else
+ add_path(grouped_rel, (Path *)
+ create_group_path(root,
+ grouped_rel,
+ gmpath,
+ group_target,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ dNumGroups), false);
+ }
+ }
+ }
+
+ /*
+ * Subroutine of create_grouping_paths() to apply GatherPath and AGG_HASHED
+ * AggPath to cases which only differ in the GatherPath target.
+ */
+ static void
+ hash_agg_partial_grouped_path(PlannerInfo *root, Path *path,
+ AggClauseCosts *agg_final_costs,
+ double dNumGroups, RelOptInfo *grouped_rel,
+ PathTarget *gather_target,
+ PathTarget *group_target)
+ {
+ Size hashaggtablesize = estimate_hashagg_tablesize(path, agg_final_costs,
+ dNumGroups);
+ Query *parse = root->parse;
+ double total_groups;
+
+ if (hashaggtablesize >= work_mem * 1024L)
+ return;
+
+ total_groups = path->rows * path->parallel_workers;
+
+ path = (Path *) create_gather_path(root,
+ grouped_rel,
+ path,
+ gather_target,
+ NULL,
+ &total_groups);
+
+ add_path(grouped_rel, (Path *)
+ create_agg_path(root,
+ grouped_rel,
+ path,
+ group_target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ agg_final_costs,
+ dNumGroups), false);
+ }
+
+ /*
* create_window_paths
*
* Build a new upperrel containing Paths for window-function evaluation.
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index ea95b80..87dec17
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 114,119 ****
--- 114,120 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
*************** estimate_hash_bucket_stats(PlannerInfo *
*** 3863,3868 ****
--- 3864,3902 ----
ReleaseVariableStats(vardata);
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 199a631..e2435b6
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern void estimate_hash_bucket_stats(P
*** 210,215 ****
--- 210,219 ----
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
+
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
agg_pushdown_v5/05_base_rel.diff
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index 55bb925..f09647e
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, ExprState *s
*** 789,794 ****
--- 789,831 ----
break;
}
+ case T_GroupedVar:
+
+ /*
+ * If GroupedVar appears in targetlist of Agg node, it can
+ * represent either Aggref or grouping expression.
+ */
+ if (state->parent && (IsA(state->parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * The only reason to execute GroupedVar is to generate either
+ * aggregate transient state or grouping expression value. So
+ * any contained Aggref must be partial.
+ *
+ * (The purpose of propagating GroupedVars to upper plan nodes
+ * is just to transfer the value, no execution takes place
+ * there.)
+ */
+ if (IsA(gvar->gvexpr, Aggref))
+
+ ExecInitExprRec((Expr *) gvar->agg_partial, state,
+ resv, resnull);
+ else
+ ExecInitExprRec((Expr *) gvar->gvexpr, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ */
+ elog(ERROR, "parent of GroupedVar is not Agg node");
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
new file mode 100644
index da6ef1a..2dabe4c
*** a/src/backend/executor/nodeAgg.c
--- b/src/backend/executor/nodeAgg.c
*************** find_unaggregated_cols_walker(Node *node
*** 1875,1880 ****
--- 1875,1887 ----
/* do not descend into aggregate exprs */
return false;
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ if (IsA(gvar->gvexpr, Aggref))
+ return false;
+ }
return expression_tree_walker(node, find_unaggregated_cols_walker,
(void *) colnos);
}
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index 3cf268c..e0dc8d9
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 268,274 ****
generate_partition_wise_join_paths(root, joinrel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
--- 268,274 ----
generate_partition_wise_join_paths(root, joinrel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 6872e14..1ccb834
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 488,494 ****
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! generate_gather_paths(root, rel);
/*
* Allow a plugin to editorialize on the set of Paths for this base
--- 488,497 ----
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! {
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
! }
/*
* Allow a plugin to editorialize on the set of Paths for this base
*************** static void
*** 689,694 ****
--- 692,699 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
+ PartialAggInfo *agg_info = NULL;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 697,710 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0), false);
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
! /* Consider index scans */
create_index_paths(root, rel);
/* Consider TID scans */
--- 702,742 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! add_path(rel, seq_path, false);
!
! if (rel->gpi != NULL && rel->gpi->target != NULL &&
! required_outer == NULL)
! {
! /*
! * Retrieve the necessary info for the grouping expressions and
! * aggregates.
! *
! * All that create_agg_plan eventually needs of the clause is
! * tleSortGroupRef, so we don't have to care that the clause
! * expression might differ from texpr, in case texpr was derived from
! * EC.
! */
! agg_info = get_partial_agg_info(root, rel->gpi->target,
! rel->gpi->sortgroupclauses);
!
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input +
! * set to be sorted.
! */
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED,
! agg_info);
! }
!
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
! /*
! * Consider index scans.
! */
create_index_paths(root, rel);
/* Consider TID scans */
*************** static void
*** 719,724 ****
--- 751,757 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
parallel_workers = compute_parallel_worker(rel, rel->pages, -1);
*************** create_plain_partial_paths(PlannerInfo *
*** 727,734 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers),
! false);
}
/*
--- 760,887 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
! add_partial_path(rel, path, false);
!
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it. Only AGG_HASHED is suitable here as it does not expect
! * the input set to be sorted.
! */
! if (rel->gpi != NULL && rel->gpi->target != NULL)
! {
! PartialAggInfo *agg_info = get_partial_agg_info(root,
! rel->gpi->target,
! rel->gpi->sortgroupclauses);
!
! create_grouped_path(root, rel, path, false, true, AGG_HASHED,
! agg_info);
! }
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the
! * appropriate pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck().
! *
! * If "partial" is true, the resulting path is considered partial in terms of
! * parallel execution.
! *
! * The path we create here shouldn't be parameterized because of supposedly
! * high startup cost of aggregation (whether due to build of hash table for
! * AGG_HASHED strategy or due to explicit sort for AGG_SORTED).
! *
! * XXX IndexPath as an input for AGG_SORTED seems to be an exception ---
! * consider implementing parameterized AGG_SORTED unless the IndexPath is
! * partial.
! */
! void
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy,
! PartialAggInfo *agg_info)
! {
! Path *agg_path;
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Grouped path should never be parameterized, so we're not supposed to
! * receive parameterized subpath.
! */
! Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
!
! /*
! * Non-var grouping expressions will eventually require projection using
! * Result plan, but that does not work with SRFs.
! */
! Assert(rel->gpi != NULL);
! if (rel->gpi->group_exprs != NULL)
! {
! ListCell *lc;
!
! foreach(lc, rel->gpi->group_exprs->exprs)
! {
! GroupedVar *gvar = lfirst_node(GroupedVar, lc);
!
! if (IsA(gvar->gvexpr, FuncExpr))
! {
! FuncExpr *fexpr = (FuncExpr *) gvar->gvexpr;
!
! if (fexpr->funcretset)
! return;
! }
! }
! }
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! agg_info,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! agg_info,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL,
! true))
! return;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost, pathkeys,
! true))
! return;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path, true);
! else
! add_partial_path(rel, (Path *) agg_path, true);
! }
}
/*
*************** set_append_rel_size(PlannerInfo *root, R
*** 1138,1143 ****
--- 1291,1327 ----
adjust_appendrel_attrs(root,
(Node *) rel->joininfo,
1, &appinfo);
+ childrel->reltarget->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) rel->reltarget->exprs,
+ 1, &appinfo);
+
+ /*
+ * Setup GroupedPathInfo for the child relation if the parent has
+ * some.
+ */
+ if (rel->gpi != NULL)
+ build_child_rel_gpi(root, childrel, rel, 1, &appinfo);
+
+ /*
+ * We have to make child entries in the EquivalenceClass data
+ * structures as well. This is needed either if the parent
+ * participates in some eclass joins (because we will want to consider
+ * inner-indexscan joins on the individual children) or if the parent
+ * has useful pathkeys (because we should try to build MergeAppend
+ * paths that produce those sort orderings).
+ */
+ if (rel->has_eclass_joins || has_useful_pathkeys(root, rel))
+ add_child_rel_equivalences(root, appinfo, rel, childrel);
+ childrel->has_eclass_joins = rel->has_eclass_joins;
+
+ /*
+ * Note: we could compute appropriate attr_needed data for the child's
+ * variables, by transforming the parent's attr_needed through the
+ * translated_vars mapping. However, currently there's no need
+ * because attr_needed is only examined for base relations not
+ * otherrels. So we just leave the child's attr_needed empty.
+ */
/*
* If parallelism is allowable for this query in general, see whether
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1336,1341 ****
--- 1520,1527 ----
List *pa_partial_subpaths = NIL;
List *pa_nonpartial_subpaths = NIL;
bool partial_subpaths_valid = true;
+ List *grouped_subpaths = NIL;
+ bool grouped_subpaths_valid = true;
bool pa_subpaths_valid = enable_parallel_append;
List *all_child_pathkeys = NIL;
List *all_child_outers = NIL;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1479,1484 ****
--- 1665,1691 ----
}
/*
+ * For grouped paths, use only the unparameterized subpaths.
+ *
+ * XXX Consider if the parameterized subpaths should be processed
+ * below. It's probably not useful for sequential scans (due to
+ * repeated aggregation), but might be worthwhile for other child
+ * nodes.
+ */
+ if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
+ {
+ Path *path;
+
+ path = (Path *) linitial(childrel->gpi->pathlist);
+ if (path->param_info == NULL)
+ accumulate_append_subpath(path, &grouped_subpaths, NULL);
+ else
+ grouped_subpaths_valid = false;
+ }
+ else
+ grouped_subpaths_valid = false;
+
+ /*
* Collect lists of all the available path orderings and
* parameterizations for all the children. We use these as a
* heuristic to indicate which sort orderings and parameterizations we
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1648,1653 ****
--- 1855,1871 ----
add_partial_path(rel, (Path *) appendpath, false);
}
+ if (grouped_subpaths_valid)
+ {
+ Path *path;
+
+ path = (Path *) create_append_path(rel, grouped_subpaths, NIL, NULL, 0,
+ false, partitioned_rels, -1);
+ /* pathtarget will produce the grouped relation.. */
+ path->pathtarget = rel->gpi->target;
+ add_path(rel, path, true);
+ }
+
/*
* Also build unparameterized MergeAppend paths based on the collected
* list of child pathkeys.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1699,1705 ****
add_path(rel, (Path *)
create_append_path(rel, subpaths, NIL,
required_outer, 0, false,
! partitioned_rels, -1), false);
}
}
--- 1917,1924 ----
add_path(rel, (Path *)
create_append_path(rel, subpaths, NIL,
required_outer, 0, false,
! partitioned_rels, -1),
! false);
}
}
*************** set_subquery_pathlist(PlannerInfo *root,
*** 2180,2187 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer),
! false);
}
}
--- 2399,2405 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer), false);
}
}
*************** set_worktable_pathlist(PlannerInfo *root
*** 2452,2465 ****
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
ListCell *lc;
/* If there are no partial paths, there's nothing to do here. */
! if (rel->partial_pathlist == NIL)
return;
/*
--- 2670,2690 ----
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
+ List *pathlist = NIL;
+ PathTarget *partial_target;
ListCell *lc;
+ if (!grouped)
+ pathlist = rel->partial_pathlist;
+ else if (rel->gpi != NULL)
+ pathlist = rel->gpi->partial_pathlist;
+
/* If there are no partial paths, there's nothing to do here. */
! if (pathlist == NIL)
return;
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2467,2483 ****
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(rel->partial_pathlist);
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path, false);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach(lc, rel->partial_pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
--- 2692,2714 ----
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(pathlist);
!
! if (!grouped)
! partial_target = rel->reltarget;
! else if (rel->gpi != NULL && rel->gpi->target != NULL)
! partial_target = rel->gpi->target;
!
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, partial_target,
NULL, NULL);
! add_path(rel, simple_gather_path, grouped);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach(lc, pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
*************** generate_gather_paths(PlannerInfo *root,
*** 2485,2493 ****
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, false);
}
}
--- 2716,2724 ----
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, partial_target,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, grouped);
}
}
*************** standard_join_search(PlannerInfo *root,
*** 2659,2665 ****
generate_partition_wise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
--- 2890,2897 ----
generate_partition_wise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index b0c7a0c..9f489c3
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
*************** bool enable_gathermerge = true;
*** 131,136 ****
--- 131,137 ----
bool enable_partition_wise_join = false;
bool enable_parallel_append = true;
bool enable_parallel_hash = true;
+ bool enable_agg_pushdown = false;
typedef struct
{
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index faa654b..79de6cf
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
*************** add_grouping_info_to_base_rels(PlannerIn
*** 259,264 ****
--- 259,270 ----
int i;
ListCell *lc;
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
/* No grouping in the query? */
if (!root->parse->groupClause)
return;
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 351fb93..73bccee
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,230 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual),
! false);
}
/*
--- 223,229 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual), false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index c1939a1..2bf4c8b
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,90 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,89 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals), false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 6371c34..94b1f8e
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** inheritance_planner(PlannerInfo *root)
*** 1529,1536 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)),
! false);
}
/*--------------------
--- 1529,1535 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)), false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 1933,1939 ****
newpath = (Path *) create_projection_path(root,
current_rel,
subpath,
! scanjoin_target);
lfirst(lc) = newpath;
}
}
--- 1932,1939 ----
newpath = (Path *) create_projection_path(root,
current_rel,
subpath,
! scanjoin_target,
! false);
lfirst(lc) = newpath;
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3581,3588 ****
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *grouped_rel;
PathTarget *partial_grouping_target = NULL;
! AggClauseCosts agg_partial_costs; /* parallel only */
! AggClauseCosts agg_final_costs; /* parallel only */
Size hashaggtablesize;
double dNumGroups;
double dNumPartialGroups = 0;
--- 3581,3588 ----
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *grouped_rel;
PathTarget *partial_grouping_target = NULL;
! AggClauseCosts agg_final_costs;
! bool agg_final_costs_known = false;
Size hashaggtablesize;
double dNumGroups;
double dNumPartialGroups = 0;
*************** create_grouping_paths(PlannerInfo *root,
*** 3767,3772 ****
--- 3767,3779 ----
}
/*
+ * Collect statistics about aggregates for estimating costs of performing
+ * aggregation in parallel.
+ */
+ MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
+ agg_final_costs_known = false;
+
+ /*
* Before generating paths for grouped_rel, we first generate any possible
* partial paths; that way, later code can easily consider both parallel
* and non-parallel approaches to grouping. Note that the partial paths
*************** create_grouping_paths(PlannerInfo *root,
*** 3777,3782 ****
--- 3784,3792 ----
if (try_parallel_aggregation)
{
Path *cheapest_partial_path = linitial(input_rel->partial_pathlist);
+ AggClauseCosts agg_partial_costs;
+
+ MemSet(&agg_partial_costs, 0, sizeof(AggClauseCosts));
/*
* Build target list for partial aggregate paths. These paths cannot
*************** create_grouping_paths(PlannerInfo *root,
*** 3792,3817 ****
cheapest_partial_path->rows,
gd);
- /*
- * Collect statistics about aggregates for estimating costs of
- * performing aggregation in parallel.
- */
- MemSet(&agg_partial_costs, 0, sizeof(AggClauseCosts));
- MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
if (parse->hasAggs)
{
/* partial phase */
get_agg_clause_costs(root, (Node *) partial_grouping_target->exprs,
AGGSPLIT_INITIAL_SERIAL,
&agg_partial_costs);
-
- /* final phase */
- get_agg_clause_costs(root, (Node *) target->exprs,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
- get_agg_clause_costs(root, parse->havingQual,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
}
if (can_sort)
--- 3802,3813 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3893,3899 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups), false);
}
}
}
--- 3889,3896 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3901,3917 ****
/* Build final grouping paths */
if (can_sort)
{
/*
* Use any available suitably-sorted path as input, and also consider
* sorting the cheapest-total path.
*/
! foreach(lc, input_rel->pathlist)
{
Path *path = (Path *) lfirst(lc);
bool is_sorted;
is_sorted = pathkeys_contained_in(root->group_pathkeys,
path->pathkeys);
if (path == cheapest_path || is_sorted)
{
/* Sort the cheapest-total path if it isn't already sorted */
--- 3898,3933 ----
/* Build final grouping paths */
if (can_sort)
{
+ List *pathlist = input_rel->pathlist;
+ int nplain = list_length(input_rel->pathlist);
+ int i;
+
+ /*
+ * The grouped paths created out of grouped base relations or joins
+ * can be treated almost identically here (the major difference is
+ * that their targetlists contain GroupedVars instead aggregate input
+ * vars, but this is handled by using the appropriate value of
+ * AggSplit), so process them in the same loop.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL)
+ pathlist = list_concat(list_copy(pathlist),
+ input_rel->gpi->pathlist);
+
/*
* Use any available suitably-sorted path as input, and also consider
* sorting the cheapest-total path.
*/
! i = 0;
! foreach(lc, pathlist)
{
Path *path = (Path *) lfirst(lc);
bool is_sorted;
+ bool is_grouped;
is_sorted = pathkeys_contained_in(root->group_pathkeys,
path->pathkeys);
+ is_grouped = i >= nplain;
+
if (path == cheapest_path || is_sorted)
{
/* Sort the cheapest-total path if it isn't already sorted */
*************** create_grouping_paths(PlannerInfo *root,
*** 3925,3936 ****
--- 3941,3979 ----
/* Now decide what to stick atop it */
if (parse->groupingSets)
{
+ /*
+ * No grouping should have taken place at base relation /
+ * join level, i.e. pathlist should be equal to
+ * input_rel->pathlist.
+ */
+ Assert(!is_grouped);
+
consider_groupingsets_paths(root, grouped_rel,
path, true, can_hash, target,
gd, agg_costs, dNumGroups);
}
else if (parse->hasAggs)
{
+ AggStrategy aggstrategy;
+ AggSplit aggsplit;
+
+ if (!is_grouped)
+ {
+ aggstrategy = parse->groupClause ? AGG_SORTED : AGG_PLAIN;
+ aggsplit = AGGSPLIT_SIMPLE;
+ }
+ else
+ {
+ /*
+ * Like above, no grouping of base relation is not
+ * possible w/o this.
+ */
+ Assert(parse->groupClause);
+
+ aggstrategy = AGG_SORTED;
+ aggsplit = AGGSPLIT_FINAL_DESERIAL;
+ }
+
/*
* We have aggregation, possibly with plain GROUP BY. Make
* an AggPath.
*************** create_grouping_paths(PlannerInfo *root,
*** 3940,3947 ****
grouped_rel,
path,
target,
! parse->groupClause ? AGG_SORTED : AGG_PLAIN,
! AGGSPLIT_SIMPLE,
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
--- 3983,3990 ----
grouped_rel,
path,
target,
! aggstrategy,
! aggsplit,
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
*************** create_grouping_paths(PlannerInfo *root,
*** 3968,3978 ****
Assert(false);
}
}
}
/*
! * Now generate a complete GroupAgg Path atop of the cheapest partial
! * path. We can do this using either Gather or Gather Merge.
*/
if (grouped_rel->partial_pathlist)
sort_agg_partial_grouped_paths(root,
--- 4011,4040 ----
Assert(false);
}
}
+
+ i++;
}
/*
! * Compute agg_final_costs iff the aggregation should take place in 2
! * steps.
! */
! if (parse->hasAggs &&
! (grouped_rel->partial_pathlist ||
! (input_rel->gpi != NULL && input_rel->gpi->partial_pathlist)))
! {
! get_agg_clause_costs(root, (Node *) target->exprs,
! AGGSPLIT_FINAL_DESERIAL,
! &agg_final_costs);
! get_agg_clause_costs(root, parse->havingQual,
! AGGSPLIT_FINAL_DESERIAL,
! &agg_final_costs);
!
! agg_final_costs_known = true;
! }
!
! /*
! * Gather grouped partial paths and apply AGG_SORTED to them.
*/
if (grouped_rel->partial_pathlist)
sort_agg_partial_grouped_paths(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3980,3985 ****
--- 4042,4060 ----
&agg_final_costs,
dNumGroups, grouped_rel,
partial_grouping_target, target);
+
+ /*
+ * The same for the grouped partial paths involving base relation /
+ * join grouping.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->target != NULL &&
+ input_rel->gpi->partial_pathlist)
+ sort_agg_partial_grouped_paths(root,
+ input_rel->gpi->partial_pathlist,
+ &agg_final_costs, dNumGroups,
+ grouped_rel,
+ input_rel->gpi->target,
+ target);
}
if (can_hash)
*************** create_grouping_paths(PlannerInfo *root,
*** 4026,4031 ****
--- 4101,4122 ----
}
/*
+ * Compute agg_final_costs if needed below and if not done above.
+ */
+ if (parse->hasAggs && !agg_final_costs_known &&
+ (grouped_rel->partial_pathlist ||
+ (input_rel->gpi != NULL &&
+ (input_rel->gpi->pathlist || input_rel->gpi->partial_pathlist))))
+ {
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ }
+
+ /*
* Generate a HashAgg Path atop of the cheapest partial path. Once
* again, we'll only do this if it looks as though the hash table
* won't exceed work_mem.
*************** create_grouping_paths(PlannerInfo *root,
*** 4038,4043 ****
--- 4129,4181 ----
dNumGroups, grouped_rel,
partial_grouping_target, target);
}
+
+ /*
+ * If input_rel has partially aggregated paths, perform the final
+ * aggregation.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->pathlist);
+
+ hashaggtablesize = estimate_hashagg_tablesize(path,
+ &agg_final_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * The top-level grouped_rel needs to receive the path into
+ * regular pathlist, as opposed grouped_rel->gpi->pathlist. So
+ * pass FALSE for grouped.
+ */
+ add_path(grouped_rel,
+ (Path *) create_agg_path(root, grouped_rel,
+ path,
+ target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ &agg_final_costs,
+ dNumGroups),
+ false);
+ }
+ }
+
+ /*
+ * If input_rel has partially aggregated partial paths, gather them
+ * and perform the final aggregation.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->target != NULL &&
+ input_rel->gpi->partial_pathlist != NIL)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->partial_pathlist);
+
+ hash_agg_partial_grouped_path(root, path, &agg_final_costs, dNumGroups,
+ grouped_rel, input_rel->gpi->target,
+ target);
+ }
}
/* Give a helpful error if we failed to find any implementation */
*************** create_distinct_paths(PlannerInfo *root,
*** 4870,4876 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
--- 5008,5015 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows),
! false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4897,4903 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
--- 5036,5043 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows),
! false);
}
/*
*************** adjust_paths_for_srfs(PlannerInfo *root,
*** 6011,6017 ****
newpath = (Path *) create_projection_path(root,
rel,
newpath,
! thistarget);
}
}
lfirst(lc) = newpath;
--- 6151,6158 ----
newpath = (Path *) create_projection_path(root,
rel,
newpath,
! thistarget,
! false);
}
}
lfirst(lc) = newpath;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index b5c4124..b5be81c
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
bool has_conv_whole_rows; /* are there ConvertRowtypeExpr
* entries encapsulating a whole-row
*************** set_upper_references(PlannerInfo *root,
*** 1739,1747 ****
--- 1740,1802 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the current
+ * subplan->targetlist.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ replace_grouped_vars_with_aggrefs(root, subplan->targetlist);
+ }
+ else if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /*
+ * Partial aggregation node can have GroupedVar's on the input
+ * if those represent generic (non-Var) grouping expressions.
+ * Unlike above, the restored expressions should stay there.
+ */
+ subplan->targetlist =
+ replace_grouped_vars_with_aggrefs(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1996,2001 ****
--- 2051,2057 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
itlist->has_conv_whole_rows = false;
*************** build_tlist_index(List *tlist)
*** 2016,2021 ****
--- 2072,2079 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else if (is_converted_whole_row_reference((Node *) tle->expr))
itlist->has_conv_whole_rows = true;
else
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2299,2304 ****
--- 2357,2387 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2461,2467 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
--- 2544,2551 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index a1bea53..0aaac38
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 26,31 ****
--- 26,32 ----
#include "optimizer/planmain.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if create_grouped_path ends up in another module. */
#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
*************** static int append_startup_cost_compare(c
*** 56,62 ****
static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
!
/*****************************************************************************
* MISC. PATH UTILITIES
--- 57,65 ----
static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
! static Path *add_grouping_expressions_to_subpath(PlannerInfo *root,
! Path *subpath,
! PathTarget *group_exprs);
/*****************************************************************************
* MISC. PATH UTILITIES
*************** add_partial_path_precheck(RelOptInfo *pa
*** 973,979 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, false))
return false;
return true;
--- 976,982 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, grouped))
return false;
return true;
*************** create_unique_path(PlannerInfo *root, Re
*** 1543,1550 ****
MemoryContext oldcontext;
int numCols;
! /* Caller made a mistake if subpath isn't cheapest_total ... */
! Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
Assert(sjinfo->jointype == JOIN_SEMI);
--- 1546,1557 ----
MemoryContext oldcontext;
int numCols;
! /*
! * Caller made a mistake if subpath isn't cheapest_total (or the cheapest
! * grouped).
! */
! Assert(subpath == rel->cheapest_total_path ||
! (rel->gpi != NULL && subpath == linitial(rel->gpi->pathlist)));
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
Assert(sjinfo->jointype == JOIN_SEMI);
*************** create_hashjoin_path(PlannerInfo *root,
*** 2394,2405 ****
* 'rel' is the parent relation associated with the result
* 'subpath' is the path representing the source of data
* 'target' is the PathTarget to be computed
*/
ProjectionPath *
create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target)
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget = subpath->pathtarget;
--- 2401,2414 ----
* 'rel' is the parent relation associated with the result
* 'subpath' is the path representing the source of data
* 'target' is the PathTarget to be computed
+ * 'force_result' enforces implementation using Result plan.
*/
ProjectionPath *
create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target,
! bool force_result)
{
ProjectionPath *pathnode = makeNode(ProjectionPath);
PathTarget *oldtarget = subpath->pathtarget;
*************** create_projection_path(PlannerInfo *root
*** 2418,2423 ****
--- 2427,2433 ----
pathnode->path.pathkeys = subpath->pathkeys;
pathnode->subpath = subpath;
+ pathnode->force_result = force_result;
/*
* We might not need a separate Result node. If the input plan node type
*************** create_projection_path(PlannerInfo *root
*** 2428,2435 ****
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs))
{
/* No separate Result node needed */
pathnode->dummypp = true;
--- 2438,2446 ----
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (!force_result &&
! (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
*************** apply_projection_to_path(PlannerInfo *ro
*** 2499,2505 ****
* separate ProjectionPath.
*/
if (!is_projection_capable_path(path))
! return (Path *) create_projection_path(root, rel, path, target);
/*
* We can just jam the desired tlist into the existing path, being sure to
--- 2510,2517 ----
* separate ProjectionPath.
*/
if (!is_projection_capable_path(path))
! return (Path *) create_projection_path(root, rel, path, target,
! false);
/*
* We can just jam the desired tlist into the existing path, being sure to
*************** apply_projection_to_path(PlannerInfo *ro
*** 2539,2545 ****
create_projection_path(root,
gpath->subpath->parent,
gpath->subpath,
! target);
}
else
{
--- 2551,2558 ----
create_projection_path(root,
gpath->subpath->parent,
gpath->subpath,
! target,
! false);
}
else
{
*************** apply_projection_to_path(PlannerInfo *ro
*** 2549,2555 ****
create_projection_path(root,
gmpath->subpath->parent,
gmpath->subpath,
! target);
}
}
else if (path->parallel_safe &&
--- 2562,2569 ----
create_projection_path(root,
gmpath->subpath->parent,
gmpath->subpath,
! target,
! false);
}
}
else if (path->parallel_safe &&
*************** create_agg_path(PlannerInfo *root,
*** 2848,2853 ****
--- 2862,3123 ----
}
/*
+ * get_partial_agg_data
+ *
+ * Given a "grouped target" (i.e. target where each non-GroupedVar
+ * element must have sortgroupref set), build a list of the referencing
+ * SortGroupClauses, a list of the corresponding grouping expressions and
+ * a list of aggregate expressions.
+ *
+ * target_group_clauses can contain a list of target-specific
+ * SortGroupClause's, which correspond to grouping expressions possibly
+ * added to the target by initialize_grouped_target. These are not
+ * necessarily present in root->query->groupClause.
+ */
+ PartialAggInfo *
+ get_partial_agg_info(PlannerInfo *root, PathTarget *target,
+ List *target_group_clauses)
+ {
+ ListCell *l;
+ int i = 0;
+ PartialAggInfo *result;
+
+ /* The target should contain at least one grouping column. */
+ Assert(target->sortgrouprefs != NULL);
+
+ result = palloc0(sizeof(PartialAggInfo));
+
+ foreach(l, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(l);
+
+ if (IsA(texpr, GroupedVar) &&
+ IsA(((GroupedVar *) texpr)->gvexpr, Aggref))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ cl = get_sortgroupref_clause_noerr(sortgroupref, root->parse->groupClause);
+
+ /*
+ * If query does not have this clause, it must be target-specific.
+ */
+ if (cl == NULL)
+ cl = get_sortgroupref_clause(sortgroupref, target_group_clauses);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /* Now collect the aggregates. */
+ while (l != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(l));
+
+ /* Currently, GroupedVarInfo can only represent aggregate. */
+ Assert(gvar->agg_partial != NULL);
+ result->agg_exprs = lappend(result->agg_exprs, gvar->agg_partial);
+ l = lnext(l);
+ }
+
+ return result;
+ }
+
+ /*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * check_pathkeys can be passed FALSE if the function was already called for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * agg_info contains both aggregate and grouping expressions.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool check_pathkeys,
+ PartialAggInfo *agg_info,
+ double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+ Assert(rel->gpi->target != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Add generic grouping expressions to the subpath if there are some, and
+ * adjust the costs accordingly.
+ */
+ if (rel->gpi->group_exprs != NULL)
+ subpath = add_grouping_expressions_to_subpath(root, subpath,
+ rel->gpi->group_exprs);
+
+ if (check_pathkeys)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(agg_info->agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) agg_info->agg_exprs,
+ AGGSPLIT_INITIAL_SERIAL, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ Assert(agg_info->group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, rel->gpi->target,
+ AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
+ agg_info->group_clauses, NIL, &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Apply partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ PartialAggInfo *agg_info, double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+ Assert(rel->gpi->target != NULL);
+
+ /*
+ * Add generic grouping expressions to the subpath if there are some, and
+ * adjust the costs accordingly.
+ */
+ if (rel->gpi->group_exprs != NULL)
+ subpath = add_grouping_expressions_to_subpath(root, subpath,
+ rel->gpi->group_exprs);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(agg_info->agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) agg_info->agg_exprs,
+ AGGSPLIT_INITIAL_SERIAL, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(agg_info->group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ rel->gpi->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ agg_info->group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
*************** reparameterize_pathlist_by_child(Planner
*** 3936,3938 ****
--- 4206,4261 ----
return result;
}
+
+ /*
+ * Add (non-Var) grouping expressions contained in group_exprs target to a
+ * subpath. The passed in subpath should not be touched so that it can still
+ * be used as a subpath for non-grouped paths. Therefore we wrap it into
+ * ResultPath, to which we actually add those expressions.
+ *
+ * Costs of the grouping expression evaluation is added to the ResultPath
+ * because the evaluation will take place here, rather than at the AggPath
+ * we're preparing data for.
+ */
+ static Path *
+ add_grouping_expressions_to_subpath(PlannerInfo *root, Path *subpath,
+ PathTarget *group_exprs)
+ {
+ PathTarget *target;
+ ListCell *lc;
+ int i;
+
+ /*
+ * The subpath must stay intact because, besides producing input data for
+ * aggregation, it can participate in a plan that does not use aggregation
+ * push-down. For the same reason we pass force_result=true to
+ * create_projection_path below.
+ */
+ target = copy_pathtarget(subpath->pathtarget);
+
+ Assert(group_exprs->sortgrouprefs != NULL);
+ i = 0;
+ foreach(lc, group_exprs->exprs)
+ {
+ Index sortgroupref;
+ GroupedVar *gvar;
+ QualCost cost;
+
+ sortgroupref = group_exprs->sortgrouprefs[i++];
+ gvar = lfirst_node(GroupedVar, lc);
+ add_column_to_pathtarget(target, (Expr *) gvar, sortgroupref);
+
+ /*
+ * Account for the cost to evaluate the grouping expressions.
+ */
+ cost_qual_eval_node(&cost, (Node *) gvar->gvexpr, root);
+ target->cost.startup += cost.startup;
+ target->cost.per_tuple += cost.per_tuple;
+ }
+
+ return (Path *) create_projection_path(root,
+ subpath->parent,
+ subpath,
+ target,
+ true);
+ }
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 08686f6..e4c2313
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 27,32 ****
--- 27,33 ----
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
#include "parser/parse_oper.h"
#include "utils/hsearch.h"
*************** build_simple_rel(PlannerInfo *root, int
*** 126,131 ****
--- 127,133 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->gpi = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** build_child_join_rel(PlannerInfo *root,
*** 797,802 ****
--- 799,857 ----
}
/*
+ * Initialize GroupedPathInfo of a child relation according to that of the
+ * parent.
+ */
+ void
+ build_child_rel_gpi(PlannerInfo *root, RelOptInfo *child, RelOptInfo *parent,
+ int nappinfos, AppendRelInfo **appinfos)
+ {
+ GroupedPathInfo *gpi,
+ *gpi_parent;
+
+ Assert(child->gpi == NULL);
+
+ gpi_parent = parent->gpi;
+ child->gpi = gpi = makeNode(GroupedPathInfo);
+
+ /*
+ * Create grouping target translated to the child varnos.
+ *
+ * We need a copy of the target so that the translation does not affect
+ * the parent.
+ */
+ gpi->target = copy_pathtarget(gpi_parent->target);
+ gpi->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->target->exprs,
+ nappinfos, appinfos);
+
+ /*
+ * Non-var grouping expressions need to be translated as well.
+ */
+ if (gpi_parent->group_exprs != NULL)
+ {
+ gpi->group_exprs = copy_pathtarget(gpi_parent->group_exprs);
+ gpi->group_exprs->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) gpi->group_exprs->exprs,
+ nappinfos, appinfos);
+ }
+
+ /*
+ * sortgroupclauses need no kind of translation, so just copy the pointer.
+ */
+ gpi->sortgroupclauses = gpi_parent->sortgroupclauses;
+
+ /*
+ * If the output of the child rel's paths should be aggregated,
+ * sortgrouprefs are also needed. (initialize_grouped_targets should have
+ * initialized sortgrouprefs of the parent rel.)
+ */
+ child->reltarget->sortgrouprefs = parent->reltarget->sortgrouprefs;
+ }
+
+ /*
* min_join_parameterization
*
* Determine the minimum possible parameterization of a joinrel, that is, the
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index abdc5d9..b9c2d13
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,414 ****
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
--- 408,413 ----
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 782,841 ----
}
/*
+ * Replace each GroupedVar in the source targetlist with the original
+ * expression --- either Aggref or a non-Var grouping expression.
+ *
+ * Even if the query targetlist has the Aggref wrapped in a generic
+ * expression, any subplan should emit the corresponding GroupedVar
+ * alone. (Aggregate finalization is needed before the aggregate result can be
+ * used for any purposes and that happens at the top level of the query.)
+ * Therefore we do not have to recurse into the target expressions here.
+ */
+ List *
+ replace_grouped_vars_with_aggrefs(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te,
+ *te_new;
+ Expr *expr_new = NULL;
+
+ te = lfirst_node(TargetEntry, l);
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Partial aggregate should appear in the targetlist so that
+ * it looks as if convert_combining_aggrefs arranged it.
+ */
+ expr_new = (Expr *) gvar->agg_partial;
+ }
+ else
+ expr_new = gvar->gvexpr;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
* For each aggregate add GroupedVar to the grouped target.
*
* Caller passes the aggregates in the form of GroupedVarInfos so that we
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index 8514c21..ad8a0b9
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7647,7652 ****
--- 7647,7669 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+ Expr *expr = gvar->gvexpr;
+
+ if (IsA(expr, Aggref))
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ else if (IsA(expr, Var))
+ (void) get_variable((Var *) expr, 0, false, context);
+ else
+ {
+ Assert(IsA(gvar->gvexpr, OpExpr));
+ get_oper_expr((OpExpr *) expr, context);
+ }
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 9132,9141 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 9149,9166 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index e32901d..90703c6
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_bool ConfigureNames
*** 921,926 ****
--- 921,935 ----
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index 1c93688..00e73ff
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct HashPath
*** 1537,1548 ****
--- 1537,1552 ----
* ProjectionPath node, which is marked dummy to indicate that we intend to
* assign the work to the input plan node. The estimated cost for the
* ProjectionPath node will account for whether a Result will be used or not.
+ *
+ * force_result field tells that the Result node must be used for some reason
+ * even though the subpath could normally handle the projection.
*/
typedef struct ProjectionPath
{
Path path;
Path *subpath; /* path representing input source */
bool dummypp; /* true if no separate Result is needed */
+ bool force_result; /* Is Result node required? */
} ProjectionPath;
/*
*************** typedef struct AggPath
*** 1618,1623 ****
--- 1622,1638 ----
} AggPath;
/*
+ * Information needed to create grouped paths out of base relation scan or
+ * join.
+ */
+ typedef struct PartialAggInfo
+ {
+ List *group_clauses;
+ List *group_exprs;
+ List *agg_exprs;
+ } PartialAggInfo;
+
+ /*
* Various annotations used for grouping sets in the planner.
*/
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
new file mode 100644
index 27afc2e..12d8134
*** a/src/include/optimizer/cost.h
--- b/src/include/optimizer/cost.h
*************** extern bool enable_mergejoin;
*** 68,73 ****
--- 68,74 ----
extern bool enable_hashjoin;
extern bool enable_gathermerge;
extern bool enable_partition_wise_join;
+ extern bool enable_agg_pushdown;
extern bool enable_parallel_append;
extern bool enable_parallel_hash;
extern int constraint_exclusion;
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 9c05492..b11bd47
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern HashPath *create_hashjoin_path(Pl
*** 163,169 ****
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target);
extern Path *apply_projection_to_path(PlannerInfo *root,
RelOptInfo *rel,
Path *path,
--- 163,170 ----
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
! PathTarget *target,
! bool force_result);
extern Path *apply_projection_to_path(PlannerInfo *root,
RelOptInfo *rel,
Path *path,
*************** extern AggPath *create_agg_path(PlannerI
*** 199,204 ****
--- 200,217 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern PartialAggInfo *get_partial_agg_info(PlannerInfo *root,
+ PathTarget *target,
+ List *target_group_clauses);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool check_pathkeys,
+ PartialAggInfo *agg_info,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ PartialAggInfo *agg_info,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern RelOptInfo *build_child_join_rel(
*** 303,306 ****
--- 316,322 ----
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
+ extern void build_child_rel_gpi(PlannerInfo *root, RelOptInfo *child,
+ RelOptInfo *parent, int nappinfos,
+ AppendRelInfo **appinfos);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index ea886b6..7815138
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void set_dummy_rel_pathlist(RelOp
*** 53,59 ****
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
--- 53,65 ----
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
!
! extern void create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
! Path *subpath, bool precheck, bool partial,
! AggStrategy aggstrategy,
! PartialAggInfo *agg_info);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index a725818..dfc9352
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
***************
*** 16,22 ****
#include "nodes/relation.h"
-
extern TargetEntry *tlist_member(Expr *node, List *targetlist);
extern TargetEntry *tlist_member_ignore_relabel(Expr *node, List *targetlist);
--- 16,21 ----
*************** extern Node *get_sortgroupclause_expr(So
*** 41,51 ****
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
- extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
- List *target_group_clauses,
- List **grouping_clauses,
- List **grouping_exprs, List **agg_exprs);
-
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
extern SortGroupClause *get_sortgroupref_clause_noerr(Index sortref,
--- 40,45 ----
*************** extern void split_pathtarget_at_srfs(Pla
*** 69,74 ****
--- 63,71 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *replace_grouped_vars_with_aggrefs(PlannerInfo *root, List *src);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
new file mode 100644
index c9c8f51..e8ebafe
*** a/src/test/regress/expected/sysviews.out
--- b/src/test/regress/expected/sysviews.out
*************** select count(*) >= 0 as ok from pg_prepa
*** 72,77 ****
--- 72,78 ----
select name, setting from pg_settings where name like 'enable%';
name | setting
----------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
*************** select name, setting from pg_settings wh
*** 87,93 ****
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (15 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
--- 88,94 ----
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (16 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
agg_pushdown_v5/06_join_prepare.diff
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 2a931af..ab739db
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** set_join_pathlist_hook_type set_join_pat
*** 39,57 ****
#define PATH_PARAM_BY_REL(path, rel) \
(PATH_PARAM_BY_REL_SELF(path, rel) || PATH_PARAM_BY_PARENT(path, rel))
static void try_partial_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
Path *outer_path,
Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 39,72 ----
#define PATH_PARAM_BY_REL(path, rel) \
(PATH_PARAM_BY_REL_SELF(path, rel) || PATH_PARAM_BY_PARENT(path, rel))
+ /*
+ * The sort-specific input for try_mergejoin_... functions.
+ */
+ typedef struct MergeJoinSortInfo
+ {
+ List *outerkeys;
+ List *innerkeys;
+ List *merge_pathkeys;
+ List *merge_clauses;
+ } MergeJoinSortInfo;
+
static void try_partial_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
Path *outer_path,
Path *inner_path,
! MergeJoinSortInfo *si,
JoinType jointype,
JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
+ static void sort_inner_and_outer_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ List *sort_infos);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** try_partial_nestloop_path(PlannerInfo *r
*** 547,552 ****
--- 562,680 ----
NULL), false);
}
+ static void
+ try_nestloop_path_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ Path *outer_path,
+ Path *inner_path,
+ List *pathkeys,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ bool partial)
+ {
+
+ /* Only join plain paths. */
+ if (!partial)
+ try_nestloop_path(root,
+ joinrel,
+ outer_path,
+ inner_path,
+ pathkeys,
+ jointype,
+ extra);
+ else
+ try_partial_nestloop_path(root,
+ joinrel,
+ outer_path,
+ inner_path,
+ pathkeys,
+ jointype,
+ extra);
+ }
+
+
+ /*
+ * Create one MergeJoinSortInfo for each possible use of merge clauses.
+ */
+ static List *
+ get_merge_join_sort_info(PlannerInfo *root, JoinPathExtraData *extra,
+ RelOptInfo *joinrel, JoinType jointype)
+ {
+ List *all_pathkeys;
+ ListCell *l;
+ List *result = NIL;
+
+ /*
+ * Each possible ordering of the available mergejoin clauses will generate
+ * a differently-sorted result path at essentially the same cost. We have
+ * no basis for choosing one over another at this level of joining, but
+ * some sort orders may be more useful than others for higher-level
+ * mergejoins, so it's worth considering multiple orderings.
+ *
+ * Actually, it's not quite true that every mergeclause ordering will
+ * generate a different path order, because some of the clauses may be
+ * partially redundant (refer to the same EquivalenceClasses). Therefore,
+ * what we do is convert the mergeclause list to a list of canonical
+ * pathkeys, and then consider different orderings of the pathkeys.
+ *
+ * Generating a path for *every* permutation of the pathkeys doesn't seem
+ * like a winning strategy; the cost in planning time is too high. For
+ * now, we generate one path for each pathkey, listing that pathkey first
+ * and the rest in random order. This should allow at least a one-clause
+ * mergejoin without re-sorting against any other possible mergejoin
+ * partner path. But if we've not guessed the right ordering of secondary
+ * keys, we may end up evaluating clauses as qpquals when they could have
+ * been done as mergeclauses. (In practice, it's rare that there's more
+ * than two or three mergeclauses, so expending a huge amount of thought
+ * on that is probably not worth it.)
+ *
+ * The pathkey order returned by select_outer_pathkeys_for_merge() has
+ * some heuristics behind it (see that function), so be sure to try it
+ * exactly as-is as well as making variants.
+ */
+ all_pathkeys = select_outer_pathkeys_for_merge(root,
+ extra->mergeclause_list,
+ joinrel);
+
+ foreach(l, all_pathkeys)
+ {
+ List *front_pathkey = (List *) lfirst(l);
+ MergeJoinSortInfo *si;
+
+ si = (MergeJoinSortInfo *) palloc(sizeof(MergeJoinSortInfo));
+
+ /* Make a pathkey list with this guy first */
+ if (l != list_head(all_pathkeys))
+ si->outerkeys = lcons(front_pathkey,
+ list_delete_ptr(list_copy(all_pathkeys),
+ front_pathkey));
+ else
+ si->outerkeys = all_pathkeys; /* no work at first one... */
+
+ /* Sort the mergeclauses into the corresponding ordering */
+ si->merge_clauses = find_mergeclauses_for_pathkeys(root,
+ si->outerkeys,
+ true,
+ extra->mergeclause_list);
+
+ /* Should have used them all... */
+ Assert(list_length(si->merge_clauses) == list_length(extra->mergeclause_list));
+
+ /* Build sort pathkeys for the inner side */
+ si->innerkeys = make_inner_pathkeys_for_merge(root,
+ si->merge_clauses,
+ si->outerkeys);
+
+ /* Build pathkeys representing output sort order */
+ si->merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
+ si->outerkeys);
+
+ result = lappend(result, si);
+ }
+
+ return result;
+ }
+
/*
* try_mergejoin_path
* Consider a merge join path; if it appears useful, push it into
*************** try_mergejoin_path(PlannerInfo *root,
*** 557,587 ****
RelOptInfo *joinrel,
Path *outer_path,
Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
!
! if (is_partial)
! {
! try_partial_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra);
! return;
! }
/*
* Check to see if proposed path is still parameterized, and reject if the
--- 685,698 ----
RelOptInfo *joinrel,
Path *outer_path,
Path *inner_path,
! MergeJoinSortInfo *si,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
! List *outersortkeys = si->outerkeys;
! List *innersortkeys = si->innerkeys;
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_mergejoin_path(PlannerInfo *root,
*** 611,624 ****
/*
* See comments in try_nestloop_path().
*/
! initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
--- 722,734 ----
/*
* See comments in try_nestloop_path().
*/
! initial_cost_mergejoin(root, &workspace, jointype, si->merge_clauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! si->merge_pathkeys, required_outer, false))
{
add_path(joinrel, (Path *)
create_mergejoin_path(root,
*************** try_mergejoin_path(PlannerInfo *root,
*** 629,637 ****
outer_path,
inner_path,
extra->restrictlist,
! pathkeys,
required_outer,
! mergeclauses,
outersortkeys,
innersortkeys), false);
}
--- 739,747 ----
outer_path,
inner_path,
extra->restrictlist,
! si->merge_pathkeys,
required_outer,
! si->merge_clauses,
outersortkeys,
innersortkeys), false);
}
*************** try_partial_mergejoin_path(PlannerInfo *
*** 652,665 ****
RelOptInfo *joinrel,
Path *outer_path,
Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 762,774 ----
RelOptInfo *joinrel,
Path *outer_path,
Path *inner_path,
! MergeJoinSortInfo *si,
JoinType jointype,
JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
+ List *outersortkeys = si->outerkeys;
+ List *innersortkeys = si->innerkeys;
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 687,698 ****
/*
* See comments in try_partial_nestloop_path().
*/
! initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
false))
return;
--- 796,807 ----
/*
* See comments in try_partial_nestloop_path().
*/
! initial_cost_mergejoin(root, &workspace, jointype, si->merge_clauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost,
! si->merge_pathkeys,
false))
return;
*************** try_partial_mergejoin_path(PlannerInfo *
*** 706,719 ****
outer_path,
inner_path,
extra->restrictlist,
! pathkeys,
NULL,
! mergeclauses,
outersortkeys,
innersortkeys), false);
}
/*
* try_hashjoin_path
* Consider a hash join path; if it appears useful, push it into
* the joinrel's pathlist via add_path().
--- 815,861 ----
outer_path,
inner_path,
extra->restrictlist,
! si->merge_pathkeys,
NULL,
! si->merge_clauses,
outersortkeys,
innersortkeys), false);
}
/*
+ * Generic front-end to create combinations of full/partial and grouped/plain
+ * merge joins.
+ */
+ static void
+ try_mergejoin_path_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ Path *outer_path,
+ Path *inner_path,
+ MergeJoinSortInfo *si,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ bool partial)
+ {
+ /* Only join plain paths. */
+ if (!partial)
+ try_mergejoin_path(root,
+ joinrel,
+ outer_path,
+ inner_path,
+ si,
+ jointype,
+ extra);
+ else
+ try_partial_mergejoin_path(root,
+ joinrel,
+ outer_path,
+ inner_path,
+ si,
+ jointype,
+ extra);
+ }
+
+ /*
* try_hashjoin_path
* Consider a hash join path; if it appears useful, push it into
* the joinrel's pathlist via add_path().
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 836,841 ****
--- 978,1021 ----
}
/*
+ * Generic front-end to create combinations of full/partial and grouped/plain
+ * hash joins.
+ */
+ static void
+ try_hashjoin_path_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ Path *outer_path,
+ Path *inner_path,
+ List *hashclauses,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ bool partial,
+ bool parallel_hash)
+ {
+ /* parallel_hash can only be set if partial is too. */
+ Assert(!parallel_hash || partial);
+
+ /* Only join plain paths. */
+ if (!partial)
+ try_hashjoin_path(root,
+ joinrel,
+ outer_path,
+ inner_path,
+ hashclauses,
+ jointype,
+ extra);
+ else
+ try_partial_hashjoin_path(root,
+ joinrel,
+ outer_path,
+ inner_path,
+ hashclauses,
+ jointype,
+ extra,
+ parallel_hash);
+ }
+
+ /*
* clause_sides_match_join
* Determine whether a join clause is of the right form to use in this join.
*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 885,896 ****
JoinType jointype,
JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
Path *outer_path;
Path *inner_path;
Path *cheapest_partial_outer = NULL;
Path *cheapest_safe_inner = NULL;
- List *all_pathkeys;
ListCell *l;
/*
--- 1065,1098 ----
JoinType jointype,
JoinPathExtraData *extra)
{
+ List *sort_infos = get_merge_join_sort_info(root, extra, joinrel,
+ jointype);
+
+
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, sort_infos);
+ }
+
+ /*
+ * The workhorse of sort_inner_and_outer().
+ *
+ * sort_infos contains one MergeJoinSortInfo per possible sorting of the input
+ * paths that we should consider.
+ */
+ static void
+ sort_inner_and_outer_common(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra,
+ List *sort_infos)
+ {
JoinType save_jointype = jointype;
Path *outer_path;
Path *inner_path;
Path *cheapest_partial_outer = NULL;
Path *cheapest_safe_inner = NULL;
ListCell *l;
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 962,1032 ****
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
! /*
! * Each possible ordering of the available mergejoin clauses will generate
! * a differently-sorted result path at essentially the same cost. We have
! * no basis for choosing one over another at this level of joining, but
! * some sort orders may be more useful than others for higher-level
! * mergejoins, so it's worth considering multiple orderings.
! *
! * Actually, it's not quite true that every mergeclause ordering will
! * generate a different path order, because some of the clauses may be
! * partially redundant (refer to the same EquivalenceClasses). Therefore,
! * what we do is convert the mergeclause list to a list of canonical
! * pathkeys, and then consider different orderings of the pathkeys.
! *
! * Generating a path for *every* permutation of the pathkeys doesn't seem
! * like a winning strategy; the cost in planning time is too high. For
! * now, we generate one path for each pathkey, listing that pathkey first
! * and the rest in random order. This should allow at least a one-clause
! * mergejoin without re-sorting against any other possible mergejoin
! * partner path. But if we've not guessed the right ordering of secondary
! * keys, we may end up evaluating clauses as qpquals when they could have
! * been done as mergeclauses. (In practice, it's rare that there's more
! * than two or three mergeclauses, so expending a huge amount of thought
! * on that is probably not worth it.)
! *
! * The pathkey order returned by select_outer_pathkeys_for_merge() has
! * some heuristics behind it (see that function), so be sure to try it
! * exactly as-is as well as making variants.
! */
! all_pathkeys = select_outer_pathkeys_for_merge(root,
! extra->mergeclause_list,
! joinrel);
!
! foreach(l, all_pathkeys)
{
! List *front_pathkey = (List *) lfirst(l);
! List *cur_mergeclauses;
! List *outerkeys;
! List *innerkeys;
! List *merge_pathkeys;
!
! /* Make a pathkey list with this guy first */
! if (l != list_head(all_pathkeys))
! outerkeys = lcons(front_pathkey,
! list_delete_ptr(list_copy(all_pathkeys),
! front_pathkey));
! else
! outerkeys = all_pathkeys; /* no work at first one... */
!
! /* Sort the mergeclauses into the corresponding ordering */
! cur_mergeclauses = find_mergeclauses_for_pathkeys(root,
! outerkeys,
! true,
! extra->mergeclause_list);
!
! /* Should have used them all... */
! Assert(list_length(cur_mergeclauses) == list_length(extra->mergeclause_list));
!
! /* Build sort pathkeys for the inner side */
! innerkeys = make_inner_pathkeys_for_merge(root,
! cur_mergeclauses,
! outerkeys);
!
! /* Build pathkeys representing output sort order */
! merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerkeys);
/*
* And now we can make the path.
--- 1164,1172 ----
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
! foreach(l, sort_infos)
{
! MergeJoinSortInfo *si = (MergeJoinSortInfo *) lfirst(l);
/*
* And now we can make the path.
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1035,1067 ****
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra);
}
}
--- 1175,1193 ----
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! si, jointype, extra, false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_mergejoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! si, jointype, extra,
! true);
}
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1091,1113 ****
List *merge_pathkeys,
bool is_partial)
{
- List *mergeclauses;
- List *innersortkeys;
List *trialsortkeys;
Path *cheapest_startup_inner;
Path *cheapest_total_inner;
JoinType save_jointype = jointype;
int num_sortkeys;
int sortkeycnt;
if (jointype == JOIN_UNIQUE_OUTER || jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
/* Look for useful mergeclauses (if any) */
! mergeclauses = find_mergeclauses_for_pathkeys(root,
! outerpath->pathkeys,
! true,
! extra->mergeclause_list);
/*
* Done with this outer path if no chance for a mergejoin.
--- 1217,1240 ----
List *merge_pathkeys,
bool is_partial)
{
List *trialsortkeys;
Path *cheapest_startup_inner;
Path *cheapest_total_inner;
JoinType save_jointype = jointype;
int num_sortkeys;
int sortkeycnt;
+ MergeJoinSortInfo si;
if (jointype == JOIN_UNIQUE_OUTER || jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
+ si.merge_pathkeys = merge_pathkeys;
/* Look for useful mergeclauses (if any) */
! si.merge_clauses = find_mergeclauses_for_pathkeys(root,
! outerpath->pathkeys,
! true,
! extra->mergeclause_list);
! si.outerkeys = NIL;
/*
* Done with this outer path if no chance for a mergejoin.
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1118,1124 ****
* without any join clauses, it's necessary to generate a clauseless
* mergejoin path instead.
*/
! if (mergeclauses == NIL)
{
if (jointype == JOIN_FULL)
/* okay to try for mergejoin */ ;
--- 1245,1251 ----
* without any join clauses, it's necessary to generate a clauseless
* mergejoin path instead.
*/
! if (si.merge_clauses == NIL)
{
if (jointype == JOIN_FULL)
/* okay to try for mergejoin */ ;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1126,1138 ****
return;
}
if (useallclauses &&
! list_length(mergeclauses) != list_length(extra->mergeclause_list))
return;
/* Compute the required ordering of the inner path */
! innersortkeys = make_inner_pathkeys_for_merge(root,
! mergeclauses,
! outerpath->pathkeys);
/*
* Generate a mergejoin on the basis of sorting the cheapest inner. Since
--- 1253,1265 ----
return;
}
if (useallclauses &&
! list_length(si.merge_clauses) != list_length(extra->mergeclause_list))
return;
/* Compute the required ordering of the inner path */
! si.innerkeys = make_inner_pathkeys_for_merge(root,
! si.merge_clauses,
! outerpath->pathkeys);
/*
* Generate a mergejoin on the basis of sorting the cheapest inner. Since
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1140,1156 ****
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1267,1280 ----
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! &si,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1185,1191 ****
* sorting inner_cheapest_total, whereas we did sort it above, so the
* plans being considered are different.
*/
! if (pathkeys_contained_in(innersortkeys,
inner_cheapest_total->pathkeys))
{
/* inner_cheapest_total didn't require a sort */
--- 1309,1315 ----
* sorting inner_cheapest_total, whereas we did sort it above, so the
* plans being considered are different.
*/
! if (pathkeys_contained_in(si.innerkeys,
inner_cheapest_total->pathkeys))
{
/* inner_cheapest_total didn't require a sort */
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1198,1213 ****
cheapest_startup_inner = NULL;
cheapest_total_inner = NULL;
}
! num_sortkeys = list_length(innersortkeys);
if (num_sortkeys > 1 && !useallclauses)
! trialsortkeys = list_copy(innersortkeys); /* need modifiable copy */
else
! trialsortkeys = innersortkeys; /* won't really truncate */
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
! List *newclauses = NIL;
/*
* Look for an inner path ordered well enough for the first
--- 1322,1342 ----
cheapest_startup_inner = NULL;
cheapest_total_inner = NULL;
}
! num_sortkeys = list_length(si.innerkeys);
if (num_sortkeys > 1 && !useallclauses)
! trialsortkeys = list_copy(si.innerkeys); /* need modifiable copy */
else
! trialsortkeys = si.innerkeys; /* won't really truncate */
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
! MergeJoinSortInfo si_trial;
!
! si_trial.outerkeys = NIL;
! si_trial.innerkeys = NIL;
! si_trial.merge_pathkeys = si.merge_pathkeys;
! si_trial.merge_clauses = NIL;
/*
* Look for an inner path ordered well enough for the first
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1229,1254 ****
/* Select the right mergeclauses, if we didn't already */
if (sortkeycnt < num_sortkeys)
{
! newclauses =
find_mergeclauses_for_pathkeys(root,
trialsortkeys,
false,
! mergeclauses);
! Assert(newclauses != NIL);
}
else
! newclauses = mergeclauses;
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
--- 1358,1381 ----
/* Select the right mergeclauses, if we didn't already */
if (sortkeycnt < num_sortkeys)
{
! si_trial.merge_clauses =
find_mergeclauses_for_pathkeys(root,
trialsortkeys,
false,
! si.merge_clauses);
! Assert(si_trial.merge_clauses != NIL);
}
else
! si_trial.merge_clauses = si.merge_clauses;
!
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! &si,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1269,1299 ****
* Avoid rebuilding clause list if we already made one; saves
* memory in big join trees...
*/
! if (newclauses == NIL)
{
if (sortkeycnt < num_sortkeys)
{
! newclauses =
find_mergeclauses_for_pathkeys(root,
trialsortkeys,
false,
! mergeclauses);
! Assert(newclauses != NIL);
}
else
! newclauses = mergeclauses;
}
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1396,1423 ----
* Avoid rebuilding clause list if we already made one; saves
* memory in big join trees...
*/
! if (si_trial.merge_clauses == NIL)
{
if (sortkeycnt < num_sortkeys)
{
! si_trial.merge_clauses =
find_mergeclauses_for_pathkeys(root,
trialsortkeys,
false,
! si.merge_clauses);
! Assert(si_trial.merge_clauses != NIL);
}
else
! si_trial.merge_clauses = si.merge_clauses;
}
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! &si,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1306,1341 ****
}
}
- /*
- * match_unsorted_outer
- * Creates possible join paths for processing a single join relation
- * 'joinrel' by employing either iterative substitution or
- * mergejoining on each of its possible outer paths (considering
- * only outer paths that are already ordered well enough for merging).
- *
- * We always generate a nestloop path for each available outer path.
- * In fact we may generate as many as five: one on the cheapest-total-cost
- * inner path, one on the same with materialization, one on the
- * cheapest-startup-cost inner path (if different), one on the
- * cheapest-total inner-indexscan path (if any), and one on the
- * cheapest-startup inner-indexscan path (if different).
- *
- * We also consider mergejoins if mergejoin clauses are available. See
- * detailed comments in generate_mergejoin_paths.
- *
- * 'joinrel' is the join relation
- * 'outerrel' is the outer join relation
- * 'innerrel' is the inner join relation
- * 'jointype' is the type of join to do
- * 'extra' contains additional input values
- */
static void
! match_unsorted_outer(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
--- 1430,1442 ----
}
}
static void
! match_unsorted_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1396,1401 ****
--- 1497,1503 ----
/* No way to do this with an inner path parameterized by outer rel */
if (inner_cheapest_total == NULL)
return;
+
inner_cheapest_total = (Path *)
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
*************** match_unsorted_outer(PlannerInfo *root,
*** 1433,1438 ****
--- 1535,1541 ----
{
if (outerpath != outerrel->cheapest_total_path)
continue;
+
outerpath = (Path *) create_unique_path(root, outerrel,
outerpath, extra->sjinfo);
Assert(outerpath);
*************** match_unsorted_outer(PlannerInfo *root,
*** 1452,1464 ****
* Consider nestloop join, but only with the unique-ified cheapest
* inner path
*/
! try_nestloop_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra);
}
else if (nestjoinOK)
{
--- 1555,1568 ----
* Consider nestloop join, but only with the unique-ified cheapest
* inner path
*/
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra,
! false);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1474,1497 ****
{
Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra);
}
! /* Also consider materialized form of the cheapest inner path */
if (matpath != NULL)
! try_nestloop_path(root,
! joinrel,
! outerpath,
! matpath,
! merge_pathkeys,
! jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 1578,1603 ----
{
Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! false);
}
! /* Also consider materialized form of the cheapest inner path. */
if (matpath != NULL)
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! matpath,
! merge_pathkeys,
! jointype,
! extra,
! false);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1552,1557 ****
--- 1658,1698 ----
}
/*
+ * match_unsorted_outer
+ * Creates possible join paths for processing a single join relation
+ * 'joinrel' by employing either iterative substitution or
+ * mergejoining on each of its possible outer paths (considering
+ * only outer paths that are already ordered well enough for merging).
+ *
+ * We always generate a nestloop path for each available outer path.
+ * In fact we may generate as many as five: one on the cheapest-total-cost
+ * inner path, one on the same with materialization, one on the
+ * cheapest-startup-cost inner path (if different), one on the
+ * cheapest-total inner-indexscan path (if any), and one on the
+ * cheapest-startup inner-indexscan path (if different).
+ *
+ * We also consider mergejoins if mergejoin clauses are available. See
+ * detailed comments in generate_mergejoin_paths.
+ *
+ * 'joinrel' is the join relation
+ * 'outerrel' is the outer join relation
+ * 'innerrel' is the inner join relation
+ * 'jointype' is the type of join to do
+ * 'extra' contains additional input values
+ */
+ static void
+ match_unsorted_outer(PlannerInfo *root,
+ RelOptInfo *joinrel,
+ RelOptInfo *outerrel,
+ RelOptInfo *innerrel,
+ JoinType jointype,
+ JoinPathExtraData *extra)
+ {
+ match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra);
+ }
+
+ /*
* consider_parallel_mergejoin
* Try to build partial paths for a joinrel by joining a partial path
* for the outer relation to a complete path for the inner relation.
*************** consider_parallel_nestloop(PlannerInfo *
*** 1658,1667 ****
Assert(innerpath);
}
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
/*
--- 1799,2029 ----
Assert(innerpath);
}
! try_nestloop_path_common(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra, true);
! }
! }
! }
!
! /*
! * The workhorse of hash_inner_and_outer(). The hashclauses list is passed by
! * caller because it does not differ between calls.
! */
! static void
! hash_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! List *hashclauses)
! {
! JoinType save_jointype = jointype;
!
! /*
! * We consider both the cheapest-total-cost and cheapest-startup-cost
! * outer paths. There's no need to consider any but the
! * cheapest-total-cost inner path, however.
! */
! Path *cheapest_startup_outer,
! *cheapest_total_outer,
! *cheapest_total_inner;
!
! /* Shouldn't be called w/o hashclauses. */
! Assert(hashclauses != NIL);
!
! cheapest_startup_outer = outerrel->cheapest_startup_path;
! cheapest_total_outer = outerrel->cheapest_total_path;
! cheapest_total_inner = innerrel->cheapest_total_path;
!
! /*
! * If either cheapest-total path is parameterized by the other rel, we
! * can't use a hashjoin. (There's no use looking for alternative input
! * paths, since these should already be the least-parameterized available
! * paths.)
! */
! if (PATH_PARAM_BY_REL(cheapest_total_outer, innerrel) ||
! PATH_PARAM_BY_REL(cheapest_total_inner, outerrel))
! return;
!
! /* Unique-ify if need be; we ignore parameterized possibilities */
! if (jointype == JOIN_UNIQUE_OUTER)
! {
! cheapest_total_outer = (Path *)
! create_unique_path(root, outerrel,
! cheapest_total_outer, extra->sjinfo);
! Assert(cheapest_total_outer);
! jointype = JOIN_INNER;
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! false);
! /* no possibility of cheap startup here */
! }
! else if (jointype == JOIN_UNIQUE_INNER)
! {
! cheapest_total_inner = (Path *)
! create_unique_path(root, innerrel,
! cheapest_total_inner, extra->sjinfo);
! Assert(cheapest_total_inner);
! jointype = JOIN_INNER;
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! false);
!
! if (cheapest_startup_outer != NULL &&
! cheapest_startup_outer != cheapest_total_outer)
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! false);
! }
! else
! {
! /*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider pairings
! * of cheapest-total paths including parameterized ones. There is no
! * use in generating parameterized paths on the basis of possibly
! * cheap startup cost, so this is sufficient.
! *
! * For if either side of the join is grouped, we simply use
! * rel->gpi->pathlist. (cheapest_startup_outer should be NULL if
! * grouped_outer.)
! */
! ListCell *lc1;
! ListCell *lc2;
!
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path_common(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false,
! false);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
! {
! Path *outerpath = (Path *) lfirst(lc1);
!
! /*
! * We cannot use an outer path that is parameterized by the inner
! * rel.
! */
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
!
! /*
! * We cannot use an inner path that is parameterized by the
! * outer rel, either.
! */
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
! continue;
!
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
!
! try_hashjoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra,
! false,
! false);
! }
}
}
+
+ /*
+ * If the joinrel is parallel-safe, we may be able to consider a partial
+ * hash join. However, we can't handle JOIN_UNIQUE_OUTER, because the
+ * outer path will be partial, and therefore we won't be able to properly
+ * guarantee uniqueness. Similarly, we can't handle JOIN_FULL and
+ * JOIN_RIGHT, because they can produce false null extended rows. Also,
+ * the resulting path must not be parameterized. We would be able to
+ * support JOIN_FULL and JOIN_RIGHT for Parallel Hash, since in that case
+ * we're back to a single hash table with a single set of match bits for
+ * each batch, but that will require figuring out a deadlock-free way to
+ * wait for the probe to finish.
+ */
+ if (joinrel->consider_parallel &&
+ save_jointype != JOIN_UNIQUE_OUTER &&
+ save_jointype != JOIN_FULL &&
+ save_jointype != JOIN_RIGHT &&
+ outerrel->partial_pathlist != NIL &&
+ bms_is_empty(joinrel->lateral_relids))
+ {
+ Path *cheapest_partial_outer;
+ Path *cheapest_partial_inner = NULL;
+ Path *cheapest_safe_inner = NULL;
+
+ cheapest_partial_outer =
+ (Path *) linitial(outerrel->partial_pathlist);
+
+ /*
+ * Can we use a partial inner plan too, so that we can build a shared
+ * hash table in parallel?
+ */
+ if (innerrel->partial_pathlist != NIL && enable_parallel_hash)
+ {
+ cheapest_partial_inner =
+ (Path *) linitial(innerrel->partial_pathlist);
+ try_hashjoin_path_common(root, joinrel,
+ cheapest_partial_outer,
+ cheapest_partial_inner,
+ hashclauses, jointype, extra,
+ true,
+ true /* parallel_hash */ );
+ }
+
+
+ /*
+ * Normally, given that the joinrel is parallel-safe, the cheapest
+ * total inner path will also be parallel-safe, but if not, we'll have
+ * to search cheapest_parameterized_paths for the cheapest safe,
+ * unparameterized inner path. If doing JOIN_UNIQUE_INNER, we can't
+ * use any alternative inner path.
+ */
+ if (cheapest_total_inner->parallel_safe)
+ cheapest_safe_inner = cheapest_total_inner;
+ else if (save_jointype != JOIN_UNIQUE_INNER)
+ cheapest_safe_inner =
+ get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
+
+ if (cheapest_safe_inner != NULL)
+ try_hashjoin_path_common(root, joinrel,
+ cheapest_partial_outer,
+ cheapest_safe_inner,
+ hashclauses, jointype, extra,
+ true,
+ false); /* parallel_hash */
+ }
}
/*
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1683,1691 ****
JoinType jointype,
JoinPathExtraData *extra)
{
- JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
! List *hashclauses;
ListCell *l;
/*
--- 2045,2052 ----
JoinType jointype,
JoinPathExtraData *extra)
{
bool isouterjoin = IS_OUTER_JOIN(jointype);
! List *hashclauses = NIL;
ListCell *l;
/*
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1695,1701 ****
* Scan the join's restrictinfo list to find hashjoinable clauses that are
* usable with this pair of sub-relations.
*/
- hashclauses = NIL;
foreach(l, extra->restrictlist)
{
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(l);
--- 2056,2061 ----
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1720,1908 ****
hashclauses = lappend(hashclauses, restrictinfo);
}
! /* If we found any usable hashclauses, make paths */
! if (hashclauses)
! {
! /*
! * We consider both the cheapest-total-cost and cheapest-startup-cost
! * outer paths. There's no need to consider any but the
! * cheapest-total-cost inner path, however.
! */
! Path *cheapest_startup_outer = outerrel->cheapest_startup_path;
! Path *cheapest_total_outer = outerrel->cheapest_total_path;
! Path *cheapest_total_inner = innerrel->cheapest_total_path;
!
! /*
! * If either cheapest-total path is parameterized by the other rel, we
! * can't use a hashjoin. (There's no use looking for alternative
! * input paths, since these should already be the least-parameterized
! * available paths.)
! */
! if (PATH_PARAM_BY_REL(cheapest_total_outer, innerrel) ||
! PATH_PARAM_BY_REL(cheapest_total_inner, outerrel))
! return;
!
! /* Unique-ify if need be; we ignore parameterized possibilities */
! if (jointype == JOIN_UNIQUE_OUTER)
! {
! cheapest_total_outer = (Path *)
! create_unique_path(root, outerrel,
! cheapest_total_outer, extra->sjinfo);
! Assert(cheapest_total_outer);
! jointype = JOIN_INNER;
! try_hashjoin_path(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
! /* no possibility of cheap startup here */
! }
! else if (jointype == JOIN_UNIQUE_INNER)
! {
! cheapest_total_inner = (Path *)
! create_unique_path(root, innerrel,
! cheapest_total_inner, extra->sjinfo);
! Assert(cheapest_total_inner);
! jointype = JOIN_INNER;
! try_hashjoin_path(root,
! joinrel,
! cheapest_total_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
! if (cheapest_startup_outer != NULL &&
! cheapest_startup_outer != cheapest_total_outer)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
! }
! else
! {
! /*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized ones.
! * There is no use in generating parameterized paths on the basis
! * of possibly cheap startup cost, so this is sufficient.
! */
! ListCell *lc1;
! ListCell *lc2;
!
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
! {
! Path *outerpath = (Path *) lfirst(lc1);
!
! /*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
! */
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
!
! /*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
! */
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
! continue;
!
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
!
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra);
! }
! }
! }
!
! /*
! * If the joinrel is parallel-safe, we may be able to consider a
! * partial hash join. However, we can't handle JOIN_UNIQUE_OUTER,
! * because the outer path will be partial, and therefore we won't be
! * able to properly guarantee uniqueness. Similarly, we can't handle
! * JOIN_FULL and JOIN_RIGHT, because they can produce false null
! * extended rows. Also, the resulting path must not be parameterized.
! * We would be able to support JOIN_FULL and JOIN_RIGHT for Parallel
! * Hash, since in that case we're back to a single hash table with a
! * single set of match bits for each batch, but that will require
! * figuring out a deadlock-free way to wait for the probe to finish.
! */
! if (joinrel->consider_parallel &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_FULL &&
! save_jointype != JOIN_RIGHT &&
! outerrel->partial_pathlist != NIL &&
! bms_is_empty(joinrel->lateral_relids))
! {
! Path *cheapest_partial_outer;
! Path *cheapest_partial_inner = NULL;
! Path *cheapest_safe_inner = NULL;
!
! cheapest_partial_outer =
! (Path *) linitial(outerrel->partial_pathlist);
!
! /*
! * Can we use a partial inner plan too, so that we can build a
! * shared hash table in parallel?
! */
! if (innerrel->partial_pathlist != NIL && enable_parallel_hash)
! {
! cheapest_partial_inner =
! (Path *) linitial(innerrel->partial_pathlist);
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_partial_inner,
! hashclauses, jointype, extra,
! true /* parallel_hash */ );
! }
!
! /*
! * Normally, given that the joinrel is parallel-safe, the cheapest
! * total inner path will also be parallel-safe, but if not, we'll
! * have to search for the cheapest safe, unparameterized inner
! * path. If doing JOIN_UNIQUE_INNER, we can't use any alternative
! * inner path.
! */
! if (cheapest_total_inner->parallel_safe)
! cheapest_safe_inner = cheapest_total_inner;
! else if (save_jointype != JOIN_UNIQUE_INNER)
! cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra,
! false /* parallel_hash */ );
! }
! }
}
/*
--- 2080,2090 ----
hashclauses = lappend(hashclauses, restrictinfo);
}
! if (hashclauses == NIL)
! return;
! hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, hashclauses);
}
/*
agg_pushdown_v5/07_join.diff
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index e0dc8d9..30ac459
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 269,274 ****
--- 269,275 ----
/* Create GatherPaths for any useful partial paths for rel */
generate_gather_paths(root, joinrel, false);
+ generate_gather_paths(root, joinrel, true);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 1ccb834..36e038e
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** static void set_append_rel_pathlist(Plan
*** 97,103 ****
static void generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels);
static Path *get_cheapest_parameterized_child_path(PlannerInfo *root,
RelOptInfo *rel,
Relids required_outer);
--- 97,104 ----
static void generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels,
! bool grouped);
static Path *get_cheapest_parameterized_child_path(PlannerInfo *root,
RelOptInfo *rel,
Relids required_outer);
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1873,1879 ****
if (subpaths_valid)
generate_mergeappend_paths(root, rel, live_childrels,
all_child_pathkeys,
! partitioned_rels);
/*
* Build Append paths for each parameterization seen among the child rels.
--- 1874,1884 ----
if (subpaths_valid)
generate_mergeappend_paths(root, rel, live_childrels,
all_child_pathkeys,
! partitioned_rels, false);
! if (grouped_subpaths_valid)
! generate_mergeappend_paths(root, rel, live_childrels,
! all_child_pathkeys,
! partitioned_rels, true);
/*
* Build Append paths for each parameterization seen among the child rels.
*************** static void
*** 1949,1957 ****
generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels)
{
ListCell *lcp;
foreach(lcp, all_child_pathkeys)
{
--- 1954,1964 ----
generate_mergeappend_paths(PlannerInfo *root, RelOptInfo *rel,
List *live_childrels,
List *all_child_pathkeys,
! List *partitioned_rels,
! bool grouped)
{
ListCell *lcp;
+ PathTarget *target = NULL;
foreach(lcp, all_child_pathkeys)
{
*************** generate_mergeappend_paths(PlannerInfo *
*** 1960,1982 ****
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
Path *cheapest_startup,
*cheapest_total;
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
TOTAL_COST,
--- 1967,1998 ----
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
+ Path *path;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
+ List *pathlist;
Path *cheapest_startup,
*cheapest_total;
+ if (grouped &&
+ (childrel->gpi == NULL || childrel->gpi->pathlist == NIL))
+ return;
+
+ pathlist = !grouped ? childrel->pathlist :
+ childrel->gpi->pathlist;
+
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
TOTAL_COST,
*************** generate_mergeappend_paths(PlannerInfo *
*** 2008,2029 ****
&total_subpaths, NULL);
}
/* ... and build the MergeAppend paths */
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels),
! false);
}
}
--- 2024,2070 ----
&total_subpaths, NULL);
}
+ /*
+ * Special target is needed if the path output should be grouped.
+ */
+ if (grouped)
+ {
+ Assert(rel->gpi != NULL);
+ target = rel->gpi->target;
+ }
+
/* ... and build the MergeAppend paths */
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
!
! /* pathtarget will produce the grouped relation.. */
! if (grouped)
! {
! Assert(rel->gpi != NULL && rel->gpi->target != NULL);
! path->pathtarget = rel->gpi->target;
! }
!
! add_path(rel, path, grouped);
!
if (startup_neq_total)
! {
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
! if (grouped)
! path->pathtarget = rel->gpi->target;
! add_path(rel, path, grouped);
! }
!
}
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index ab739db..7422fc4
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***************
*** 22,27 ****
--- 22,28 ----
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
+ #include "optimizer/tlist.h"
/* Hook for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
*************** set_join_pathlist_hook_type set_join_pat
*** 39,44 ****
--- 40,49 ----
#define PATH_PARAM_BY_REL(path, rel) \
(PATH_PARAM_BY_REL_SELF(path, rel) || PATH_PARAM_BY_PARENT(path, rel))
+ #define REL_HAS_GROUPED_PATHS(rel) ((rel)->gpi && (rel)->gpi->pathlist)
+ #define REL_HAS_PARTIAL_GROUPED_PATHS(rel) \
+ ((rel)->gpi && (rel)->gpi->partial_pathlist)
+
/*
* The sort-specific input for try_mergejoin_... functions.
*/
*************** static void try_partial_mergejoin_path(P
*** 56,62 ****
Path *inner_path,
MergeJoinSortInfo *si,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 61,69 ----
Path *inner_path,
MergeJoinSortInfo *si,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** static void sort_inner_and_outer_common(
*** 66,72 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! List *sort_infos);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 73,82 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! List *sort_infos,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** static void consider_parallel_nestloop(P
*** 75,88 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
--- 85,101 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped_outer,
! bool do_aggregate);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, JoinPathExtraData *extra);
*************** static void generate_mergejoin_paths(Pla
*** 102,108 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 115,124 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
/*
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 317,324 ****
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
! hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 333,340 ----
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
! hash_inner_and_outer(root, joinrel, outerrel, innerrel, jointype,
! &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 351,357 ****
* across joins unless there's a join-order-constraint-based reason to do so.
* So we ignore the param_source_rels restriction when this case applies.
*
! * allow_star_schema_join() returns true if the param_source_rels restriction
* should be overridden, ie, it's okay to perform this join.
*/
static inline bool
--- 367,373 ----
* across joins unless there's a join-order-constraint-based reason to do so.
* So we ignore the param_source_rels restriction when this case applies.
*
! * allow_star_schema_join() returns TRUE if the param_source_rels restriction
* should be overridden, ie, it's okay to perform this join.
*/
static inline bool
*************** try_nestloop_path(PlannerInfo *root,
*** 379,385 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
--- 395,403 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
*************** try_nestloop_path(PlannerInfo *root,
*** 389,394 ****
--- 407,426 ----
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi || !grouped);
+
+ /*
+ * Aggregation target is necessary to produce grouped join output.
+ */
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* Paths are parameterized by top-level parents, so run parameterization
*************** try_nestloop_path(PlannerInfo *root,
*** 409,420 ****
* parameterization wouldn't be sensible --- unless allow_star_schema_join
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
! * still parameterized.
*/
required_outer = calc_nestloop_required_outer(outerrelids, outer_paramrels,
innerrelids, inner_paramrels);
if (required_outer &&
! ((!bms_overlap(required_outer, extra->param_source_rels) &&
!allow_star_schema_join(root, outerrelids, inner_paramrels)) ||
have_dangerous_phv(root, outerrelids, inner_paramrels)))
{
--- 441,453 ----
* parameterization wouldn't be sensible --- unless allow_star_schema_join
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
! * still parameterized. Finally reject parameterized aggregation.
*/
required_outer = calc_nestloop_required_outer(outerrelids, outer_paramrels,
innerrelids, inner_paramrels);
if (required_outer &&
! (do_aggregate ||
! (!bms_overlap(required_outer, extra->param_source_rels) &&
!allow_star_schema_join(root, outerrelids, inner_paramrels)) ||
have_dangerous_phv(root, outerrelids, inner_paramrels)))
{
*************** try_nestloop_path(PlannerInfo *root,
*** 435,476 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, false))
{
/*
! * If the inner path is parameterized, it is parameterized by the
! * topmost parent of the outer rel, not the outer rel itself. Fix
! * that.
*/
! if (PATH_PARAM_BY_PARENT(inner_path, outer_path->parent))
{
! inner_path = reparameterize_path_by_child(root, inner_path,
! outer_path->parent);
!
! /*
! * If we could not translate the path, we can't create nest loop
! * path.
! */
! if (!inner_path)
! {
! bms_free(required_outer);
! return;
! }
}
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer), false);
}
else
{
--- 468,526 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! /*
! * If the inner path is parameterized, it is parameterized by the topmost
! * parent of the outer rel, not the outer rel itself. Fix that.
! */
! if (PATH_PARAM_BY_PARENT(inner_path, outer_path->parent))
{
+ inner_path = reparameterize_path_by_child(root, inner_path,
+ outer_path->parent);
+
/*
! * If we could not translate the path, we can't create nest loop path.
*/
! if (!inner_path)
{
! bms_free(required_outer);
! return;
}
+ }
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! required_outer, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate && required_outer == NULL)
! {
! PartialAggInfo *agg_info = get_partial_agg_info(root,
! joinrel->gpi->target,
! joinrel->gpi->sortgroupclauses);
!
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED, agg_info);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED, agg_info);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
! {
! add_path(joinrel, join_path, grouped);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 491,499 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 541,559 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_nestloop_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 528,535 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys,
! false))
return;
/*
--- 588,655 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! NULL, join_target);
!
! if (do_aggregate)
! {
! PartialAggInfo *agg_info = get_partial_agg_info(root,
! joinrel->gpi->target,
! joinrel->gpi->sortgroupclauses);
!
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED,
! agg_info);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED,
! agg_info);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_nestloop_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
! return;
!
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target == NULL)
return;
/*
*************** try_partial_nestloop_path(PlannerInfo *r
*** 548,565 ****
return;
}
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL), false);
}
static void
--- 668,680 ----
return;
}
! if (!partial)
! try_nestloop_path(root, joinrel, outer_path, inner_path, pathkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_nestloop_path(root, joinrel, outer_path, inner_path,
! pathkeys, jointype, extra, true,
! do_aggregate);
}
static void
*************** try_nestloop_path_common(PlannerInfo *ro
*** 570,598 ****
List *pathkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool partial)
{
! /* Only join plain paths. */
! if (!partial)
! try_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra);
else
! try_partial_nestloop_path(root,
joinrel,
outer_path,
inner_path,
pathkeys,
jointype,
! extra);
}
-
/*
* Create one MergeJoinSortInfo for each possible use of merge clauses.
*/
--- 685,744 ----
List *pathkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
+ bool grouped_join;
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! grouped_join,
! do_aggregate);
! else
! try_partial_nestloop_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! jointype,
! extra,
! grouped_join,
! do_aggregate);
! }
else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_nestloop_path(root,
joinrel,
outer_path,
inner_path,
pathkeys,
jointype,
! extra,
! partial,
! do_aggregate);
! }
}
/*
* Create one MergeJoinSortInfo for each possible use of merge clauses.
*/
*************** try_mergejoin_path(PlannerInfo *root,
*** 687,711 ****
Path *inner_path,
MergeJoinSortInfo *si,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
List *outersortkeys = si->outerkeys;
List *innersortkeys = si->innerkeys;
/*
* Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 833,876 ----
Path *inner_path,
MergeJoinSortInfo *si,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
List *outersortkeys = si->outerkeys;
List *innersortkeys = si->innerkeys;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi || !grouped);
+
+ /*
+ * Aggregation target is necessary to produce grouped join output.
+ */
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible. Also reject parameterized
! * aggregation.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (!bms_overlap(required_outer, extra->param_source_rels) ||
! do_aggregate)
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 726,749 ****
outer_path, inner_path,
outersortkeys, innersortkeys, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! si->merge_pathkeys, required_outer, false))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! si->merge_pathkeys,
! required_outer,
! si->merge_clauses,
! outersortkeys,
! innersortkeys), false);
}
else
{
--- 891,939 ----
outer_path, inner_path,
outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! si->merge_pathkeys,
! required_outer,
! si->merge_clauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
{
! PartialAggInfo *agg_info = get_partial_agg_info(root,
! joinrel->gpi->target,
! joinrel->gpi->sortgroupclauses);
!
! create_grouped_path(root, joinrel, join_path, true, false, AGG_HASHED,
! agg_info);
! create_grouped_path(root, joinrel, join_path, true, false, AGG_SORTED,
! agg_info);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! si->merge_pathkeys, required_outer, grouped))
! {
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 764,774 ****
Path *inner_path,
MergeJoinSortInfo *si,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
List *outersortkeys = si->outerkeys;
List *innersortkeys = si->innerkeys;
/*
* See comments in try_partial_hashjoin_path().
--- 954,974 ----
Path *inner_path,
MergeJoinSortInfo *si,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
List *outersortkeys = si->outerkeys;
List *innersortkeys = si->innerkeys;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_mergejoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 800,825 ****
outer_path, inner_path,
outersortkeys, innersortkeys, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost,
! si->merge_pathkeys,
! false))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! si->merge_pathkeys,
! NULL,
! si->merge_clauses,
! outersortkeys,
! innersortkeys), false);
}
/*
--- 1000,1083 ----
outer_path, inner_path,
outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! si->merge_pathkeys,
! NULL,
! si->merge_clauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! if (do_aggregate)
! {
! PartialAggInfo *agg_info = get_partial_agg_info(root,
! joinrel->gpi->target,
! joinrel->gpi->sortgroupclauses);
!
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED,
! agg_info);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED,
! agg_info);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! si->merge_pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_mergejoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! MergeJoinSortInfo *si,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target == NULL)
! return;
!
! if (!partial)
! try_mergejoin_path(root, joinrel, outer_path, inner_path, si,
! jointype, extra, true, do_aggregate);
! else
! try_partial_mergejoin_path(root, joinrel, outer_path, inner_path, si,
! jointype, extra, true, do_aggregate);
}
/*
*************** try_mergejoin_path_common(PlannerInfo *r
*** 834,858 ****
MergeJoinSortInfo *si,
JoinType jointype,
JoinPathExtraData *extra,
! bool partial)
{
! /* Only join plain paths. */
! if (!partial)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! si,
! jointype,
! extra);
else
! try_partial_mergejoin_path(root,
joinrel,
outer_path,
inner_path,
si,
jointype,
! extra);
}
/*
--- 1092,1147 ----
MergeJoinSortInfo *si,
JoinType jointype,
JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! si,
! jointype,
! extra,
! false, false);
! else
! try_partial_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! si,
! jointype,
! extra,
! false, false);
! }
else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_mergejoin_path(root,
joinrel,
outer_path,
inner_path,
si,
jointype,
! extra,
! partial,
! do_aggregate);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 867,889 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 1156,1197 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi || !grouped);
+
+ /*
+ * Aggregation target is necessary to produce grouped join output.
+ */
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible. Also reject parameterized
! * aggregation.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (!bms_overlap(required_outer, extra->param_source_rels) ||
! do_aggregate)
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 893,914 ****
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, false))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! false, /* parallel_hash */
! extra->restrictlist,
! required_outer,
! hashclauses), false);
}
else
{
--- 1201,1245 ----
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! false, /* parallel_hash */
! extra->restrictlist,
! required_outer, hashclauses,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
{
! PartialAggInfo *agg_info = get_partial_agg_info(root,
! joinrel->gpi->target,
! joinrel->gpi->sortgroupclauses);
!
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, joinrel, join_path, true, false, AGG_HASHED,
! agg_info);
! }
! else if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, grouped))
! {
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 934,942 ****
List *hashclauses,
JoinType jointype,
JoinPathExtraData *extra,
! bool parallel_hash)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1265,1283 ----
List *hashclauses,
JoinType jointype,
JoinPathExtraData *extra,
! bool parallel_hash,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_hashjoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi || !grouped);
+ Assert((joinrel->gpi && joinrel->gpi->target) || !grouped ||
+ !do_aggregate);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 958,980 ****
* cost. Bail out right away if it looks terrible.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
! outer_path, inner_path, extra, true);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL, false))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! parallel_hash,
! extra->restrictlist,
! NULL,
! hashclauses), false);
}
/*
--- 1299,1393 ----
* cost. Bail out right away if it looks terrible.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
! outer_path, inner_path, extra, parallel_hash);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! parallel_hash,
! extra->restrictlist, NULL,
! hashclauses, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! PartialAggInfo *agg_info = get_partial_agg_info(root,
! joinrel->gpi->target,
! joinrel->gpi->sortgroupclauses);
!
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED,
! agg_info);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! NIL, grouped))
! {
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! /*
! * Create a new grouped hash join path by joining a grouped path to plain
! * (non-grouped) one, or by joining 2 plain relations and applying grouping on
! * the result.
! *
! * Joining of 2 grouped paths is not supported. If a grouped relation A was
! * joined to grouped relation B, then the grouping of B reduces the number of
! * times each group of A is appears in the join output. This makes difference
! * for some aggregates, e.g. sum().
! *
! * If do_aggregate is true, neither input rel is grouped so we need to
! * aggregate the join result explicitly.
! *
! * partial argument tells whether the join path should be considered partial.
! */
! static void
! try_grouped_hashjoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool parallel_hash,
! bool do_aggregate
! )
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Can't preform explicit aggregation w/o the appropriate target.
! */
! if (do_aggregate && joinrel->gpi->target == NULL)
! return;
!
! if (!partial)
! try_hashjoin_path(root, joinrel, outer_path, inner_path, hashclauses,
! jointype, extra, true, do_aggregate);
! else
! try_partial_hashjoin_path(root, joinrel, outer_path, inner_path,
! hashclauses, jointype, extra, parallel_hash,
! true, do_aggregate);
}
/*
*************** try_hashjoin_path_common(PlannerInfo *ro
*** 990,1020 ****
JoinType jointype,
JoinPathExtraData *extra,
bool partial,
! bool parallel_hash)
{
! /* parallel_hash can only be set if partial is too. */
! Assert(!parallel_hash || partial);
! /* Only join plain paths. */
! if (!partial)
! try_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra);
else
! try_partial_hashjoin_path(root,
joinrel,
outer_path,
inner_path,
hashclauses,
jointype,
extra,
! parallel_hash);
}
/*
* clause_sides_match_join
* Determine whether a join clause is of the right form to use in this join.
--- 1403,1463 ----
JoinType jointype,
JoinPathExtraData *extra,
bool partial,
! bool parallel_hash,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
! bool grouped_join;
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! if (!partial)
! try_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! false, false);
! else
! try_partial_hashjoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! hashclauses,
! jointype,
! extra,
! parallel_hash,
! false, false);
! }
else
! {
! /*
! * Either exactly one of the input paths should be grouped, or no one
! * is grouped and do_aggregate is true.
! */
! try_grouped_hashjoin_path(root,
joinrel,
outer_path,
inner_path,
hashclauses,
jointype,
extra,
! partial,
! parallel_hash,
! do_aggregate);
! }
}
+
/*
* clause_sides_match_join
* Determine whether a join clause is of the right form to use in this join.
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1068,1076 ****
List *sort_infos = get_merge_join_sort_info(root, extra, joinrel,
jointype);
sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, sort_infos);
}
/*
--- 1511,1533 ----
List *sort_infos = get_merge_join_sort_info(root, extra, joinrel,
jointype);
+ /* Plain (non-grouped) join. */
+ sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
+ jointype, extra, sort_infos, false, false,
+ false);
+ /* Use all the supported strategies to generate grouped join. */
sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, sort_infos, true, false,
! false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, sort_infos, false, true,
! false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, sort_infos, false, false,
! true);
!
! list_free_deep(sort_infos);
}
/*
*************** sort_inner_and_outer_common(PlannerInfo
*** 1086,1092 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! List *sort_infos)
{
JoinType save_jointype = jointype;
Path *outer_path;
--- 1543,1552 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! List *sort_infos,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
Path *outer_path;
*************** sort_inner_and_outer_common(PlannerInfo
*** 1108,1115 ****
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! outer_path = outerrel->cheapest_total_path;
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 1568,1592 ----
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! if (grouped_outer)
! {
! if (REL_HAS_GROUPED_PATHS(outerrel))
! outer_path = linitial(outerrel->gpi->pathlist);
! else
! return;
! }
! else
! outer_path = outerrel->cheapest_total_path;
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** sort_inner_and_outer_common(PlannerInfo
*** 1127,1132 ****
--- 1604,1619 ----
*/
if (jointype == JOIN_UNIQUE_OUTER)
{
+ /*
+ * TODO This is just a temporary limitation. Before lifting it, make
+ * sure that the UniquePath does emit GroupedVars. Also try to avoid
+ * the unique-ification if outer_path comes directly from AggPath
+ * (i.e. it's not grouped path combined with plain one) and the
+ * grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_outer)
+ return;
+
outer_path = (Path *) create_unique_path(root, outerrel,
outer_path, extra->sjinfo);
Assert(outer_path);
*************** sort_inner_and_outer_common(PlannerInfo
*** 1134,1139 ****
--- 1621,1630 ----
}
else if (jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary limitation, like above. */
+ if (grouped_inner)
+ return;
+
inner_path = (Path *) create_unique_path(root, innerrel,
inner_path, extra->sjinfo);
Assert(inner_path);
*************** sort_inner_and_outer_common(PlannerInfo
*** 1155,1167 ****
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! cheapest_partial_outer = (Path *) linitial(outerrel->partial_pathlist);
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
foreach(l, sort_infos)
--- 1646,1695 ----
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! if (grouped_outer)
! {
! if (REL_HAS_PARTIAL_GROUPED_PATHS(outerrel))
! cheapest_partial_outer = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
! else
! return;
! }
! else
! cheapest_partial_outer = (Path *)
! linitial(outerrel->partial_pathlist);
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else
+ {
+ Assert(innerrel->gpi != NULL);
+ inner_pathlist = innerrel->gpi->pathlist;
+ }
+
+ /*
+ * All the grouped paths should be unparameterized, so the
+ * function is overly stringent in the grouped_inner case, but
+ * still useful.
+ */
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
}
foreach(l, sort_infos)
*************** sort_inner_and_outer_common(PlannerInfo
*** 1176,1182 ****
* explicit sort step, so we needn't do so here.
*/
try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! si, jointype, extra, false);
/*
* If we have partial outer and parallel safe inner path then try
--- 1704,1712 ----
* explicit sort step, so we needn't do so here.
*/
try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! si, jointype, extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
/*
* If we have partial outer and parallel safe inner path then try
*************** sort_inner_and_outer_common(PlannerInfo
*** 1187,1193 ****
cheapest_partial_outer,
cheapest_safe_inner,
si, jointype, extra,
! true);
}
}
--- 1717,1724 ----
cheapest_partial_outer,
cheapest_safe_inner,
si, jointype, extra,
! true, grouped_outer, grouped_inner,
! do_aggregate);
}
}
*************** sort_inner_and_outer_common(PlannerInfo
*** 1204,1209 ****
--- 1735,1744 ----
* some sort key requirements). So, we consider truncations of the
* mergeclause list as well as the full list. (Ideally we'd consider all
* subsets of the mergeclause list, but that seems way too expensive.)
+ *
+ * grouped_outer - is outerpath grouped?
+ * grouped_inner - use grouped paths of innerrel?
+ * do_aggregate - apply (partial) aggregation to the output?
*/
static void
generate_mergejoin_paths(PlannerInfo *root,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1215,1221 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *trialsortkeys;
Path *cheapest_startup_inner;
--- 1750,1759 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
List *trialsortkeys;
Path *cheapest_startup_inner;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1274,1280 ****
&si,
jointype,
extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1812,1819 ----
&si,
jointype,
extra,
! is_partial,
! grouped_outer, grouped_inner, do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1330,1335 ****
--- 1869,1875 ----
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
+ List *inner_pathlist = NIL;
Path *innerpath;
MergeJoinSortInfo si_trial;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1338,1350 ****
si_trial.merge_pathkeys = si.merge_pathkeys;
si_trial.merge_clauses = NIL;
/*
* Look for an inner path ordered well enough for the first
! * 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
--- 1878,1895 ----
si_trial.merge_pathkeys = si.merge_pathkeys;
si_trial.merge_clauses = NIL;
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else if (innerrel->gpi != NULL)
+ inner_pathlist = innerrel->gpi->pathlist;
+
/*
* Look for an inner path ordered well enough for the first
! * 'sortkeycnt' si.innerkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1372,1385 ****
joinrel,
outerpath,
innerpath,
! &si,
jointype,
extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
--- 1917,1933 ----
joinrel,
outerpath,
innerpath,
! &si_trial,
jointype,
extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
!
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1414,1423 ****
joinrel,
outerpath,
innerpath,
! &si,
jointype,
extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1962,1973 ----
joinrel,
outerpath,
innerpath,
! &si_trial,
jointype,
extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** match_unsorted_outer_common(PlannerInfo
*** 1436,1449 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
bool useallclauses;
! Path *inner_cheapest_total = innerrel->cheapest_total_path;
Path *matpath = NULL;
ListCell *lc1;
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
--- 1986,2006 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
bool useallclauses;
! Path *inner_cheapest_total;
Path *matpath = NULL;
ListCell *lc1;
+ List *outer_pathlist;
+
+ /* At most one input path may be grouped. */
+ Assert(!(grouped_outer && grouped_inner));
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
*************** match_unsorted_outer_common(PlannerInfo
*** 1480,1492 ****
--- 2037,2074 ----
break;
}
+ if (grouped_outer)
+ {
+ if (REL_HAS_GROUPED_PATHS(outerrel))
+ outer_pathlist = outerrel->gpi->pathlist;
+ else
+ return;
+ }
+ else
+ outer_pathlist = outerrel->pathlist;
+
+ if (grouped_inner)
+ {
+ if (REL_HAS_GROUPED_PATHS(innerrel))
+ inner_cheapest_total = linitial(innerrel->gpi->pathlist);
+ else
+ return;
+ }
+ else
+ inner_cheapest_total = innerrel->cheapest_total_path;
+
/*
* If inner_cheapest_total is parameterized by the outer rel, ignore it;
* we will consider it below as a member of cheapest_parameterized_paths,
* but the other possibilities considered in this routine aren't usable.
*/
if (PATH_PARAM_BY_REL(inner_cheapest_total, outerrel))
+ {
+ /* Grouped path should not be parameterized at all. */
+ Assert(!grouped_inner);
+
inner_cheapest_total = NULL;
+ }
/*
* If we need to unique-ify the inner path, we will consider only the
*************** match_unsorted_outer_common(PlannerInfo
*** 1498,1513 ****
if (inner_cheapest_total == NULL)
return;
inner_cheapest_total = (Path *)
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK)
{
/*
* Consider materializing the cheapest inner path, unless
* enable_material is off or the path in question materializes its
* output anyway.
*/
if (enable_material && inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
--- 2080,2109 ----
if (inner_cheapest_total == NULL)
return;
+ /*
+ * TODO This is just a temporary limitation. Before lifting it, make
+ * sure that the UniquePath does emit GroupedVars. Also try to avoid
+ * the unique-ification if outer_path comes directly from AggPath
+ * (i.e. it's not grouped path combined with plain one) and the
+ * grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_inner)
+ return;
+
inner_cheapest_total = (Path *)
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK && !grouped_inner)
{
/*
* Consider materializing the cheapest inner path, unless
* enable_material is off or the path in question materializes its
* output anyway.
+ *
+ * TODO Verify that this is ok even if grouped_inner is true (at least
+ * make sure that mathpath does contain GroupedVars) and remove
+ * grouped_inner from the condition above.
*/
if (enable_material && inner_cheapest_total != NULL &&
!ExecMaterializesOutput(inner_cheapest_total->pathtype))
*************** match_unsorted_outer_common(PlannerInfo
*** 1515,1521 ****
create_material_path(innerrel, inner_cheapest_total);
}
! foreach(lc1, outerrel->pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
--- 2111,2117 ----
create_material_path(innerrel, inner_cheapest_total);
}
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
*************** match_unsorted_outer_common(PlannerInfo
*** 1536,1541 ****
--- 2132,2141 ----
if (outerpath != outerrel->cheapest_total_path)
continue;
+ /* TODO Temporary restriction, see above. */
+ if (grouped_outer)
+ continue;
+
outerpath = (Path *) create_unique_path(root, outerrel,
outerpath, extra->sjinfo);
Assert(outerpath);
*************** match_unsorted_outer_common(PlannerInfo
*** 1551,1556 ****
--- 2151,2160 ----
if (save_jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary restriction, see above. */
+ if (grouped_inner)
+ continue;
+
/*
* Consider nestloop join, but only with the unique-ified cheapest
* inner path
*************** match_unsorted_outer_common(PlannerInfo
*** 1562,1568 ****
merge_pathkeys,
jointype,
extra,
! false);
}
else if (nestjoinOK)
{
--- 2166,2173 ----
merge_pathkeys,
jointype,
extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer_common(PlannerInfo
*** 1574,1594 ****
*/
ListCell *lc2;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
! Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! false);
}
! /* Also consider materialized form of the cheapest inner path. */
if (matpath != NULL)
try_nestloop_path_common(root,
joinrel,
--- 2179,2208 ----
*/
ListCell *lc2;
! /*
! * There are no grouped paths in cheapest_parameterized_paths.
! */
! if (!grouped_inner)
{
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
! }
}
! /*
! * Also consider materialized form of the cheapest inner path.
! */
if (matpath != NULL)
try_nestloop_path_common(root,
joinrel,
*************** match_unsorted_outer_common(PlannerInfo
*** 1597,1603 ****
merge_pathkeys,
jointype,
extra,
! false);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 2211,2218 ----
merge_pathkeys,
jointype,
extra,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer_common(PlannerInfo
*** 1612,1618 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 2227,2234 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, grouped_outer, grouped_inner,
! do_aggregate);
}
/*
*************** match_unsorted_outer_common(PlannerInfo
*** 1623,1639 ****
* we handle extra_lateral_rels, since partial paths must not be
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
! bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 2239,2261 ----
* we handle extra_lateral_rels, since partial paths must not be
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
+ *
+ * grouped_inner must be false because we're not interested in nest loop
+ * joins with the grouped path on the inner side (i.e. repeated
+ * aggregation).
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
outerrel->partial_pathlist != NIL &&
! bms_is_empty(joinrel->lateral_relids) &&
! !grouped_inner)
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra,
! grouped_outer, do_aggregate);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer_common(PlannerInfo
*** 1653,1659 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
--- 2275,2282 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total,
! grouped_outer, do_aggregate);
}
}
*************** match_unsorted_outer_common(PlannerInfo
*** 1679,1684 ****
--- 2302,2309 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'grouped' indicates that the at least one relation in the join has been
+ * aggregated.
*/
static void
match_unsorted_outer(PlannerInfo *root,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1688,1695 ****
JoinType jointype,
JoinPathExtraData *extra)
{
match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra);
}
/*
--- 2313,2329 ----
JoinType jointype,
JoinPathExtraData *extra)
{
+ /* Plain (non-grouped) join. */
match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, false);
!
! /* Use all the supported strategies to generate grouped join. */
! match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, true, false, false);
! match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, true, false);
! match_unsorted_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, true);
}
/*
*************** consider_parallel_mergejoin(PlannerInfo
*** 1711,1722 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
/* generate merge join path for each partial outer path */
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
--- 2345,2380 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped_outer,
! bool do_aggregate)
{
ListCell *lc1;
+ List *outer_pathlist;
+ bool grouped = grouped_outer || do_aggregate;
+
+ if (!grouped || do_aggregate)
+ {
+ /*
+ * If creating grouped paths by explicit aggregation, the input paths
+ * must be plain.
+ */
+ outer_pathlist = outerrel->partial_pathlist;
+ }
+ else if (outerrel->gpi != NULL)
+ {
+ /*
+ * Only the outer paths are accepted as grouped when we try to combine
+ * grouped and plain ones. Grouped inner path implies repeated
+ * aggregation, which doesn't sound as a good idea.
+ */
+ outer_pathlist = outerrel->gpi->partial_pathlist;
+ }
+ else
+ return;
/* generate merge join path for each partial outer path */
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *merge_pathkeys;
*************** consider_parallel_mergejoin(PlannerInfo
*** 1727,1735 ****
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
! extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 2385,2394 ----
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, grouped_outer, false, do_aggregate);
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1750,1764 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
--- 2409,2446 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer, bool do_aggregate)
{
JoinType save_jointype = jointype;
+ List *outer_pathlist;
ListCell *lc1;
+ bool grouped = grouped_outer || do_aggregate;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! if (!grouped || do_aggregate)
! {
! /*
! * If creating grouped paths by explicit aggregation, the input paths
! * must be plain.
! */
! outer_pathlist = outerrel->partial_pathlist;
! }
! else if (outerrel->gpi != NULL)
! {
! /*
! * Only the outer paths are accepted as grouped when we try to combine
! * grouped and plain ones. Grouped inner path implies repeated
! * aggregation, which doesn't sound as a good idea.
! */
! outer_pathlist = outerrel->gpi->partial_pathlist;
! }
! else
! return;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1789,1795 ****
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
--- 2471,2477 ----
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER && !grouped)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1800,1806 ****
}
try_nestloop_path_common(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra, true);
}
}
}
--- 2482,2490 ----
}
try_nestloop_path_common(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! true, grouped_outer, false,
! do_aggregate);
}
}
}
*************** hash_inner_and_outer_common(PlannerInfo
*** 1816,1822 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! List *hashclauses)
{
JoinType save_jointype = jointype;
--- 2500,2509 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! List *hashclauses,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
*************** hash_inner_and_outer_common(PlannerInfo
*** 1832,1840 ****
/* Shouldn't be called w/o hashclauses. */
Assert(hashclauses != NIL);
! cheapest_startup_outer = outerrel->cheapest_startup_path;
! cheapest_total_outer = outerrel->cheapest_total_path;
! cheapest_total_inner = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 2519,2550 ----
/* Shouldn't be called w/o hashclauses. */
Assert(hashclauses != NIL);
! if (grouped_outer)
! {
! if (REL_HAS_GROUPED_PATHS(outerrel))
! {
! cheapest_total_outer = linitial(outerrel->gpi->pathlist);
! /* No parameterized grouped paths. */
! cheapest_startup_outer = NULL;
! }
! else
! return;
! }
! else
! {
! cheapest_total_outer = outerrel->cheapest_total_path;
! cheapest_startup_outer = outerrel->cheapest_startup_path;
! }
!
! if (grouped_inner)
! {
! if (REL_HAS_GROUPED_PATHS(innerrel))
! cheapest_total_inner = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! cheapest_total_inner = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** hash_inner_and_outer_common(PlannerInfo
*** 1849,1854 ****
--- 2559,2574 ----
/* Unique-ify if need be; we ignore parameterized possibilities */
if (jointype == JOIN_UNIQUE_OUTER)
{
+ /*
+ * TODO This is just a temporary limitation. Before lifting it, make
+ * sure that the UniquePath does emit GroupedVars. Also try to avoid
+ * the unique-ification if the outer path comes directly from AggPath
+ * (i.e. it's not grouped path combined with plain one) and the
+ * grouping keys guaranteed the uniqueness.
+ */
+ if (grouped_outer)
+ return;
+
cheapest_total_outer = (Path *)
create_unique_path(root, outerrel,
cheapest_total_outer, extra->sjinfo);
*************** hash_inner_and_outer_common(PlannerInfo
*** 1862,1872 ****
jointype,
extra,
false,
! false);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
{
cheapest_total_inner = (Path *)
create_unique_path(root, innerrel,
cheapest_total_inner, extra->sjinfo);
--- 2582,2598 ----
jointype,
extra,
false,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
{
+ /* TODO Temporary restriction, see above. */
+ if (grouped_inner)
+ return;
+
cheapest_total_inner = (Path *)
create_unique_path(root, innerrel,
cheapest_total_inner, extra->sjinfo);
*************** hash_inner_and_outer_common(PlannerInfo
*** 1880,1886 ****
jointype,
extra,
false,
! false);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
--- 2606,2614 ----
jointype,
extra,
false,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
*************** hash_inner_and_outer_common(PlannerInfo
*** 1892,1898 ****
jointype,
extra,
false,
! false);
}
else
{
--- 2620,2628 ----
jointype,
extra,
false,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
}
else
{
*************** hash_inner_and_outer_common(PlannerInfo
*** 1909,1914 ****
--- 2639,2645 ----
*/
ListCell *lc1;
ListCell *lc2;
+ List *outer_pathlist;
if (cheapest_startup_outer != NULL)
try_hashjoin_path_common(root,
*************** hash_inner_and_outer_common(PlannerInfo
*** 1919,1929 ****
jointype,
extra,
false,
! false);
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
Path *outerpath = (Path *) lfirst(lc1);
/*
* We cannot use an outer path that is parameterized by the inner
--- 2650,2667 ----
jointype,
extra,
false,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
! outer_pathlist = !grouped_outer ?
! outerrel->cheapest_parameterized_paths :
! outerrel->gpi->pathlist;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
+ List *inner_pathlist;
/*
* We cannot use an outer path that is parameterized by the inner
*************** hash_inner_and_outer_common(PlannerInfo
*** 1932,1938 ****
if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
Path *innerpath = (Path *) lfirst(lc2);
--- 2670,2680 ----
if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! inner_pathlist = !grouped_inner ?
! innerrel->cheapest_parameterized_paths :
! innerrel->gpi->pathlist;
!
! foreach(lc2, inner_pathlist)
{
Path *innerpath = (Path *) lfirst(lc2);
*************** hash_inner_and_outer_common(PlannerInfo
*** 1955,1961 ****
jointype,
extra,
false,
! false);
}
}
}
--- 2697,2705 ----
jointype,
extra,
false,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
}
}
}
*************** hash_inner_and_outer_common(PlannerInfo
*** 1966,1976 ****
* outer path will be partial, and therefore we won't be able to properly
* guarantee uniqueness. Similarly, we can't handle JOIN_FULL and
* JOIN_RIGHT, because they can produce false null extended rows. Also,
! * the resulting path must not be parameterized. We would be able to
! * support JOIN_FULL and JOIN_RIGHT for Parallel Hash, since in that case
! * we're back to a single hash table with a single set of match bits for
! * each batch, but that will require figuring out a deadlock-free way to
! * wait for the probe to finish.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
--- 2710,2716 ----
* outer path will be partial, and therefore we won't be able to properly
* guarantee uniqueness. Similarly, we can't handle JOIN_FULL and
* JOIN_RIGHT, because they can produce false null extended rows. Also,
! * the resulting path must not be parameterized.
*/
if (joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
*************** hash_inner_and_outer_common(PlannerInfo
*** 1983,1990 ****
Path *cheapest_partial_inner = NULL;
Path *cheapest_safe_inner = NULL;
! cheapest_partial_outer =
! (Path *) linitial(outerrel->partial_pathlist);
/*
* Can we use a partial inner plan too, so that we can build a shared
--- 2723,2741 ----
Path *cheapest_partial_inner = NULL;
Path *cheapest_safe_inner = NULL;
! if (grouped_outer)
! {
! if (REL_HAS_PARTIAL_GROUPED_PATHS(outerrel))
! {
! cheapest_partial_outer =
! (Path *) linitial(outerrel->gpi->partial_pathlist);
! }
! else
! return;
! }
! else
! cheapest_partial_outer =
! (Path *) linitial(outerrel->partial_pathlist);
/*
* Can we use a partial inner plan too, so that we can build a shared
*************** hash_inner_and_outer_common(PlannerInfo
*** 1992,2008 ****
*/
if (innerrel->partial_pathlist != NIL && enable_parallel_hash)
{
! cheapest_partial_inner =
! (Path *) linitial(innerrel->partial_pathlist);
try_hashjoin_path_common(root, joinrel,
cheapest_partial_outer,
cheapest_partial_inner,
hashclauses, jointype, extra,
true,
! true /* parallel_hash */ );
}
-
/*
* Normally, given that the joinrel is parallel-safe, the cheapest
* total inner path will also be parallel-safe, but if not, we'll have
--- 2743,2772 ----
*/
if (innerrel->partial_pathlist != NIL && enable_parallel_hash)
{
! if (grouped_inner)
! {
! if (REL_HAS_PARTIAL_GROUPED_PATHS(innerrel))
! {
! cheapest_partial_inner =
! (Path *) linitial(innerrel->gpi->partial_pathlist);
! }
! else
! return;
! }
! else
! cheapest_partial_inner =
! (Path *) linitial(innerrel->partial_pathlist);
!
try_hashjoin_path_common(root, joinrel,
cheapest_partial_outer,
cheapest_partial_inner,
hashclauses, jointype, extra,
true,
! true /* parallel_hash */ ,
! grouped_outer, grouped_inner,
! do_aggregate);
}
/*
* Normally, given that the joinrel is parallel-safe, the cheapest
* total inner path will also be parallel-safe, but if not, we'll have
*************** hash_inner_and_outer_common(PlannerInfo
*** 2013,2020 ****
if (cheapest_total_inner->parallel_safe)
cheapest_safe_inner = cheapest_total_inner;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
if (cheapest_safe_inner != NULL)
try_hashjoin_path_common(root, joinrel,
--- 2777,2792 ----
if (cheapest_total_inner->parallel_safe)
cheapest_safe_inner = cheapest_total_inner;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ inner_pathlist = !grouped_inner ?
+ innerrel->cheapest_parameterized_paths :
+ innerrel->gpi->pathlist;
+
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
if (cheapest_safe_inner != NULL)
try_hashjoin_path_common(root, joinrel,
*************** hash_inner_and_outer_common(PlannerInfo
*** 2022,2028 ****
cheapest_safe_inner,
hashclauses, jointype, extra,
true,
! false); /* parallel_hash */
}
}
--- 2794,2802 ----
cheapest_safe_inner,
hashclauses, jointype, extra,
true,
! false /* parallel_hash */ ,
! grouped_outer, grouped_inner,
! do_aggregate);
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 2045,2052 ****
JoinType jointype,
JoinPathExtraData *extra)
{
- bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses = NIL;
ListCell *l;
/*
--- 2819,2826 ----
JoinType jointype,
JoinPathExtraData *extra)
{
List *hashclauses = NIL;
+ bool isouterjoin = IS_OUTER_JOIN(jointype);
ListCell *l;
/*
*************** hash_inner_and_outer(PlannerInfo *root,
*** 2083,2090 ****
if (hashclauses == NIL)
return;
hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, hashclauses);
}
/*
--- 2857,2877 ----
if (hashclauses == NIL)
return;
+ /* Plain (non-grouped) join. */
hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, hashclauses,
! false, false, false);
!
! /* Use all the supported strategies to generate grouped join. */
! hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, hashclauses,
! true, false, false);
! hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, hashclauses,
! false, true, false);
! hash_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, hashclauses,
! false, false, true);
}
/*
*************** hash_inner_and_outer(PlannerInfo *root,
*** 2092,2098 ****
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
*
! * *mergejoin_allowed is normally set to true, but it is set to false if
* this is a right/full join and there are nonmergejoinable join clauses.
* The executor's mergejoin machinery cannot handle such cases, so we have
* to avoid generating a mergejoin plan. (Note that this flag does NOT
--- 2879,2885 ----
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
*
! * *mergejoin_allowed is normally set to TRUE, but it is set to FALSE if
* this is a right/full join and there are nonmergejoinable join clauses.
* The executor's mergejoin machinery cannot handle such cases, so we have
* to avoid generating a mergejoin plan. (Note that this flag does NOT
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 285f269..1823ee3
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** try_partition_wise_join(PlannerInfo *roo
*** 1405,1411 ****
(List *) adjust_appendrel_attrs(root,
(Node *) parent_restrictlist,
nappinfos, appinfos);
- pfree(appinfos);
child_joinrel = joinrel->part_rels[cnt_parts];
if (!child_joinrel)
--- 1405,1410 ----
*************** try_partition_wise_join(PlannerInfo *roo
*** 1415,1421 ****
--- 1414,1428 ----
child_sjinfo,
child_sjinfo->jointype);
joinrel->part_rels[cnt_parts] = child_joinrel;
+
+ /*
+ * If the parent join can be grouped, so can be its children.
+ */
+ if (joinrel->gpi != NULL)
+ build_child_rel_gpi(root, child_joinrel, joinrel, nappinfos,
+ appinfos);
}
+ pfree(appinfos);
Assert(bms_equal(child_joinrel->relids, child_joinrelids));
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index 1a0d3a8..b11afd9
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** use_physical_tlist(PlannerInfo *root, Pa
*** 809,814 ****
--- 809,820 ----
return false;
/*
+ * Grouped relation's target list contains GroupedVars.
+ */
+ if (rel->gpi != NULL)
+ return false;
+
+ /*
* If a bitmap scan's tlist is empty, keep it as-is. This may allow the
* executor to skip heap page fetches, and in any case, the benefit of
* using a physical tlist instead would be minimal.
*************** create_projection_plan(PlannerInfo *root
*** 1587,1594 ****
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
--- 1593,1601 ----
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (!best_path->force_result &&
! (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist)))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 0aaac38..c3528fc
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
*************** append_startup_cost_compare(const void *
*** 1359,1369 ****
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
List *subpaths,
List *pathkeys,
Relids required_outer,
--- 1359,1371 ----
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode. target can be supplied by caller. If NULL is passed, the field
! * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** create_merge_append_path(PlannerInfo *ro
*** 1376,1382 ****
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1378,1384 ----
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = target ? target : rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** calc_non_nestloop_required_outer(Path *o
*** 2179,2184 ****
--- 2181,2187 ----
* 'restrict_clauses' are the RestrictInfo nodes to apply at the join
* 'pathkeys' are the path keys of the new join path
* 'required_outer' is the set of required outer rels
+ * 'target' can be passed to override that of joinrel.
*
* Returns the resulting path node.
*/
*************** create_nestloop_path(PlannerInfo *root,
*** 2192,2198 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
--- 2195,2202 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
*************** create_nestloop_path(PlannerInfo *root,
*** 2225,2231 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2229,2235 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target == NULL ? joinrel->reltarget : target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2283,2295 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2287,2301 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2335,2340 ****
--- 2341,2347 ----
* 'required_outer' is the set of required outer rels
* 'hashclauses' are the RestrictInfo nodes to use as hash clauses
* (this should be a subset of the restrict_clauses list)
+ * 'target' can be passed to override that of joinrel.
*/
HashPath *
create_hashjoin_path(PlannerInfo *root,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2347,2359 ****
bool parallel_hash,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2354,2368 ----
bool parallel_hash,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index e4c2313..3fa9261
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
*************** build_join_rel(PlannerInfo *root,
*** 537,542 ****
--- 537,543 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 590,595 ****
--- 591,655 ----
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
+ * If grouping is applicable at relation level, try to build grouped
+ * target.
+ */
+ if (root->grouped_var_list != NIL)
+ {
+ /*
+ * Try to create a grouping target.
+ */
+ prepare_rel_for_grouping(root, joinrel);
+
+ /*
+ * If the relation appears to be eligible for grouping, joinrel->gpi
+ * has been initialized. Compute width of the grouping target.
+ */
+ if (joinrel->gpi != NULL)
+ {
+ ListCell *lc;
+ int32 width = 0;
+
+ Assert(joinrel->gpi->target != NULL);
+ foreach(lc, joinrel->gpi->target->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, Var))
+ {
+ Var *var = castNode(Var, expr);
+ RelOptInfo *rel;
+ int ndx;
+
+ /*
+ * If set_rel_width() managed to set the width, it should
+ * be cached in the underlying base relation. Otherwise we
+ * can hardly find out more w/o inadequate effort.
+ */
+ rel = find_base_rel(root, var->varno);
+ ndx = var->varattno - rel->min_attr;
+ width += rel->attr_widths[ndx];
+ }
+ else if (IsA(expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, expr);
+ GroupedVarInfo *gvinfo = find_grouped_var_info(root,
+ gvar);
+
+ width += gvinfo->gv_width;
+ }
+ else
+
+ /*
+ * No other kind of expression should appear in a grouped
+ * target.
+ */
+ Assert(false);
+ }
+ }
+ }
+
+ /*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
* now we can finish computing that. This is much like the computation of
*************** build_child_join_rel(PlannerInfo *root,
*** 711,716 ****
--- 771,777 ----
joinrel->cheapest_parameterized_paths = NIL;
joinrel->direct_lateral_relids = NULL;
joinrel->lateral_relids = NULL;
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_child_join_rel(PlannerInfo *root,
*** 759,765 ****
(Node *) parent_joinrel->joininfo,
nappinfos,
appinfos);
- pfree(appinfos);
/*
* Lateral relids referred in child join will be same as that referred in
--- 820,825 ----
*************** build_child_join_rel(PlannerInfo *root,
*** 795,800 ****
--- 855,862 ----
/* Add the relation to the PlannerInfo. */
add_join_rel(root, joinrel);
+ pfree(appinfos);
+
return joinrel;
}
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index b11bd47..bbcfc7f
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern AppendPath *create_append_path(Re
*** 73,78 ****
--- 73,79 ----
List *partitioned_rels, double rows);
extern MergeAppendPath *create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** extern NestPath *create_nestloop_path(Pl
*** 132,138 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 133,140 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern MergePath *create_mergejoin_path(
*** 146,152 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 148,155 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern HashPath *create_hashjoin_path(Pl
*** 158,164 ****
bool parallel_hash,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
--- 161,168 ----
bool parallel_hash,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
*************** extern RelOptInfo *build_child_join_rel(
*** 315,320 ****
--- 319,327 ----
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
+ extern void build_child_rel_gpi(PlannerInfo *root, RelOptInfo *child,
+ RelOptInfo *parent, int nappinfos,
+ AppendRelInfo **appinfos);
extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
extern void build_child_rel_gpi(PlannerInfo *root, RelOptInfo *child,
RelOptInfo *parent, int nappinfos,
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index ...e3071de
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
***************
*** 0 ****
--- 1,259 ----
+ BEGIN;
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ CREATE INDEX ON agg_pushdown_child1(parent);
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ ANALYZE;
+ SET enable_agg_pushdown TO on;
+ -- The supposed advantage of pushing aggregation down below joins is that
+ -- joins will then receive less input tuples. Let's stress this difference.
+ SET cpu_tuple_cost TO 0.1;
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (8 rows)
+
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (12 rows)
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ----------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+ -> Hash
+ -> Seq Scan on agg_pushdown_parent p
+ (13 rows)
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (14 rows)
+
+ -- Generic grouping expression.
+ -- Make any processing of the grouping expression expensive. If the final
+ -- relation output needs to be sorted before GroupAggregate is applied, the
+ -- (p.i / 2) expression is also evaluated by the Sort node. Thus the plan that
+ -- uses the partial aggregation is preferrable because it emits the value of
+ -- the exprssion and no node above has to evaluate it again.
+ SET cpu_operator_cost TO 0.1;
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Result
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (15 rows)
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+ SET enable_agg_pushdown TO on;
+ SET cpu_tuple_cost TO 0.1;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (14 rows)
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ -- TODO Force the Hash Join node to stay below Gather.
+ -- EXPLAIN (COSTS off)
+ -- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ -- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ -- c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (16 rows)
+
+ SET cpu_operator_cost TO 0.1;
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Gather
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Result
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (17 rows)
+
+ ROLLBACK;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
new file mode 100644
index e224977..cf52728
*** a/src/test/regress/parallel_schedule
--- b/src/test/regress/parallel_schedule
*************** test: rules psql_crosstab amutils
*** 98,103 ****
--- 98,106 ----
test: select_parallel
test: write_parallel
+ # this one runs parallel workers too
+ test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
new file mode 100644
index 9fc5f1a..ca0bf76
*** a/src/test/regress/serial_schedule
--- b/src/test/regress/serial_schedule
*************** test: stats_ext
*** 136,141 ****
--- 136,142 ----
test: rules
test: psql_crosstab
test: select_parallel
+ test: agg_pushdown
test: write_parallel
test: publication
test: subscription
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index ...e6e40cb
*** a/src/test/regress/sql/agg_pushdown.sql
--- b/src/test/regress/sql/agg_pushdown.sql
***************
*** 0 ****
--- 1,138 ----
+ BEGIN;
+
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ CREATE INDEX ON agg_pushdown_child1(parent);
+
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ ANALYZE;
+
+ SET enable_agg_pushdown TO on;
+
+ -- The supposed advantage of pushing aggregation down below joins is that
+ -- joins will then receive less input tuples. Let's stress this difference.
+ SET cpu_tuple_cost TO 0.1;
+
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- Generic grouping expression.
+
+ -- Make any processing of the grouping expression expensive. If the final
+ -- relation output needs to be sorted before GroupAggregate is applied, the
+ -- (p.i / 2) expression is also evaluated by the Sort node. Thus the plan that
+ -- uses the partial aggregation is preferrable because it emits the value of
+ -- the exprssion and no node above has to evaluate it again.
+ SET cpu_operator_cost TO 0.1;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+
+ SET enable_agg_pushdown TO on;
+ SET cpu_tuple_cost TO 0.1;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ -- TODO Force the Hash Join node to stay below Gather.
+ -- EXPLAIN (COSTS off)
+ -- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ -- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ -- c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET cpu_operator_cost TO 0.1;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ ROLLBACK;
+
agg_pushdown_v5/08_index.diff
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 36e038e..045dd21
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 738,744 ****
/*
* Consider index scans.
*/
! create_index_paths(root, rel);
/* Consider TID scans */
create_tidscan_paths(root, rel);
--- 738,744 ----
/*
* Consider index scans.
*/
! create_index_paths(root, rel, agg_info);
/* Consider TID scans */
create_tidscan_paths(root, rel);
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index c8bd820..5c39ef6
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** typedef struct
*** 78,83 ****
--- 79,93 ----
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
+ /*
+ * A set of lists containing index paths to which specific kind of aggregation
+ * was applied.
+ */
+ typedef struct GroupedIndexPaths
+ {
+ List *paths;
+ List *paths_partial;
+ } GroupedIndexPaths;
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
*************** static bool eclass_already_used(Equivale
*** 107,123 ****
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses);
static Path *choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel,
List *paths);
static int path_usage_comparator(const void *a, const void *b);
--- 117,139 ----
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, PartialAggInfo *agg_info);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! PartialAggInfo *agg_info,
! GroupedIndexPaths *agg_sorted,
! GroupedIndexPaths *agg_hashed);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses,
! PartialAggInfo *agg_info,
! GroupedIndexPaths *agg_hashed);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses,
! PartialAggInfo *agg_info);
static Path *choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel,
List *paths);
static int path_usage_comparator(const void *a, const void *b);
*************** static Const *string_to_const(const char
*** 227,235 ****
* index quals ... but for now, it doesn't seem worth troubling over.
* In particular, comments below about "unparameterized" paths should be read
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel)
{
List *indexpaths;
List *bitindexpaths;
--- 243,254 ----
* index quals ... but for now, it doesn't seem worth troubling over.
* In particular, comments below about "unparameterized" paths should be read
* as meaning "unparameterized so far as the indexquals are concerned".
+ *
+ * If agg_info is passed, grouped paths are generated too.
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! PartialAggInfo *agg_info)
{
List *indexpaths;
List *bitindexpaths;
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 293,300 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
! agg_info);
/*
* Identify the join clauses that can match the index. For the moment
*************** create_index_paths(PlannerInfo *root, Re
*** 310,326 ****
/*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
indexpaths = generate_bitmap_or_paths(root, rel,
! rel->baserestrictinfo, NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
* Likewise, generate BitmapOrPaths for any suitable OR-clauses present in
* the joinclause list. Add these to bitjoinpaths.
*/
indexpaths = generate_bitmap_or_paths(root, rel,
! joinorclauses, rel->baserestrictinfo);
bitjoinpaths = list_concat(bitjoinpaths, indexpaths);
/*
--- 329,351 ----
/*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
+ *
+ * Include the grouped path.
*/
indexpaths = generate_bitmap_or_paths(root, rel,
! rel->baserestrictinfo, NIL,
! agg_info);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
* Likewise, generate BitmapOrPaths for any suitable OR-clauses present in
* the joinclause list. Add these to bitjoinpaths.
+ *
+ * Do not include the grouped paths, as these cab be parameterized.
*/
indexpaths = generate_bitmap_or_paths(root, rel,
! joinorclauses, rel->baserestrictinfo,
! NULL);
bitjoinpaths = list_concat(bitjoinpaths, indexpaths);
/*
*************** get_join_index_paths(PlannerInfo *root,
*** 667,673 ****
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
--- 692,698 ----
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths, NULL);
/*
* Remember we considered paths for this set of relids. We use lcons not
*************** bms_equal_any(Relids relids, List *relid
*** 717,723 ****
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
--- 742,747 ----
*************** bms_equal_any(Relids relids, List *relid
*** 736,765 ****
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths)
{
List *indexpaths;
bool skip_nonnative_saop = false;
bool skip_lower_saop = false;
ListCell *lc;
/*
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
* clauses only if the index AM supports them natively, and skip any such
* clauses for index columns after the first (so that we produce ordered
* paths if possible).
*/
indexpaths = build_index_paths(root, rel,
index, clauses,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
* that supports them, then try again including those clauses. This will
* produce paths with more selectivity but no ordering.
*/
if (skip_lower_saop)
{
--- 760,826 ----
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, PartialAggInfo *agg_info)
{
List *indexpaths;
+ GroupedIndexPaths agg_sorted;
+ GroupedIndexPaths *agg_sorted_p = NULL;
+ GroupedIndexPaths agg_hashed;
+ GroupedIndexPaths *agg_hashed_p = NULL;
bool skip_nonnative_saop = false;
bool skip_lower_saop = false;
ListCell *lc;
/*
+ * Set all lists of grouped paths to NIL.
+ */
+ MemSet(&agg_sorted, 0, sizeof(GroupedIndexPaths));
+ MemSet(&agg_hashed, 0, sizeof(GroupedIndexPaths));
+
+ /*
+ * If grouping at relation level might be useful for the current query,
+ * prepare for collection of the grouped paths.
+ */
+ if (agg_info != NULL)
+ {
+ /*
+ * AGG_HASHED is always possible.
+ */
+ agg_hashed_p = &agg_hashed;
+
+ /*
+ * Can this index provide input for AGG_SORTED aggregation?
+ */
+ if (index->amhasgettuple)
+ agg_sorted_p = &agg_sorted;
+ }
+
+ /*
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
* clauses only if the index AM supports them natively, and skip any such
* clauses for index columns after the first (so that we produce ordered
* paths if possible).
+ *
+ * These paths are good candidates for AGG_SORTED, so pass the output
+ * lists for this strategy. AGG_HASHED should be applied to paths with no
+ * pathkeys.
*/
indexpaths = build_index_paths(root, rel,
index, clauses,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop,
! agg_info,
! agg_sorted_p, agg_hashed_p);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
* that supports them, then try again including those clauses. This will
* produce paths with more selectivity but no ordering.
+ *
+ * As for the grouping paths, only AGG_HASHED is considered due to the
+ * missing ordering.
*/
if (skip_lower_saop)
{
*************** get_index_paths(PlannerInfo *root, RelOp
*** 769,775 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL));
}
/*
--- 830,838 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL,
! agg_info,
! NULL, agg_hashed_p));
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 801,806 ****
--- 864,872 ----
* If there were ScalarArrayOpExpr clauses that the index can't handle
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
+ *
+ * As for grouping, only AGG_HASHED is possible here. Again, because
+ * there's no ordering.
*/
if (skip_nonnative_saop)
{
*************** get_index_paths(PlannerInfo *root, RelOp
*** 809,817 ****
false,
ST_BITMAPSCAN,
NULL,
! NULL);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
/*
--- 875,909 ----
false,
ST_BITMAPSCAN,
NULL,
! NULL,
! agg_info,
! NULL, agg_hashed_p);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
+
+ /*
+ * If we collected the grouped path, process them now.
+ */
+ if (agg_info != NULL)
+ {
+ List *paths;
+ AggPath *agg_path;
+
+ paths = list_concat(agg_sorted.paths, agg_hashed.paths);
+ foreach(lc, paths)
+ {
+ agg_path = lfirst_node(AggPath, lc);
+ add_path(rel, (Path *) agg_path, true);
+ }
+
+ paths = list_concat(agg_sorted.paths_partial,
+ agg_hashed.paths_partial);
+ foreach(lc, paths)
+ {
+ agg_path = lfirst_node(AggPath, lc);
+ add_partial_path(rel, (Path *) agg_path, true);
+ }
+ }
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 847,859 ****
* NULL, we do not ignore non-first ScalarArrayOpExpr clauses, but they will
* result in considering the scan's output to be unordered.
*
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'clauses' is the collection of indexable clauses (RestrictInfo nodes)
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
! * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
--- 939,959 ----
* NULL, we do not ignore non-first ScalarArrayOpExpr clauses, but they will
* result in considering the scan's output to be unordered.
*
+ * If 'agg_info' is passed, 'agg_sorted' and / or 'agg_hashed' must be passed
+ * too. In that case AGG_SORTED and / or AGG_HASHED aggregation is applied to
+ * the index path (as long as the index path is appropriate) and the resulting
+ * grouped path is stored here.
+ *
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'clauses' is the collection of indexable clauses (RestrictInfo nodes)
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
! * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
! * 'agg_info' is information needed to aggregate the join scan.
! * 'agg_sorted' is an output lists for AGG_SORTED paths.
! * 'agg_hashed' is an output lists for AGG_HASHED paths.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
*************** build_index_paths(PlannerInfo *root, Rel
*** 861,867 ****
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop)
{
List *result = NIL;
IndexPath *ipath;
--- 961,970 ----
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! PartialAggInfo *agg_info,
! GroupedIndexPaths *agg_sorted,
! GroupedIndexPaths *agg_hashed)
{
List *result = NIL;
IndexPath *ipath;
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 981,999 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool include_grouped;
+ bool can_agg_sorted,
+ can_agg_hashed;
+ AggPath *agg_path;
+
+ /*
+ * Make some of the following tests simpler.
+ */
+ include_grouped = agg_sorted != NULL || agg_hashed != NULL;
+
+ /* Check that the agg_* arguments are consistent. */
+ Assert((agg_info == NULL && !include_grouped) ||
+ (agg_info != NULL && include_grouped));
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 1031,1037 ****
--- 1147,1158 ----
* in the current clauses, OR the index ordering is potentially useful for
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
+ *
+ * This is where grouped path start to be considered.
*/
+ can_agg_sorted = true;
+ can_agg_hashed = true;
+
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1055 ****
--- 1169,1212 ----
outer_relids,
loop_count,
false);
+
result = lappend(result, ipath);
+ if (include_grouped && outer_relids == NULL &&
+ rel->gpi != NULL && rel->gpi->target != NULL)
+ {
+ /*
+ * Try to create the grouped paths if caller is interested in
+ * them.
+ */
+ if (agg_sorted != NULL && useful_pathkeys != NIL)
+ {
+ agg_path = create_partial_agg_sorted_path(root,
+ (Path *) ipath,
+ true,
+ agg_info,
+ ipath->path.rows);
+
+ if (agg_path != NULL)
+ agg_sorted->paths = lappend(agg_sorted->paths, agg_path);
+ else
+ can_agg_sorted = false;
+ }
+
+ if (agg_hashed != NULL)
+ {
+ agg_path = create_partial_agg_hashed_path(root,
+ (Path *) ipath,
+ agg_info,
+ ipath->path.rows);
+
+ if (agg_path != NULL)
+ agg_hashed->paths = lappend(agg_hashed->paths, agg_path);
+ else
+ can_agg_hashed = false;
+ }
+ }
+
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
--- 1234,1285 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
+ {
add_partial_path(rel, (Path *) ipath, false);
+
+ if (include_grouped && outer_relids == NULL &&
+ rel->gpi != NULL && rel->gpi->target != NULL)
+ {
+ if (agg_sorted != NULL && useful_pathkeys != NIL &&
+ can_agg_sorted)
+ {
+ /*
+ * No need to check the pathkeys again.
+ */
+ agg_path = create_partial_agg_sorted_path(root,
+ (Path *) ipath,
+ false,
+ agg_info,
+ ipath->path.rows);
+
+ /*
+ * If create_agg_sorted_path succeeded once, it should
+ * always do.
+ */
+ Assert(agg_path != NULL);
+
+ agg_sorted->paths_partial =
+ lappend(agg_sorted->paths_partial, agg_path);
+ }
+
+ if (agg_hashed != NULL && can_agg_hashed)
+ {
+ agg_path = create_partial_agg_hashed_path(root,
+ (Path *) ipath,
+ agg_info,
+ ipath->path.rows);
+
+ /*
+ * If create_agg_hashed_path succeeded once, it should
+ * always do.
+ */
+ Assert(agg_path != NULL);
+
+ agg_hashed->paths_partial =
+ lappend(agg_hashed->paths_partial, agg_path);
+ }
+ }
+ }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1112 ****
--- 1307,1345 ----
outer_relids,
loop_count,
false);
+
result = lappend(result, ipath);
+ if (include_grouped && outer_relids == NULL &&
+ rel->gpi != NULL && rel->gpi->target != NULL)
+ {
+ /*
+ * As the input set ordering does not matter to AGG_HASHED,
+ * only AGG_SORTED makes sense here. (The AGG_HASHED path we'd
+ * create here should already exist.)
+ *
+ * The existing value of can_agg_sorted is not up-to-date for
+ * the new pathkeys.
+ */
+ can_agg_sorted = true;
+
+ if (agg_sorted != NULL)
+ {
+ /* pathkeys are new, so check them. */
+ agg_path = create_partial_agg_sorted_path(root,
+ (Path *) ipath,
+ true,
+ agg_info,
+ ipath->path.rows);
+
+ if (agg_path != NULL)
+ agg_sorted->paths = lappend(agg_sorted->paths,
+ agg_path);
+ else
+ can_agg_sorted = false;
+ }
+ }
+
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
--- 1362,1390 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
+ {
add_partial_path(rel, (Path *) ipath, false);
+
+ if (include_grouped && outer_relids == NULL &&
+ rel->gpi != NULL && rel->gpi->target != NULL)
+ {
+ if (agg_sorted != NULL && can_agg_sorted)
+ {
+ /*
+ * The non-partial path above should have been
+ * created, so no need to check pathkeys.
+ */
+ agg_path = create_partial_agg_sorted_path(root,
+ (Path *) ipath,
+ false,
+ agg_info,
+ ipath->path.rows);
+ Assert(agg_path != NULL);
+ agg_sorted->paths_partial =
+ lappend(agg_sorted->paths_partial, agg_path);
+ }
+ }
+ }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1164,1173 ****
* 'rel' is the relation for which we want to generate index paths
* 'clauses' is the current list of clauses (RestrictInfo nodes)
* 'other_clauses' is the list of additional upper-level clauses
*/
static List *
build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses)
{
List *result = NIL;
List *all_clauses = NIL; /* not computed till needed */
--- 1419,1430 ----
* 'rel' is the relation for which we want to generate index paths
* 'clauses' is the current list of clauses (RestrictInfo nodes)
* 'other_clauses' is the list of additional upper-level clauses
+ * 'agg_info' indicates that grouped paths should be added to 'agg_hashed'.
*/
static List *
build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses,
! PartialAggInfo *agg_info, GroupedIndexPaths *agg_hashed)
{
List *result = NIL;
List *all_clauses = NIL; /* not computed till needed */
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1244,1250 ****
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL);
result = list_concat(result, indexpaths);
}
--- 1501,1508 ----
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL,
! agg_info, NULL, agg_hashed);
result = list_concat(result, indexpaths);
}
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1260,1273 ****
* other_clauses is a list of additional clauses that can be assumed true
* for the purpose of generating indexquals, but are not to be searched for
* ORs. (See build_paths_for_OR() for motivation.)
*/
static List *
generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses)
{
List *result = NIL;
List *all_clauses;
ListCell *lc;
/*
* We can use both the current and other clauses as context for
--- 1518,1545 ----
* other_clauses is a list of additional clauses that can be assumed true
* for the purpose of generating indexquals, but are not to be searched for
* ORs. (See build_paths_for_OR() for motivation.)
+ *
+ * If agg_info is not NULL, consider grouped paths in addition.
*/
static List *
generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
! List *clauses, List *other_clauses,
! PartialAggInfo *agg_info)
{
List *result = NIL;
List *all_clauses;
ListCell *lc;
+ GroupedIndexPaths agg_hashed;
+ GroupedIndexPaths *agg_hashed_p = NULL;
+
+ /*
+ * Prepare the output for grouped paths.
+ *
+ * Bitmap scan is not supposed to deliver the tuples sorted, so only
+ * consider AGG_HASHED strategy.
+ */
+ if (agg_info != NULL)
+ agg_hashed_p = &agg_hashed;
/*
* We can use both the current and other clauses as context for
*************** generate_bitmap_or_paths(PlannerInfo *ro
*** 1287,1292 ****
--- 1559,1572 ----
continue;
/*
+ * Free what the previous iteration might have allocated.
+ */
+ list_free(agg_hashed.paths);
+ agg_hashed.paths = NIL;
+ list_free(agg_hashed.paths_partial);
+ agg_hashed.paths_partial = NIL;
+
+ /*
* We must be able to match at least one index to each of the arms of
* the OR, else we can't use it.
*/
*************** generate_bitmap_or_paths(PlannerInfo *ro
*** 1303,1315 ****
indlist = build_paths_for_OR(root, rel,
andargs,
! all_clauses);
/* Recurse in case there are sub-ORs */
indlist = list_concat(indlist,
generate_bitmap_or_paths(root, rel,
andargs,
! all_clauses));
}
else
{
--- 1583,1597 ----
indlist = build_paths_for_OR(root, rel,
andargs,
! all_clauses,
! agg_info, agg_hashed_p);
/* Recurse in case there are sub-ORs */
indlist = list_concat(indlist,
generate_bitmap_or_paths(root, rel,
andargs,
! all_clauses,
! agg_info));
}
else
{
*************** generate_bitmap_or_paths(PlannerInfo *ro
*** 1321,1327 ****
indlist = build_paths_for_OR(root, rel,
orargs,
! all_clauses);
}
/*
--- 1603,1610 ----
indlist = build_paths_for_OR(root, rel,
orargs,
! all_clauses,
! agg_info, agg_hashed_p);
}
/*
*************** generate_bitmap_or_paths(PlannerInfo *ro
*** 1351,1356 ****
--- 1634,1659 ----
bitmapqual = (Path *) create_bitmap_or_path(root, rel, pathlist);
result = lappend(result, bitmapqual);
}
+
+ /*
+ * Process the grouped paths.
+ */
+ if (pathlist != NIL && agg_info != NULL)
+ {
+ AggPath *agg_path;
+
+ foreach(lc, agg_hashed_p->paths)
+ {
+ agg_path = lfirst_node(AggPath, lc);
+ add_path(rel, (Path *) agg_path, true);
+ }
+
+ foreach(lc, agg_hashed_p->paths_partial)
+ {
+ agg_path = lfirst_node(AggPath, lc);
+ add_partial_path(rel, (Path *) agg_path, true);
+ }
+ }
}
return result;
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 7815138..b44b8a8
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void debug_print_rel(PlannerInfo
*** 75,81 ****
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
--- 75,82 ----
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! PartialAggInfo *agg_info);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index e3071de..cac70a0
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
*************** AS c1 ON c1.parent = p.i GROUP BY p.i;
*** 40,45 ****
--- 40,64 ----
Index Cond: (i = c1.parent)
(8 rows)
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (8 rows)
+
+ SET enable_seqscan TO on;
-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
-- and partially aggregate the result.
SET enable_nestloop TO on;
*************** AS c1 ON c1.parent = p.i GROUP BY p.i;
*** 171,176 ****
--- 190,214 ----
Index Cond: (i = c1.parent)
(10 rows)
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ SET enable_seqscan TO on;
SET enable_nestloop TO on;
SET enable_hashjoin TO off;
SET enable_mergejoin TO off;
*************** c2.parent = p.i WHERE c1.j = c2.k GROUP
*** 198,204 ****
SET enable_nestloop TO off;
SET enable_hashjoin TO on;
! -- TODO Force the Hash Join node to stay below Gather.
-- EXPLAIN (COSTS off)
-- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
-- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
--- 236,244 ----
SET enable_nestloop TO off;
SET enable_hashjoin TO on;
! -- TODO Force the Hash Join node to stay below Gather. Does each parallel
! -- worker still use a separate hash table? Try again once this limitation has
! -- been fixed.
-- EXPLAIN (COSTS off)
-- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
-- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index e6e40cb..f5b9b24
*** a/src/test/regress/sql/agg_pushdown.sql
--- b/src/test/regress/sql/agg_pushdown.sql
*************** EXPLAIN (COSTS off)
*** 40,45 ****
--- 40,53 ----
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ SET enable_seqscan TO on;
+
-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
-- and partially aggregate the result.
SET enable_nestloop TO on;
*************** EXPLAIN (COSTS off)
*** 100,105 ****
--- 108,119 ----
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ SET enable_seqscan TO on;
+
SET enable_nestloop TO on;
SET enable_hashjoin TO off;
SET enable_mergejoin TO off;
*************** c2.parent = p.i WHERE c1.j = c2.k GROUP
*** 112,118 ****
SET enable_nestloop TO off;
SET enable_hashjoin TO on;
! -- TODO Force the Hash Join node to stay below Gather.
-- EXPLAIN (COSTS off)
-- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
-- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
--- 126,135 ----
SET enable_nestloop TO off;
SET enable_hashjoin TO on;
! -- TODO Force the Hash Join node to stay below Gather. Does each parallel
! -- worker still use a separate hash table? Try again once this limitation has
! -- been fixed.
!
-- EXPLAIN (COSTS off)
-- SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
-- agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
agg_pushdown_v5/09_avoid_agg_finalization.diff
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 85e8ea3..b1dd21e
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyAggref(const Aggref *from)
*** 1362,1367 ****
--- 1362,1368 ----
COPY_SCALAR_FIELD(inputcollid);
COPY_SCALAR_FIELD(aggtranstype);
COPY_SCALAR_FIELD(aggcombinefn);
+ COPY_SCALAR_FIELD(aggfinalfn);
COPY_NODE_FIELD(aggargtypes);
COPY_NODE_FIELD(aggdirectargs);
COPY_NODE_FIELD(args);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index d70d210..1ccaee7
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outAggref(StringInfo str, const Aggref
*** 1137,1142 ****
--- 1137,1143 ----
WRITE_OID_FIELD(inputcollid);
WRITE_OID_FIELD(aggtranstype);
WRITE_OID_FIELD(aggcombinefn);
+ WRITE_OID_FIELD(aggfinalfn);
WRITE_NODE_FIELD(aggargtypes);
WRITE_NODE_FIELD(aggdirectargs);
WRITE_NODE_FIELD(args);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index f02a963..ed5686a
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readAggref(void)
*** 601,606 ****
--- 601,607 ----
READ_OID_FIELD(inputcollid);
READ_OID_FIELD(aggtranstype);
READ_OID_FIELD(aggcombinefn);
+ READ_OID_FIELD(aggfinalfn);
READ_NODE_FIELD(aggargtypes);
READ_NODE_FIELD(aggdirectargs);
READ_NODE_FIELD(args);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 045dd21..f95c913
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 706,711 ****
--- 706,714 ----
/* Consider sequential scan, both plain and grouped. */
seq_path = create_seqscan_path(root, rel, required_outer, 0);
+ /* Try to compute unique keys. */
+ make_uniquekeys(root, seq_path);
+
add_path(rel, seq_path, false);
if (rel->gpi != NULL && rel->gpi->target != NULL &&
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 736,742 ****
create_plain_partial_paths(root, rel);
/*
! * Consider index scans.
*/
create_index_paths(root, rel, agg_info);
--- 739,746 ----
create_plain_partial_paths(root, rel);
/*
! * Consider index scans, possibly including the grouped and grouped
! * partial paths.
*/
create_index_paths(root, rel, agg_info);
*************** create_grouped_path(PlannerInfo *root, R
*** 847,858 ****
if (aggstrategy == AGG_HASHED)
agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
agg_info,
! subpath->rows);
else if (aggstrategy == AGG_SORTED)
agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
true,
agg_info,
! subpath->rows);
else
elog(ERROR, "unexpected strategy %d", aggstrategy);
--- 851,864 ----
if (aggstrategy == AGG_HASHED)
agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
agg_info,
! subpath->rows,
! partial);
else if (aggstrategy == AGG_SORTED)
agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
true,
agg_info,
! subpath->rows,
! partial);
else
elog(ERROR, "unexpected strategy %d", aggstrategy);
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1864,1869 ****
--- 1870,1879 ----
false, partitioned_rels, -1);
/* pathtarget will produce the grouped relation.. */
path->pathtarget = rel->gpi->target;
+
+ /* Try to compute unique keys. */
+ make_uniquekeys(root, path);
+
add_path(rel, path, true);
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 2042,2047 ****
--- 2052,2060 ----
NULL,
partitioned_rels);
+ /* Try to compute unique keys. */
+ make_uniquekeys(root, path);
+
/* pathtarget will produce the grouped relation.. */
if (grouped)
{
*************** generate_mergeappend_paths(PlannerInfo *
*** 2060,2065 ****
--- 2073,2079 ----
pathkeys,
NULL,
partitioned_rels);
+ make_uniquekeys(root, path);
if (grouped)
path->pathtarget = rel->gpi->target;
add_path(rel, path, grouped);
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 5c39ef6..65db91c
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,38 ****
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
- #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
--- 32,37 ----
*************** get_index_paths(PlannerInfo *root, RelOp
*** 852,858 ****
--- 851,866 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
+ {
+ /*
+ * In case grouping may be pushed down, try to identify unique
+ * keys to possibly avoid final aggregation.
+ */
+ if (root->grouped_var_list != NIL)
+ make_uniquekeys(root, (Path *) ipath);
+
add_path(rel, (Path *) ipath, false);
+ }
if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
*************** build_index_paths(PlannerInfo *root, Rel
*** 1185,1191 ****
(Path *) ipath,
true,
agg_info,
! ipath->path.rows);
if (agg_path != NULL)
agg_sorted->paths = lappend(agg_sorted->paths, agg_path);
--- 1193,1200 ----
(Path *) ipath,
true,
agg_info,
! ipath->path.rows,
! false);
if (agg_path != NULL)
agg_sorted->paths = lappend(agg_sorted->paths, agg_path);
*************** build_index_paths(PlannerInfo *root, Rel
*** 1198,1204 ****
agg_path = create_partial_agg_hashed_path(root,
(Path *) ipath,
agg_info,
! ipath->path.rows);
if (agg_path != NULL)
agg_hashed->paths = lappend(agg_hashed->paths, agg_path);
--- 1207,1214 ----
agg_path = create_partial_agg_hashed_path(root,
(Path *) ipath,
agg_info,
! ipath->path.rows,
! false);
if (agg_path != NULL)
agg_hashed->paths = lappend(agg_hashed->paths, agg_path);
*************** build_index_paths(PlannerInfo *root, Rel
*** 1250,1256 ****
(Path *) ipath,
false,
agg_info,
! ipath->path.rows);
/*
* If create_agg_sorted_path succeeded once, it should
--- 1260,1267 ----
(Path *) ipath,
false,
agg_info,
! ipath->path.rows,
! true);
/*
* If create_agg_sorted_path succeeded once, it should
*************** build_index_paths(PlannerInfo *root, Rel
*** 1267,1273 ****
agg_path = create_partial_agg_hashed_path(root,
(Path *) ipath,
agg_info,
! ipath->path.rows);
/*
* If create_agg_hashed_path succeeded once, it should
--- 1278,1285 ----
agg_path = create_partial_agg_hashed_path(root,
(Path *) ipath,
agg_info,
! ipath->path.rows,
! true);
/*
* If create_agg_hashed_path succeeded once, it should
*************** build_index_paths(PlannerInfo *root, Rel
*** 1330,1336 ****
(Path *) ipath,
true,
agg_info,
! ipath->path.rows);
if (agg_path != NULL)
agg_sorted->paths = lappend(agg_sorted->paths,
--- 1342,1349 ----
(Path *) ipath,
true,
agg_info,
! ipath->path.rows,
! false);
if (agg_path != NULL)
agg_sorted->paths = lappend(agg_sorted->paths,
*************** build_index_paths(PlannerInfo *root, Rel
*** 1378,1384 ****
(Path *) ipath,
false,
agg_info,
! ipath->path.rows);
Assert(agg_path != NULL);
agg_sorted->paths_partial =
lappend(agg_sorted->paths_partial, agg_path);
--- 1391,1398 ----
(Path *) ipath,
false,
agg_info,
! ipath->path.rows,
! true);
Assert(agg_path != NULL);
agg_sorted->paths_partial =
lappend(agg_sorted->paths_partial, agg_path);
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 7422fc4..883d8a1
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** try_nestloop_path(PlannerInfo *root,
*** 520,525 ****
--- 520,532 ----
workspace.startup_cost, workspace.total_cost,
pathkeys, required_outer, grouped))
{
+ /*
+ * For a grouped path try to identify unique keys, to possibly avoid
+ * final aggregation.
+ */
+ if (grouped)
+ make_uniquekeys(root, (Path *) join_path);
+
add_path(joinrel, join_path, grouped);
}
else
*************** try_mergejoin_path(PlannerInfo *root,
*** 933,938 ****
--- 940,952 ----
workspace.startup_cost, workspace.total_cost,
si->merge_pathkeys, required_outer, grouped))
{
+ /*
+ * For a grouped path try to identify unique keys, to possibly avoid
+ * final aggregation.
+ */
+ if (grouped)
+ make_uniquekeys(root, (Path *) join_path);
+
add_path(joinrel, (Path *) join_path, grouped);
}
else
*************** try_hashjoin_path(PlannerInfo *root,
*** 1239,1244 ****
--- 1253,1265 ----
workspace.startup_cost, workspace.total_cost,
NIL, required_outer, grouped))
{
+ /*
+ * For a grouped path try to identify unique keys, to possibly avoid
+ * final aggregation.
+ */
+ if (grouped)
+ make_uniquekeys(root, (Path *) join_path);
+
add_path(joinrel, (Path *) join_path, grouped);
}
else
diff --git a/src/backend/optimizer/path/pathkeys.c b/src/backend/optimizer/path/pathkeys.c
new file mode 100644
index c6870d3..bee0157
*** a/src/backend/optimizer/path/pathkeys.c
--- b/src/backend/optimizer/path/pathkeys.c
*************** has_useful_pathkeys(PlannerInfo *root, R
*** 1563,1565 ****
--- 1563,1719 ----
return true; /* might be able to use them for ordering */
return false; /* definitely useless */
}
+
+ /*
+ * Add a new set of unique keys to path's list of unique key sets. If an
+ * identical set is already there --- free new_set instead of adding it.
+ */
+ void
+ add_uniquekeys_to_path(Path *path, Bitmapset *new_set)
+ {
+ ListCell *lc;
+
+ foreach(lc, path->uniquekeys)
+ {
+ Bitmapset *set = (Bitmapset *) lfirst(lc);
+
+ if (bms_equal(new_set, set))
+ break;
+ }
+ if (lc == NULL)
+ path->uniquekeys = lappend(path->uniquekeys, new_set);
+ else
+ bms_free(new_set);
+ }
+
+ /*
+ * Return true if path output is grouped in a way described by
+ * root->group_pathkeys. AGGSPLIT_FINAL_DESERIAL can be skipped in such a
+ * case.
+ */
+ bool
+ match_path_to_group_pathkeys(PlannerInfo *root, Path *path)
+ {
+ Bitmapset *uniquekeys_all = NULL;
+ ListCell *l1;
+ int i;
+ bool *is_group_expr;
+
+ /*
+ * group_pathkeys are essential for this function.
+ */
+ if (root->group_pathkeys == NIL)
+ return false;
+
+ /*
+ * The path is not aware of being unique.
+ */
+ if (path->uniquekeys == NIL)
+ return false;
+
+ /*
+ * There can be multiple known unique key sets. Gather pathkeys of all the
+ * unique expressions the sets may reference.
+ */
+ foreach(l1, path->uniquekeys)
+ {
+ Bitmapset *set = (Bitmapset *) lfirst(l1);
+
+ uniquekeys_all = bms_union(uniquekeys_all, set);
+ }
+
+ /*
+ * Find pathkeys for the expressions.
+ */
+ is_group_expr = (bool *)
+ palloc0(list_length(path->pathtarget->exprs) * sizeof(bool));
+
+ i = 0;
+ foreach(l1, path->pathtarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+
+ if (bms_is_member(i, uniquekeys_all))
+ {
+ ListCell *l2;
+ bool found = false;
+
+ /*
+ * This is an unique expression, so find its pathkey.
+ */
+ foreach(l2, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l2);
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *l3;
+ EquivalenceMember *em = NULL;
+
+ if (ec->ec_below_outer_join)
+ continue;
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(l3, ec->ec_members)
+ {
+ em = lfirst_node(EquivalenceMember, l3);
+
+ if (em->em_nullable_relids)
+ continue;
+
+ if (equal(em->em_expr, expr))
+ {
+ found = true;
+ break;
+ }
+ }
+ if (found)
+ break;
+
+ }
+ is_group_expr[i] = found;
+ }
+
+ i++;
+ }
+
+ /*
+ * Now check the unique key sets and see if any one has all its
+ * expressions among group_pathkeys.
+ */
+ foreach(l1, path->uniquekeys)
+ {
+ Bitmapset *set = (Bitmapset *) lfirst(l1);
+ bool found = false;
+
+ /*
+ * Collect PKs associated with this set.
+ */
+ for (i = 0; i < list_length(path->pathtarget->exprs); i++)
+ {
+ if (bms_is_member(i, set))
+ {
+ /*
+ * If the set misses a single grouping path key, at least one
+ * expression of the unique key is outside the grouping
+ * expressions, and thus the grouping expressions are not
+ * guaranteed to be unique. Thus the avoidance of final
+ * aggregation is not justified.
+ */
+ if (!is_group_expr[i])
+ {
+ found = true;
+ break;
+ }
+ }
+ }
+
+ /*
+ * No problem with this set. No need to check the other ones.
+ */
+ if (!found)
+ return true;
+ }
+
+ /* No match found. */
+ return false;
+ }
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 94b1f8e..6ad1218
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static PathTarget *make_sort_input_targe
*** 194,199 ****
--- 194,201 ----
bool *have_postponed_srfs);
static void adjust_paths_for_srfs(PlannerInfo *root, RelOptInfo *rel,
List *targets, List *targets_contain_srfs);
+ static Path *adjust_path_for_unique_group_keys(PlannerInfo *root, Path *path,
+ PathTarget *target);
/*****************************************************************************
*************** grouping_planner(PlannerInfo *root, bool
*** 1977,1982 ****
--- 1979,1985 ----
grouping_target,
&agg_costs,
gset_data);
+
/* Fix things up if grouping_target contains SRFs */
if (parse->hasTargetSRFs)
adjust_paths_for_srfs(root, current_rel,
*************** create_grouping_paths(PlannerInfo *root,
*** 4178,4183 ****
--- 4181,4212 ----
}
}
+ /*
+ * If input_rel has partially aggregated paths which emit an unique set of
+ * grouping keys, we only need to call aggfinalfn on each aggregate state
+ * instead of doing the final aggregation
+ */
+ if (input_rel->gpi != NULL && !parse->groupingSets)
+ {
+ foreach(lc, input_rel->gpi->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * If aggregation could have been pushed down and if the path
+ * generates unique grouping keys, replace aggregates with the
+ * aggregate final functions alone.
+ */
+ if ((parse->groupClause || parse->hasAggs || root->hasHavingQual)
+ && root->grouped_var_list != NIL &&
+ match_path_to_group_pathkeys(root, path))
+ add_path(grouped_rel,
+ adjust_path_for_unique_group_keys(root, path,
+ target),
+ false);
+ }
+ }
+
/* Give a helpful error if we failed to find any implementation */
if (grouped_rel->pathlist == NIL)
ereport(ERROR,
*************** adjust_paths_for_srfs(PlannerInfo *root,
*** 6160,6165 ****
--- 6189,6228 ----
}
/*
+ * Project a path that generates unique grouping keys to a new one whose
+ * target, instead of the (partial) aggregates, contains the aggregates' final
+ * functions, each referencing the corresponding partial aggregate.
+ */
+ static Path *
+ adjust_path_for_unique_group_keys(PlannerInfo *root, Path *path,
+ PathTarget *target)
+ {
+ Path *result;
+
+ /*
+ * While path->pathtarget contains GroupedVars, "target contains Aggrefs.
+ * setrefs.c will recognize and handle this special case.
+ *
+ * Pass true for "force_result" so that the input path's target does not
+ * receive extra entries (such as junk TLEs) from upper nodes (e.g. Sort)
+ * that it might not be able to evaluate (e.g. aggregates)
+ */
+ result = (Path *) create_projection_path(root, path->parent, path,
+ target, true);
+
+ /*
+ * TODO Account for the evaluation of the aggfinalfn function. BTW, this
+ * might be a case for a new kind of Path instead of projection. Otherwise
+ * it's hard to detect this special kind of input and output targetlist,
+ * without checking for presence of Aggrefs and GroupedVars respectively.
+ */
+ result->startup_cost = path->startup_cost;
+ result->total_cost = path->total_cost;
+ result->rows = path->rows;
+ return result;
+ }
+
+ /*
* expression_planner
* Perform planner's transformations on a standalone expression.
*
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index b5be81c..502c725
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** static Node *fix_scan_expr_mutator(Node
*** 108,113 ****
--- 108,115 ----
static bool fix_scan_expr_walker(Node *node, fix_scan_expr_context *context);
static void set_join_references(PlannerInfo *root, Join *join, int rtoffset);
static void set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset);
+ static bool contains_groupedvar(Node *node);
+ static bool contains_groupedvar_walker(Node *node, void *context);
static void set_param_references(PlannerInfo *root, Plan *plan);
static Node *convert_combining_aggrefs(Node *node, void *context);
static void set_dummy_tlist_references(Plan *plan, int rtoffset);
*************** set_upper_references(PlannerInfo *root,
*** 1786,1791 ****
--- 1788,1808 ----
replace_grouped_vars_with_aggrefs(root, subplan->targetlist);
}
}
+ else if (IsA(plan, Result))
+ {
+ /*
+ * Check if this is the projection created by
+ * adjust_path_for_unique_group_keys.
+ */
+ if (root->grouped_var_list != NIL &&
+ contains_aggref_simple((Node *) plan->targetlist) &&
+ contains_groupedvar((Node *) subplan->targetlist))
+ {
+ plan->targetlist = (List *)
+ replace_aggref_with_groupedvar((Node *) plan->targetlist,
+ root->grouped_var_list);
+ }
+ }
}
subplan_itlist = build_tlist_index(subplan->targetlist);
*************** set_upper_references(PlannerInfo *root,
*** 1841,1846 ****
--- 1858,1882 ----
}
/*
+ * Find out if the node contains at least one GroupedVar.
+ */
+ static bool
+ contains_groupedvar(Node *node)
+ {
+ return contains_groupedvar_walker(node, NULL);
+ }
+
+ static bool
+ contains_groupedvar_walker(Node *node, void *context)
+ {
+ if (node == NULL)
+ return false;
+ if (IsA(node, GroupedVar))
+ return true; /* abort the tree traversal and return true */
+ return expression_tree_walker(node, contains_groupedvar_walker, context);
+ }
+
+ /*
* set_param_references
* Initialize the initParam list in Gather or Gather merge node such that
* it contains reference of all the params that needs to be evaluated
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index c3528fc..1d01457
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
*************** static List *reparameterize_pathlist_by_
*** 60,65 ****
--- 60,68 ----
static Path *add_grouping_expressions_to_subpath(PlannerInfo *root,
Path *subpath,
PathTarget *group_exprs);
+ static void make_uniquekeys_for_unique_index(PathTarget *reltarget,
+ IndexOptInfo *index, Path *path);
+ static void make_uniquekeys_for_append_path(PlannerInfo *root, Path *path);
/*****************************************************************************
* MISC. PATH UTILITIES
*************** create_material_path(RelOptInfo *rel, Pa
*** 1515,1520 ****
--- 1518,1524 ----
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = subpath->pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
*************** create_projection_path(PlannerInfo *root
*** 2434,2439 ****
--- 2438,2444 ----
pathnode->path.parallel_workers = subpath->parallel_workers;
/* Projection does not change the sort order */
pathnode->path.pathkeys = subpath->pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
pathnode->force_result = force_result;
*************** create_sort_path(PlannerInfo *root,
*** 2685,2690 ****
--- 2690,2696 ----
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
*************** get_partial_agg_info(PlannerInfo *root,
*** 2963,2969 ****
/*
* Apply partial AGG_SORTED aggregation path to subpath if it's suitably
! * sorted.
*
* check_pathkeys can be passed FALSE if the function was already called for
* given index --- since the target should not change, we can skip the check
--- 2969,2976 ----
/*
* Apply partial AGG_SORTED aggregation path to subpath if it's suitably
! * sorted. ("partial" in the function name refers to AGGSPLIT_INITIAL_SERIAL
! * strategy, as opposed to parallel processing.)
*
* check_pathkeys can be passed FALSE if the function was already called for
* given index --- since the target should not change, we can skip the check
*************** AggPath *
*** 2977,2983 ****
create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
bool check_pathkeys,
PartialAggInfo *agg_info,
! double input_rows)
{
RelOptInfo *rel;
AggClauseCosts agg_costs;
--- 2984,2991 ----
create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
bool check_pathkeys,
PartialAggInfo *agg_info,
! double input_rows,
! bool parallel)
{
RelOptInfo *rel;
AggClauseCosts agg_costs;
*************** create_partial_agg_sorted_path(PlannerIn
*** 3047,3065 ****
Assert(agg_info->group_clauses != NIL);
result = create_agg_path(root, rel, subpath, rel->gpi->target,
AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
! agg_info->group_clauses, NIL, &agg_costs, dNumGroups);
return result;
}
/*
! * Apply partial AGG_HASHED aggregation to subpath.
*
* Arguments have the same meaning as those of create_agg_sorted_path.
*/
AggPath *
create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
! PartialAggInfo *agg_info, double input_rows)
{
RelOptInfo *rel;
bool can_hash;
--- 3055,3083 ----
Assert(agg_info->group_clauses != NIL);
result = create_agg_path(root, rel, subpath, rel->gpi->target,
AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
! agg_info->group_clauses, NIL, &agg_costs,
! dNumGroups);
!
! /*
! * Check if there's a chance to avoid final aggregation.
! */
! if (!parallel)
! make_uniquekeys(root, (Path *) result);
return result;
}
/*
! * Apply partial AGG_HASHED aggregation to subpath. ("partial" in the function
! * name refers to AGGSPLIT_INITIAL_SERIAL strategy, as opposed to parallel
! * processing.)
*
* Arguments have the same meaning as those of create_agg_sorted_path.
*/
AggPath *
create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
! PartialAggInfo *agg_info, double input_rows,
! bool parallel)
{
RelOptInfo *rel;
bool can_hash;
*************** create_partial_agg_hashed_path(PlannerIn
*** 3123,3128 ****
--- 3141,3152 ----
}
}
+ /*
+ * Check if there's a chance to avoid final aggregation.
+ */
+ if (result != NULL && !parallel)
+ make_uniquekeys(root, (Path *) result);
+
return result;
}
*************** add_grouping_expressions_to_subpath(Plan
*** 4268,4270 ****
--- 4292,4881 ----
target,
true);
}
+
+ /*
+ * Find out if path produces an unique set of expressions and set uniquekeys
+ * accordingly.
+ *
+ * TODO Check if any expression of any unique key isn't nullable, whether in
+ * the table / index or by an outer join.
+ */
+ void
+ make_uniquekeys(PlannerInfo *root, Path *path)
+ {
+ RelOptInfo *rel;
+
+ /*
+ * In general, uniquekeys make no sense if the query has HAVING clause
+ * because it requires the aggregate finalization anyway.
+ *
+ * XXX If all aggregates, including those in the HAVING clause had no
+ * aggfinalfn, we could add the corresponding GroupedVars to the grouped
+ * target and construct the uniquekeys. Maybe even evaluate the HAVING
+ * quals at relation / join level?
+ */
+ if (root->parse->havingQual != NULL)
+ return;
+
+ /*
+ * The unique keys are not interesting if there's no chance to push
+ * aggregation down to base relations / joins.
+ */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Do not accept repeated calls of the function on the same path.
+ */
+ if (path->uniquekeys != NIL)
+ return;
+
+ rel = path->parent;
+
+ /*
+ * Base relations.
+ */
+ if (IsA(path, IndexPath) ||
+ (IsA(path, Path) &&path->pathtype == T_SeqScan) ||
+ IsA(path, AppendPath) ||IsA(path, MergeAppendPath))
+ {
+ ListCell *lc;
+
+ /*
+ * Derive grouping keys from unique indexes.
+ */
+ if (IsA(path, IndexPath) ||IsA(path, Path))
+ {
+ foreach(lc, rel->indexlist)
+ {
+ IndexOptInfo *index = lfirst_node(IndexOptInfo, lc);
+
+ make_uniquekeys_for_unique_index(rel->reltarget, index, path);
+ }
+ }
+ else if (IsA(path, AppendPath) ||IsA(path, MergeAppendPath))
+ make_uniquekeys_for_append_path(root, path);
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(false);
+ #endif
+ return;
+ }
+
+ if (IsA(path, AggPath))
+ {
+ /*
+ * The immediate output of aggregation essentially produces an unique
+ * set of grouping keys.
+ */
+ make_uniquekeys_for_agg_path(root, path);
+ }
+ else if (IS_JOIN_REL(path->parent))
+ {
+ JoinPath *jpath = (JoinPath *) path;
+ Path *outerpath = jpath->outerjoinpath;
+ Path *innerpath = jpath->innerjoinpath;
+ ListCell *l1;
+
+ /*
+ * Find out if the join produces unique keys for various combinations
+ * of input sets of unique keys.
+ *
+ * TODO Implement heuristic that picks a few most useful sets on each
+ * side, to avoid exponential growth of the uniquekeys list as we
+ * proceed from lower to higher joins. Maybe also discard the
+ * resulting sets containing unique expressions which are not grouping
+ * expressions (and of course which are not aggregates) of this join's
+ * target.
+ */
+ foreach(l1, outerpath->uniquekeys)
+ {
+ Bitmapset *outerset = (Bitmapset *) lfirst(l1);
+ ListCell *l2;
+
+ foreach(l2, innerpath->uniquekeys)
+ {
+ Bitmapset *innerset = (Bitmapset *) lfirst(l2);
+ Bitmapset *joinset;
+
+ /*
+ * Given that unique keys of each input relation are contained
+ * in that relation's target, the union of the key sets should
+ * be contained in the join target.
+ */
+ joinset = bms_union(outerset, innerset);
+
+ /* Add the set to the path. */
+ add_uniquekeys_to_path((Path *) jpath, joinset);
+ }
+ }
+ }
+ #ifdef USE_ASSERT_CHECKING
+
+ /*
+ * TODO Consider other ones, e.g. UniquePath.
+ */
+ else
+ Assert(false);
+ #endif
+ }
+
+ /*
+ * Create a set of positions of expressions in reltarget if the index is
+ * unique and if reltarget contains all the index columns. Add the set to
+ * uniquekeys if identical one is not already there.
+ */
+ static void
+ make_uniquekeys_for_unique_index(PathTarget *reltarget, IndexOptInfo *index,
+ Path *path)
+ {
+ int i;
+ Bitmapset *new_set = NULL;
+
+ /*
+ * Give up if the index does not guarantee uniqueness.
+ */
+ if (!index->unique || !index->immediate ||
+ (index->indpred != NIL && !index->predOK))
+ return;
+
+ /*
+ * For the index path to be acceptable, reltarget must contain all the
+ * index columns.
+ *
+ * reltarget is not supposed to contain non-var expressions, so the index
+ * should neither.
+ */
+ if (index->indexprs != NULL)
+ return;
+
+ for (i = 0; i < index->ncolumns; i++)
+ {
+ int indkey = index->indexkeys[i];
+ ListCell *lc;
+ bool found = false;
+ int j = 0;
+
+ foreach(lc, reltarget->exprs)
+ {
+ Var *var = lfirst_node(Var, lc);
+
+ if (var->varno == index->rel->relid && var->varattno == indkey)
+ {
+ new_set = bms_add_member(new_set, j);
+ found = true;
+ break;
+ }
+
+ j++;
+ }
+
+ /*
+ * If rel needs less than the whole index key then the values of the
+ * columns matched so far can be duplicate.
+ */
+ if (!found)
+ {
+ bms_free(new_set);
+ return;
+ }
+ }
+
+ /*
+ * Add the set to the path, unless it's already there.
+ */
+ add_uniquekeys_to_path(path, new_set);
+ }
+
+ /*
+ * Create uniquekeys for a path that has Aggrefs in its target.
+ * set.
+ *
+ * Besides AggPath, ForeignPath is a known use case for this function.
+ */
+ void
+ make_uniquekeys_for_agg_path(PlannerInfo *root, Path *path)
+ {
+ PathTarget *target;
+ ListCell *lc;
+ Bitmapset *keyset = NULL;
+ int i = 0;
+
+ target = path->pathtarget;
+ Assert(target->sortgrouprefs != NULL);
+
+ foreach(lc, target->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, expr);
+
+ if (!IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Generic grouping expression.
+ */
+ keyset = bms_add_member(keyset, i);
+ }
+ }
+ else
+ {
+ Index sortgroupref = target->sortgrouprefs[i];
+
+ Assert(IsA(expr, Var));
+
+ /*
+ * Ignore "extra grouping expressions" that might have been added
+ * by prepare_rel_for_grouping() --- these exist merely for join
+ * purposes and thus should not appear in the final targetlist.
+ */
+ if (sortgroupref > 0 && sortgroupref <= root->max_sortgroupref)
+ {
+ /*
+ * Plain Var grouping expression.
+ */
+ keyset = bms_add_member(keyset, i);
+ }
+ else
+ {
+ /*
+ * A column functionally dependent on the GROUP BY clause?
+ */
+ }
+ }
+
+ i++;
+ }
+
+ add_uniquekeys_to_path((Path *) path, keyset);
+ }
+
+ /*
+ * Create uniquekeys for a AppendPath or MergeAppendPath.
+ *
+ * Besides AggPath, ForeignPath is a known use case for this function.
+ */
+ static void
+ make_uniquekeys_for_append_path(PlannerInfo *root, Path *path)
+ {
+ RelOptInfo *rel = path->parent;
+ List *subpaths;
+ Path *subpath;
+ ListCell *l1;
+ bool first = true;
+ List *uniquekeys_common = NIL;
+
+ /*
+ * In addition to the requirement that all subpaths must have uniquekeys
+ * set, the table needs to be partitioned in a specific way. Reject
+ * non-partitioned tables immediately, the other check to follow.
+ */
+ if (!IS_PARTITIONED_REL(rel))
+ return;
+
+ if (IsA(path, AppendPath))
+ subpaths = ((AppendPath *) path)->subpaths;
+ else if (IsA(path, MergeAppendPath))
+ subpaths = ((MergeAppendPath *) path)->subpaths;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(false);
+ #endif
+
+ /*
+ * Check if each subpath has uniquekeys.
+ */
+ foreach(l1, subpaths)
+ {
+ subpath = (Path *) lfirst(l1);
+
+ /*
+ * If any subpath does not have uniquekeys, the whole append path can
+ * have them neither.
+ */
+ if (subpath->uniquekeys == NIL)
+ return;
+ }
+
+ /*
+ * Choose groupkey sets that each subpath has.
+ */
+ foreach(l1, subpaths)
+ {
+ List *common_new = NIL;
+ ListCell *l2;
+
+ subpath = (Path *) lfirst(l1);
+
+ if (uniquekeys_common == NIL)
+ {
+ /* Get the initial list from the first subpath. */
+ uniquekeys_common = subpath->uniquekeys;
+ continue;
+ }
+
+ /*
+ * Remove items missing in the current subpath from uniquekeys_common.
+ */
+ foreach(l2, uniquekeys_common)
+ {
+ Bitmapset *set1 = (Bitmapset *) lfirst(l2);
+ ListCell *l3;
+
+ /*
+ * Does the current subpath contain this set?
+ */
+ foreach(l3, subpath->uniquekeys)
+ {
+ Bitmapset *set2 = (Bitmapset *) lfirst(l3);
+
+ if (bms_equal(set1, set2))
+ {
+ common_new = lappend(common_new, set1);
+ break;
+ }
+ }
+ }
+
+ /*
+ * If there are no sets in common, the next subpaths cannot help us.
+ */
+ if (common_new == NIL)
+ return;
+
+ /*
+ * Adopt the new, possibly reduced list.
+ */
+ uniquekeys_common = common_new;
+ }
+
+ foreach(l1, subpaths)
+ {
+ subpath = (Path *) lfirst(l1);
+
+ /*
+ * The following tests should ideally be performed either on
+ * rel->reltarget or rel->gpi->target outside the iteration of
+ * subpaths. However there's no easy way to find out if path is
+ * grouped or not. So we check the target of the first subpath and
+ * assume that the other ones would pass or fail in the same way:
+ * another subpath target should be the same path target whose
+ * attributes are translated to another child relations.
+ */
+ if (first)
+ {
+ List *partexprs = NIL;
+ List *partexprs_all = NIL;
+ ListCell *l2,
+ *l3;
+ AppendRelInfo **appinfos;
+ int i,
+ nappinfos;
+ bool found;
+ List *texprs = NIL;
+
+ /*
+ * Put all partition expressions into two lists --- one for
+ * non-nullable expressions, one for nullable.
+ */
+ for (i = 0; i < rel->part_scheme->partnatts; i++)
+ {
+ List *sublist;
+
+ sublist = rel->partexprs[i];
+ if (sublist != NIL)
+ {
+ partexprs = list_union(partexprs, sublist);
+ partexprs_all = list_union(partexprs_all,
+ sublist);
+ }
+
+ /*
+ * The nullable expressions should only appear in
+ * partexprs_all.
+ */
+ sublist = rel->nullable_partexprs[i];
+ if (sublist != NIL)
+ partexprs_all = list_union(partexprs_all,
+ sublist);
+ }
+ Assert(partexprs != NIL);
+
+ /*
+ * Translate the partitioning expressions so that they can match
+ * the subpath target.
+ */
+ appinfos =
+ find_appinfos_by_relids(root, subpath->parent->relids,
+ &nappinfos);
+ partexprs = (List *)
+ adjust_appendrel_attrs(root, (Node *) partexprs,
+ nappinfos, appinfos);
+ partexprs_all = (List *)
+ adjust_appendrel_attrs(root, (Node *) partexprs_all,
+ nappinfos, appinfos);
+ pfree(appinfos);
+
+ /*
+ * Since equivalence classes can be used to derive target
+ * expressions, the following checks have to consider ECs too. The
+ * targetlist we construct here contains the original expressions
+ * plus those generated from the appropriate ECs.
+ */
+ foreach(l2, subpath->pathtarget->exprs)
+ {
+ Expr *texpr = (Expr *) lfirst(l2);
+
+ if (IsA(texpr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, texpr);
+
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Aggregate should not appear in any EC.
+ */
+ texprs = lappend(texprs, texpr);
+ continue;
+ }
+
+ /*
+ * The contained expression (i.e. generic grouping
+ * expression) is what we'll search for in the ECs.
+ */
+ texpr = gvar->gvexpr;
+ }
+
+ /*
+ * Search for matching EC.
+ */
+ foreach(l3, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l3);
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *l4;
+ EquivalenceMember *em;
+
+ if (ec->ec_below_outer_join)
+ continue;
+
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(l4, ec->ec_members)
+ {
+ em = lfirst_node(EquivalenceMember, l4);
+
+ if (em->em_nullable_relids)
+ continue;
+
+ if (!bms_is_subset(em->em_relids,
+ subpath->parent->relids))
+ continue;
+
+ if (equal(em->em_expr, texpr))
+ break;
+ }
+
+ /*
+ * texpr not found in the current EC, try the next one.
+ */
+ if (l4 == NULL)
+ continue;
+
+ /*
+ * The EC does match, so add all its members to texprs.
+ */
+ foreach(l4, ec->ec_members)
+ {
+ em = lfirst_node(EquivalenceMember, l4);
+ texprs = lappend(texprs, em->em_expr);
+ }
+
+ /*
+ * No expression is supposed to appear in multiple EC.
+ */
+ continue;
+ }
+ }
+
+ /*
+ * To ensure that no output row can be produced by multiple
+ * partitions, check that:
+ *
+ * (a) at least one output column is a partitioning expression.
+ * Thus no output row of this path can appear in multiple
+ * partitions.
+ *
+ * Only search in the non-nullable partitioning expressions
+ * because NULL value can be generated by any partition.
+ */
+ found = false;
+ foreach(l2, texprs)
+ {
+ Expr *texpr = (Expr *) lfirst(l2);
+
+ /*
+ * Try to find the matching partitioning expression.
+ */
+ foreach(l3, partexprs)
+ {
+ Expr *pexpr = lfirst(l3);
+
+ if (equal(texpr, pexpr))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ if (found)
+ break;
+ }
+ if (!found)
+ {
+ list_free(uniquekeys_common);
+ return;
+ }
+
+ /*
+ * (b) there's no partitioning expression not contained in the
+ * target. If there was some, then the partitioning expression
+ * found above could appear in multiple partitions.
+ *
+ * Even nullable partition key makes harm here, so consider all
+ * partition keys this time.
+ */
+ foreach(l2, partexprs_all)
+ {
+ Expr *pexpr = lfirst(l2);
+
+ found = false;
+ foreach(l3, texprs)
+ {
+ Expr *texpr = (Expr *) lfirst(l3);
+
+ if (equal(pexpr, texpr))
+ {
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ {
+ list_free(uniquekeys_common);
+ return;
+ }
+ }
+ list_free(texprs);
+
+ /* Do not repeat the checks for the other subpaths. */
+ first = false;
+ }
+ }
+
+ /* All the checks passed. */
+ path->uniquekeys = uniquekeys_common;
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index b9c2d13..3f25949
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** replace_grouped_vars_with_aggrefs(Planne
*** 835,840 ****
--- 835,970 ----
return result;
}
+ static Node *
+ replace_aggref_with_groupedvar_mutator(Node *node, List *grouped_vars)
+ {
+ if (node == NULL)
+ return NULL;
+
+ /*
+ * No node should contain both AGGSPLIT_SIMPLE Aggref and GroupedVar.
+ * Caller should know when the replacement should be performed, otherwise
+ * he must wastes memory and time.
+ */
+ Assert(!IsA(node, GroupedVar));
+
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+ ListCell *l;
+
+ /*
+ * There's currently no usecase for other aggsplit. If there should be
+ * some, extra effort is needed to make the aggref match gvi->gvexpr.
+ */
+ Assert(aggref->aggsplit == AGGSPLIT_SIMPLE);
+
+ foreach(l, grouped_vars)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, l);
+
+ if (equal(gvi->gvexpr, aggref))
+ {
+ GroupedVar *gvar = makeNode(GroupedVar);
+
+ gvar = makeNode(GroupedVar);
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->gvid = gvi->gvid;
+ gvar->sortgroupref = gvi->sortgroupref;
+
+ /*
+ * Arrange for the call of aggfinalfn or let the transient
+ * state appear on the output unprocessed.
+ */
+ if (OidIsValid(aggref->aggfinalfn))
+ {
+ FuncExpr *fexpr = makeNode(FuncExpr);
+
+ fexpr = makeNode(FuncExpr);
+ fexpr->funcid = aggref->aggfinalfn;
+ fexpr->funcresulttype = aggref->aggtype;
+ fexpr->funcretset = false;
+ fexpr->funcvariadic = aggref->aggvariadic;
+ fexpr->funcformat = COERCE_EXPLICIT_CALL;
+ fexpr->funccollid = aggref->aggcollid;
+ fexpr->inputcollid = aggref->inputcollid;
+ fexpr->args = list_make1(gvar);
+ fexpr->location = -1;
+ return (Node *) fexpr;
+ }
+ else
+ return (Node *) gvar;
+ }
+ }
+ /* Match should have been found. */
+ Assert(l == NULL);
+ }
+
+ return expression_tree_mutator(node,
+ replace_aggref_with_groupedvar_mutator,
+ (void *) grouped_vars);
+ }
+
+ /*
+ * Replace each Aggref (only AGGSPLIT_SIMPLE expected) in the expression with
+ * the matching GroupedVar.
+ *
+ * Do not use if node may contain GroupedVars!
+ */
+ Node *
+ replace_aggref_with_groupedvar(Node *node, List *grouped_vars)
+ {
+ return expression_tree_mutator(node,
+ replace_aggref_with_groupedvar_mutator,
+ grouped_vars);
+ }
+
+ static bool
+ contains_aggref_simple_walker(Node *node, void *context)
+ {
+ bool *result_p = (bool *) context;
+
+ if (node == NULL)
+ return false;
+
+ /*
+ * Not interested in Aggrefs wrapped in GroupedVar.
+ */
+ if (IsA(node, GroupedVar))
+ return false;
+
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ *result_p = aggref->aggsplit == AGGSPLIT_SIMPLE;
+
+ /*
+ * Abort the tree traversal regardless *result_p. We assume that
+ * targetlist should never contain both AGGSPLIT_SIMPLE and any other
+ * value of aggsplit. Thus the search for AGGSPLIT_SIMPLE makes no
+ * more sense even if this aggregate has a different value.
+ */
+ return true;
+ }
+ return expression_tree_walker(node, contains_aggref_simple_walker,
+ context);
+ }
+
+ /*
+ * Find out if the node contains at least one Aggref having aggsplit equal to
+ * AGGSPLIT_SIMPLE.
+ */
+ bool
+ contains_aggref_simple(Node *node)
+ {
+ bool result = false;
+
+ contains_aggref_simple_walker(node, (void *) &result);
+
+ return result;
+ }
+
/*
* For each aggregate add GroupedVar to the grouped target.
*
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
new file mode 100644
index 7a98753..b5baa2c
*** a/src/backend/parser/parse_func.c
--- b/src/backend/parser/parse_func.c
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 99,104 ****
--- 99,105 ----
FuncDetailCode fdresult;
char aggkind = 0;
Oid aggcombinefn = InvalidOid;
+ Oid aggfinalfn = InvalidOid;
ParseCallbackState pcbstate;
/*
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 352,357 ****
--- 353,359 ----
classForm = (Form_pg_aggregate) GETSTRUCT(tup);
aggkind = classForm->aggkind;
aggcombinefn = classForm->aggcombinefn;
+ aggfinalfn = classForm->aggfinalfn;
catDirectArgs = classForm->aggnumdirectargs;
ReleaseSysCache(tup);
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 698,703 ****
--- 700,706 ----
aggref->aggvariadic = func_variadic;
aggref->aggkind = aggkind;
aggref->aggcombinefn = aggcombinefn;
+ aggref->aggfinalfn = aggfinalfn;
/* agglevelsup will be set by transformAggregateCall */
aggref->aggsplit = AGGSPLIT_SIMPLE; /* planner might change this */
aggref->location = location;
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index 97fd887..edc9ab1
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct Aggref
*** 297,302 ****
--- 297,303 ----
Oid inputcollid; /* OID of collation that function should use */
Oid aggtranstype; /* type Oid of aggregate's transition value */
Oid aggcombinefn; /* combine function (see pg_aggregate.h) */
+ Oid aggfinalfn; /* final function (see pg_aggregate.h) */
List *aggargtypes; /* type Oids of direct and aggregated args */
List *aggdirectargs; /* direct arguments, if an ordered-set agg */
List *args; /* aggregated arguments and sort expressions */
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index 00e73ff..5093809
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct GroupedPathInfo
*** 1089,1094 ****
--- 1089,1098 ----
*
* "pathkeys" is a List of PathKey nodes (see above), describing the sort
* ordering of the path's output rows.
+ *
+ * "uniquekeys" is a List of Bitmapset objects, each pointing at a set of
+ * expressions of "pathtarget" whose values within the path output are
+ * distinct.
*/
typedef struct Path
{
*************** typedef struct Path
*** 1112,1117 ****
--- 1116,1125 ----
List *pathkeys; /* sort ordering of path's output */
/* pathkeys is a List of PathKey nodes; see above */
+
+ List *uniquekeys; /* list of bitmapsets where each set contains
+ * positions of unique expressions within
+ * pathtarget. */
} Path;
/* Macro for extracting a path's parameterization relids; beware double eval */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index bbcfc7f..6f3b78d
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern AggPath *create_partial_agg_sorte
*** 211,221 ****
Path *subpath,
bool check_pathkeys,
PartialAggInfo *agg_info,
! double input_rows);
extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
Path *subpath,
PartialAggInfo *agg_info,
! double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
--- 211,223 ----
Path *subpath,
bool check_pathkeys,
PartialAggInfo *agg_info,
! double input_rows,
! bool parallel);
extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
Path *subpath,
PartialAggInfo *agg_info,
! double input_rows,
! bool parallel);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern Path *reparameterize_path(Planner
*** 276,281 ****
--- 278,285 ----
double loop_count);
extern Path *reparameterize_path_by_child(PlannerInfo *root, Path *path,
RelOptInfo *child_rel);
+ extern void make_uniquekeys(PlannerInfo *root, Path *path);
+ extern void make_uniquekeys_for_agg_path(PlannerInfo *root, Path *path);
/*
* prototypes for relnode.c
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index b44b8a8..61f3c69
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern bool has_useful_pathkeys(PlannerI
*** 240,244 ****
extern PathKey *make_canonical_pathkey(PlannerInfo *root,
EquivalenceClass *eclass, Oid opfamily,
int strategy, bool nulls_first);
!
#endif /* PATHS_H */
--- 240,245 ----
extern PathKey *make_canonical_pathkey(PlannerInfo *root,
EquivalenceClass *eclass, Oid opfamily,
int strategy, bool nulls_first);
! extern void add_uniquekeys_to_path(Path *path, Bitmapset *new_set);
! extern bool match_path_to_group_pathkeys(PlannerInfo *root, Path *path);
#endif /* PATHS_H */
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index dfc9352..d7d9ce4
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern void split_pathtarget_at_srfs(Pla
*** 66,71 ****
--- 66,73 ----
/* TODO Find the best location (position and in some cases even file) for the
* following ones. */
extern List *replace_grouped_vars_with_aggrefs(PlannerInfo *root, List *src);
+ extern Node *replace_aggref_with_groupedvar(Node *node, List *grouped_vars);
+ extern bool contains_aggref_simple(Node *node);
extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
List *expressions);
extern GroupedVar *get_grouping_expression(PlannerInfo *root, Expr *expr);
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index cac70a0..f146994
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
*************** SELECT p.i, avg(c1.v) FROM agg_pushdown_
*** 30,44 ****
AS c1 ON c1.parent = p.i GROUP BY p.i;
QUERY PLAN
-------------------------------------------------------------------------------------
! Finalize HashAggregate
! Group Key: p.i
-> Nested Loop
-> Partial HashAggregate
Group Key: c1.parent
-> Seq Scan on agg_pushdown_child1 c1
-> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
Index Cond: (i = c1.parent)
! (8 rows)
-- Scan index on agg_pushdown_child1(parent) column and partially aggregate
-- the result using AGG_SORTED strategy.
--- 30,43 ----
AS c1 ON c1.parent = p.i GROUP BY p.i;
QUERY PLAN
-------------------------------------------------------------------------------------
! Result
-> Nested Loop
-> Partial HashAggregate
Group Key: c1.parent
-> Seq Scan on agg_pushdown_child1 c1
-> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
Index Cond: (i = c1.parent)
! (7 rows)
-- Scan index on agg_pushdown_child1(parent) column and partially aggregate
-- the result using AGG_SORTED strategy.
*************** SELECT p.i, avg(c1.v) FROM agg_pushdown_
*** 48,62 ****
AS c1 ON c1.parent = p.i GROUP BY p.i;
QUERY PLAN
---------------------------------------------------------------------------------------------
! Finalize GroupAggregate
! Group Key: p.i
-> Nested Loop
-> Partial GroupAggregate
Group Key: c1.parent
-> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
-> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
Index Cond: (i = c1.parent)
! (8 rows)
SET enable_seqscan TO on;
-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
--- 47,60 ----
AS c1 ON c1.parent = p.i GROUP BY p.i;
QUERY PLAN
---------------------------------------------------------------------------------------------
! Result
-> Nested Loop
-> Partial GroupAggregate
Group Key: c1.parent
-> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
-> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
Index Cond: (i = c1.parent)
! (7 rows)
SET enable_seqscan TO on;
-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
agg_pushdown_v5/10_fdw.diff
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
new file mode 100644
index 0690bd4..1e855d6
*** a/contrib/file_fdw/file_fdw.c
--- b/contrib/file_fdw/file_fdw.c
*************** PG_FUNCTION_INFO_V1(file_fdw_validator);
*** 117,126 ****
*/
static void fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static void fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static ForeignScan *fileGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
--- 117,128 ----
*/
static void fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static void fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static ForeignScan *fileGetForeignPlan(PlannerInfo *root,
RelOptInfo *baserel,
Oid foreigntableid,
*************** get_file_fdw_attribute_options(Oid relid
*** 487,497 ****
static void
fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
FileFdwPlanState *fdw_private;
/*
* Fetch options. We only need filename (or program) at this point, but
* we might as well get everything and not need to re-fetch it later in
* planning.
--- 489,507 ----
static void
fileGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
FileFdwPlanState *fdw_private;
/*
+ * XXX Grouping at relation level is possible but this FDW does not
+ * implement it yet.
+ */
+ if (grouped)
+ return;
+
+ /*
* Fetch options. We only need filename (or program) at this point, but
* we might as well get everything and not need to re-fetch it later in
* planning.
*************** fileGetForeignRelSize(PlannerInfo *root,
*** 518,524 ****
static void
fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
FileFdwPlanState *fdw_private = (FileFdwPlanState *) baserel->fdw_private;
Cost startup_cost;
--- 528,535 ----
static void
fileGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
FileFdwPlanState *fdw_private = (FileFdwPlanState *) baserel->fdw_private;
Cost startup_cost;
*************** fileGetForeignPaths(PlannerInfo *root,
*** 526,531 ****
--- 537,549 ----
List *columns;
List *coptions = NIL;
+ /*
+ * XXX Grouping at relation level is possible but this FDW does not
+ * implement it yet.
+ */
+ if (grouped)
+ return;
+
/* Decide whether to selectively perform binary conversion */
if (check_selective_binary_conversion(baserel,
foreigntableid,
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
new file mode 100644
index 0876589..22cd89d
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
*************** typedef struct deparse_expr_cxt
*** 103,108 ****
--- 103,109 ----
* a base relation. */
StringInfo buf; /* output buffer to append to */
List **params_list; /* exprs that will become remote Params */
+ List *tlist;
} deparse_expr_cxt;
#define REL_ALIAS_PREFIX "r"
*************** static void deparseTargetList(StringInfo
*** 133,139 ****
bool qualify_col,
List **retrieved_attrs);
static void deparseExplicitTargetList(List *tlist, List **retrieved_attrs,
! deparse_expr_cxt *context);
static void deparseSubqueryTargetList(deparse_expr_cxt *context);
static void deparseReturningList(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
--- 134,140 ----
bool qualify_col,
List **retrieved_attrs);
static void deparseExplicitTargetList(List *tlist, List **retrieved_attrs,
! deparse_expr_cxt *context);
static void deparseSubqueryTargetList(deparse_expr_cxt *context);
static void deparseReturningList(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
*************** static void printRemoteParam(int paramin
*** 163,171 ****
static void printRemotePlaceholder(Oid paramtype, int32 paramtypmod,
deparse_expr_cxt *context);
static void deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context);
static void deparseLockingClause(deparse_expr_cxt *context);
! static void appendOrderByClause(List *pathkeys, deparse_expr_cxt *context);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
static void deparseFromExprForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *joinrel, bool use_alias, List **params_list);
--- 164,173 ----
static void printRemotePlaceholder(Oid paramtype, int32 paramtypmod,
deparse_expr_cxt *context);
static void deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context, bool grouped);
static void deparseLockingClause(deparse_expr_cxt *context);
! static void appendOrderByClause(deparse_expr_cxt *context, List *expressions,
! List *pathkeys);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
static void deparseFromExprForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *joinrel, bool use_alias, List **params_list);
*************** static void deparseAggref(Aggref *node,
*** 177,185 ****
static void appendGroupByClause(List *tlist, deparse_expr_cxt *context);
static void appendAggOrderBy(List *orderList, List *targetList,
deparse_expr_cxt *context);
! static void appendFunctionName(Oid funcid, deparse_expr_cxt *context);
static Node *deparseSortGroupClause(Index ref, List *tlist,
deparse_expr_cxt *context);
/*
* Helper functions
--- 179,188 ----
static void appendGroupByClause(List *tlist, deparse_expr_cxt *context);
static void appendAggOrderBy(List *orderList, List *targetList,
deparse_expr_cxt *context);
! static void appendFunctionName(Oid funcid, StringInfo buf, char *schemaname);
static Node *deparseSortGroupClause(Index ref, List *tlist,
deparse_expr_cxt *context);
+ static void deparseSortGroupExpr(Expr *expr, deparse_expr_cxt *context);
/*
* Helper functions
*************** foreign_expr_walker(Node *node,
*** 365,370 ****
--- 368,381 ----
}
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ Assert(!IsA(gvar->gvexpr, Aggref));
+ return foreign_expr_walker((Node *) gvar->gvexpr, glob_cxt,
+ outer_cxt);
+ }
case T_Const:
{
Const *c = (Const *) node;
*************** foreign_expr_walker(Node *node,
*** 676,687 ****
Aggref *agg = (Aggref *) node;
ListCell *lc;
! /* Not safe to pushdown when not in grouping context */
! if (!IS_UPPER_REL(glob_cxt->foreignrel))
! return false;
!
! /* Only non-split aggregates are pushable. */
! if (agg->aggsplit != AGGSPLIT_SIMPLE)
return false;
/* As usual, it must be shippable. */
--- 687,701 ----
Aggref *agg = (Aggref *) node;
ListCell *lc;
! /*
! * Only non-split aggregates are pushable.
! *
! * XXX Like above, AGGSPLIT_SIMPLE shouldn't appear here if
! * the aggregation of the whole upper relation gets abandoned.
! * (Check all occurrences of AGGSPLIT_SIMPLE.)
! */
! if (agg->aggsplit != AGGSPLIT_SIMPLE &&
! agg->aggsplit != AGGSPLIT_INITIAL_SERIAL)
return false;
/* As usual, it must be shippable. */
*************** build_tlist_to_deparse(RelOptInfo *forei
*** 923,931 ****
*/
extern void
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
! List *tlist, List *remote_conds, List *pathkeys,
! bool is_subquery, List **retrieved_attrs,
! List **params_list)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
--- 937,945 ----
*/
extern void
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
! List *tlist, List *remote_conds, List *order_by_exprs,
! List *pathkeys, bool is_subquery, List **retrieved_attrs,
! List **params_list, bool grouped)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
*************** deparseSelectStmtForRel(StringInfo buf,
*** 943,951 ****
context.foreignrel = rel;
context.scanrel = IS_UPPER_REL(rel) ? fpinfo->outerrel : rel;
context.params_list = params_list;
/* Construct SELECT clause */
! deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context);
/*
* For upper relations, the WHERE clause is built from the remote
--- 957,966 ----
context.foreignrel = rel;
context.scanrel = IS_UPPER_REL(rel) ? fpinfo->outerrel : rel;
context.params_list = params_list;
+ context.tlist = tlist;
/* Construct SELECT clause */
! deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context, grouped);
/*
* For upper relations, the WHERE clause is built from the remote
*************** deparseSelectStmtForRel(StringInfo buf,
*** 965,977 ****
/* Construct FROM and WHERE clauses */
deparseFromExpr(quals, &context);
! if (IS_UPPER_REL(rel))
{
/* Append GROUP BY clause */
appendGroupByClause(tlist, &context);
! /* Append HAVING clause */
! if (remote_conds)
{
appendStringInfoString(buf, " HAVING ");
appendConditions(remote_conds, &context);
--- 980,1008 ----
/* Construct FROM and WHERE clauses */
deparseFromExpr(quals, &context);
! if (IS_UPPER_REL(rel) || grouped)
{
/* Append GROUP BY clause */
appendGroupByClause(tlist, &context);
! /*
! * Append HAVING clause.
! *
! * XXX As the "partition-wise join" patch will probably be committed
! * to PG core earlier than the "aggregate push-down" patch, we don't
! * add HAVING clause to query that represents remote simple relation
! * or join. The point is that the HAVING clause can't be evaluated
! * below the final aggregation node, and the final aggregation is
! * essentially local.
! *
! * To surpass this limitation, the FDW would have to know whether the
! * relation / join is a partition or the whole table. In case it's
! * partition, HAVING is only legal if the query does not use any other
! * partition. If we eventually can add the HAVING clause to scan /
! * join remote queries, we also have to change the API so that WHERE /
! * ON conditions are passed separate from the HAVING expression.
! */
! if (IS_UPPER_REL(rel) && remote_conds)
{
appendStringInfoString(buf, " HAVING ");
appendConditions(remote_conds, &context);
*************** deparseSelectStmtForRel(StringInfo buf,
*** 979,986 ****
}
/* Add ORDER BY clause if we found any useful pathkeys */
! if (pathkeys)
! appendOrderByClause(pathkeys, &context);
/* Add any necessary FOR UPDATE/SHARE. */
deparseLockingClause(&context);
--- 1010,1020 ----
}
/* Add ORDER BY clause if we found any useful pathkeys */
! if (order_by_exprs)
! {
! Assert(pathkeys);
! appendOrderByClause(&context, order_by_exprs, pathkeys);
! }
/* Add any necessary FOR UPDATE/SHARE. */
deparseLockingClause(&context);
*************** deparseSelectStmtForRel(StringInfo buf,
*** 1001,1007 ****
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
--- 1035,1041 ----
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
! deparse_expr_cxt *context, bool grouped)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
*************** deparseSelectSql(List *tlist, bool is_su
*** 1030,1035 ****
--- 1064,1078 ----
*/
deparseExplicitTargetList(tlist, retrieved_attrs, context);
}
+ else if (grouped && IS_SIMPLE_REL(foreignrel) && foreignrel->gpi != NULL)
+ {
+ /*
+ * An explicit targetlist is also passed if aggregation should take
+ * place in the remote database.
+ */
+ deparseExplicitTargetList(fpinfo->grouped_tlist, retrieved_attrs,
+ context);
+ }
else
{
/*
*************** deparseRangeTblRef(StringInfo buf, Plann
*** 1504,1511 ****
/* Deparse the subquery representing the relation. */
appendStringInfoChar(buf, '(');
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
! fpinfo->remote_conds, NIL, true,
! &retrieved_attrs, params_list);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
--- 1547,1554 ----
/* Deparse the subquery representing the relation. */
appendStringInfoChar(buf, '(');
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
! fpinfo->remote_conds, NIL, NIL, true,
! &retrieved_attrs, params_list, false);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
*************** deparseExpr(Expr *node, deparse_expr_cxt
*** 2126,2131 ****
--- 2169,2177 ----
case T_Var:
deparseVar((Var *) node, context);
break;
+ case T_GroupedVar:
+ deparseExpr((Expr *) ((GroupedVar *) node)->gvexpr, context);
+ break;
case T_Const:
deparseConst((Const *) node, context, 0);
break;
*************** deparseFuncExpr(FuncExpr *node, deparse_
*** 2469,2475 ****
/*
* Normal function: display as proname(args).
*/
! appendFunctionName(node->funcid, context);
appendStringInfoChar(buf, '(');
/* ... and all the arguments */
--- 2515,2521 ----
/*
* Normal function: display as proname(args).
*/
! appendFunctionName(node->funcid, context->buf, NULL);
appendStringInfoChar(buf, '(');
/* ... and all the arguments */
*************** deparseAggref(Aggref *node, deparse_expr
*** 2747,2761 ****
{
StringInfo buf = context->buf;
bool use_variadic;
/* Only basic, non-split aggregation accepted. */
! Assert(node->aggsplit == AGGSPLIT_SIMPLE);
/* Check if need to print VARIADIC (cf. ruleutils.c) */
use_variadic = node->aggvariadic;
/* Find aggregate name from aggfnoid which is a pg_proc entry */
! appendFunctionName(node->aggfnoid, context);
appendStringInfoChar(buf, '(');
/* Add DISTINCT */
--- 2793,2876 ----
{
StringInfo buf = context->buf;
bool use_variadic;
+ ListCell *lc;
+ bool use_core_aggregate = true;
+ char *schemaname = NULL;
/* Only basic, non-split aggregation accepted. */
! Assert(node->aggsplit == AGGSPLIT_SIMPLE ||
! node->aggsplit == AGGSPLIT_INITIAL_SERIAL);
/* Check if need to print VARIADIC (cf. ruleutils.c) */
use_variadic = node->aggvariadic;
+ if (context->root->grouped_var_list != NIL)
+ {
+ /*
+ * The core aggregates (i.e. those in postgres catalog) do or do not
+ * perform finalization on both remote and local side. But sometimes
+ * we need the same aggregate to run w/o finalization on the remote
+ * node and with the finalization locally.
+ *
+ * Since the current Aggref is partial, it essentially returns the
+ * transient type. We need to find the original aggregate.
+ */
+ foreach(lc, context->root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi;
+ Aggref *orig;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ /* GroupedVar currently represents only aggregates. */
+ orig = castNode(Aggref, gvi->gvexpr);
+
+ if (orig->aggfnoid == node->aggfnoid)
+ {
+ /*
+ * The core aggregate can only be useful if it does not
+ * perform finalization.
+ */
+ if (!OidIsValid(orig->aggfinalfn))
+ {
+ use_core_aggregate = true;
+
+ /*
+ * If there's no finalization, the aggregate should return
+ * value of the internal type.
+ */
+ Assert(orig->aggtype == orig->aggtranstype);
+ }
+ else
+ use_core_aggregate = false;
+
+ /*
+ * If there are multiple aggregates of the same aggfnoid in
+ * the query, they should all have the same values of the
+ * fields we're going to check. So just use the first one we
+ * find.
+ */
+ break;
+ }
+ }
+
+ /* Must have found the original aggregate in the list. */
+ Assert(lc != NULL);
+ }
+
+ /*
+ * TODO
+ *
+ * 1. Make the schema name configurable as FDW option.
+ *
+ * 2. Implement the partial aggregates. These don't fit into postgres_fdw
+ * because there's no reason to install postgres_fdw on the remote node.
+ * Separate extension (contrib module?) is probably needed.
+ */
+ if (!use_core_aggregate)
+ schemaname = "partial";
+
/* Find aggregate name from aggfnoid which is a pg_proc entry */
! appendFunctionName(node->aggfnoid, context->buf, schemaname);
appendStringInfoChar(buf, '(');
/* Add DISTINCT */
*************** deparseAggref(Aggref *node, deparse_expr
*** 2829,2834 ****
--- 2944,2965 ----
}
appendStringInfoChar(buf, ')');
+
+ /*
+ * If special (non-core) aggregate is needed on the remote node, it can
+ * return data type for which postgres does not have textual format. cast
+ * to bytea seems to be an universal solution in such a case.
+ *
+ * TODO
+ *
+ * Consider checking if aggserialfn exists --- aggtranstype is used now
+ * because the info is easier to access. Also consider a requirement that
+ * the "remote aggregates" must return bytea, unless they return a
+ * primitive type. Thus we wouldn't have to deal with the cast here at
+ * all.
+ */
+ if (schemaname != NULL && node->aggtranstype == INTERNALOID)
+ appendStringInfoString(buf, "::bytea");
}
/*
*************** appendGroupByClause(List *tlist, deparse
*** 2952,2997 ****
*/
Assert(!query->groupingSets);
! foreach(lc, query->groupClause)
{
! SortGroupClause *grp = (SortGroupClause *) lfirst(lc);
! if (!first)
! appendStringInfoString(buf, ", ");
! first = false;
! deparseSortGroupClause(grp->tleSortGroupRef, tlist, context);
}
}
/*
! * Deparse ORDER BY clause according to the given pathkeys for given base
! * relation. From given pathkeys expressions belonging entirely to the given
! * base relation are obtained and deparsed.
*/
static void
! appendOrderByClause(List *pathkeys, deparse_expr_cxt *context)
{
! ListCell *lcell;
int nestlevel;
char *delim = " ";
- RelOptInfo *baserel = context->scanrel;
StringInfo buf = context->buf;
/* Make sure any constants in the exprs are printed portably */
nestlevel = set_transmission_modes();
appendStringInfoString(buf, " ORDER BY");
! foreach(lcell, pathkeys)
{
! PathKey *pathkey = lfirst(lcell);
! Expr *em_expr;
!
! em_expr = find_em_expr_for_rel(pathkey->pk_eclass, baserel);
! Assert(em_expr != NULL);
appendStringInfoString(buf, delim);
! deparseExpr(em_expr, context);
if (pathkey->pk_strategy == BTLessStrategyNumber)
appendStringInfoString(buf, " ASC");
else
--- 3083,3142 ----
*/
Assert(!query->groupingSets);
! /*
! * Due to aggregation push-down (which includes join aggregation on the
! * remote server) we should expect multiple TLEs containing different
! * expressions but having the same sortgroupref. The typical case is that
! * an equality join clause receives the input value (var) from each side
! * of a join, and each of these vars is needed above the join for a
! * different reason. That's why we don't use query->groupClause here.
! *
! * TODO Consider using get_grouping_expressions().
! */
! foreach(lc, tlist)
{
! TargetEntry *te = lfirst_node(TargetEntry, lc);
! Index sgref = te->ressortgroupref;
! /* Check whether this expression is part of GROUP BY clause */
! if (sgref &&
! get_sortgroupref_clause_noerr(sgref, query->groupClause))
! {
! if (!first)
! appendStringInfoString(buf, ", ");
! first = false;
! deparseSortGroupExpr(te->expr, context);
! }
}
}
/*
! * Deparse ORDER BY clause.
! *
! * "expressions" and "strategies" are lists of the expressions and the
! * corresponding pathkeys respectively.
*/
static void
! appendOrderByClause(deparse_expr_cxt *context, List *expressions, List *pathkeys)
{
! ListCell *l1, *l2;
int nestlevel;
char *delim = " ";
StringInfo buf = context->buf;
/* Make sure any constants in the exprs are printed portably */
nestlevel = set_transmission_modes();
appendStringInfoString(buf, " ORDER BY");
! Assert(list_length(expressions) == list_length(pathkeys));
! forboth(l1, expressions, l2, pathkeys)
{
! Expr *expr = (Expr *) lfirst(l1);
! PathKey *pathkey = lfirst_node(PathKey, l2);
appendStringInfoString(buf, delim);
! deparseExpr(expr, context);
if (pathkey->pk_strategy == BTLessStrategyNumber)
appendStringInfoString(buf, " ASC");
else
*************** appendOrderByClause(List *pathkeys, depa
*** 3012,3020 ****
* Deparses function name from given function oid.
*/
static void
! appendFunctionName(Oid funcid, deparse_expr_cxt *context)
{
- StringInfo buf = context->buf;
HeapTuple proctup;
Form_pg_proc procform;
const char *proname;
--- 3157,3164 ----
* Deparses function name from given function oid.
*/
static void
! appendFunctionName(Oid funcid, StringInfo buf, char *schemaname)
{
HeapTuple proctup;
Form_pg_proc procform;
const char *proname;
*************** appendFunctionName(Oid funcid, deparse_e
*** 3025,3035 ****
procform = (Form_pg_proc) GETSTRUCT(proctup);
/* Print schema name only if it's not pg_catalog */
! if (procform->pronamespace != PG_CATALOG_NAMESPACE)
{
! const char *schemaname;
!
! schemaname = get_namespace_name(procform->pronamespace);
appendStringInfo(buf, "%s.", quote_identifier(schemaname));
}
--- 3169,3179 ----
procform = (Form_pg_proc) GETSTRUCT(proctup);
/* Print schema name only if it's not pg_catalog */
! if (procform->pronamespace != PG_CATALOG_NAMESPACE ||
! schemaname != NULL)
{
! if (schemaname == NULL)
! schemaname = get_namespace_name(procform->pronamespace);
appendStringInfo(buf, "%s.", quote_identifier(schemaname));
}
*************** appendFunctionName(Oid funcid, deparse_e
*** 3049,3061 ****
static Node *
deparseSortGroupClause(Index ref, List *tlist, deparse_expr_cxt *context)
{
- StringInfo buf = context->buf;
TargetEntry *tle;
Expr *expr;
tle = get_sortgroupref_tle(ref, tlist);
expr = tle->expr;
if (expr && IsA(expr, Const))
{
/*
--- 3193,3214 ----
static Node *
deparseSortGroupClause(Index ref, List *tlist, deparse_expr_cxt *context)
{
TargetEntry *tle;
Expr *expr;
tle = get_sortgroupref_tle(ref, tlist);
expr = tle->expr;
+ deparseSortGroupExpr(expr, context);
+
+ return (Node *) expr;
+ }
+
+ static void
+ deparseSortGroupExpr(Expr *expr, deparse_expr_cxt *context)
+ {
+ StringInfo buf = context->buf;
+
if (expr && IsA(expr, Const))
{
/*
*************** deparseSortGroupClause(Index ref, List *
*** 3074,3081 ****
deparseExpr(expr, context);
appendStringInfoChar(buf, ')');
}
-
- return (Node *) expr;
}
--- 3227,3232 ----
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
new file mode 100644
index 683d641..5549b07
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
*************** SELECT t1.a,t1.b FROM fprt1 t1, LATERAL
*** 7525,7527 ****
--- 7525,7650 ----
(4 rows)
RESET enable_partition_wise_join;
+ -- ===================================================================
+ -- test aggregation of base relation or join
+ -- ===================================================================
+ CREATE TABLE customers(id int primary key, name varchar);
+ INSERT INTO customers VALUES (0, 'me'), (1, 'you'), (2, 'he'), (3, 'she');
+ CREATE TABLE products(id int, price float4) PARTITION BY RANGE (id);
+ CREATE TABLE products_0 (LIKE products);
+ INSERT INTO products_0(id, price) SELECT g.i, trunc(100 * random()) /
+ 10 + 1 FROM generate_series(1, 1000) AS g(i);
+ CREATE TABLE products_1 (LIKE products);
+ INSERT INTO products_1(id, price) SELECT g.i, trunc(100 * random()) /
+ 10 + 1 FROM generate_series(1001, 2000) AS g(i);
+ CREATE TABLE orders(customer_id int, product_id int) PARTITION BY RANGE
+ (product_id);
+ CREATE TABLE orders_0 (LIKE orders);
+ INSERT INTO orders_0(customer_id, product_id) SELECT trunc(2 * random()),
+ trunc(1000 * random()) FROM generate_series(1, 5000);
+ CREATE TABLE orders_1 (LIKE orders);
+ INSERT INTO orders_1(customer_id, product_id)
+ SELECT trunc(2 * random()) + 2, trunc(1000 * random()) + 1001 FROM
+ generate_series(1, 5000);
+ CREATE FOREIGN TABLE f_orders_0 PARTITION OF orders FOR VALUES FROM (1) TO
+ (1001) SERVER loopback OPTIONS (table_name 'orders_0');
+ CREATE FOREIGN TABLE f_orders_1 PARTITION OF orders FOR VALUES FROM (1001) TO
+ (2001) SERVER loopback OPTIONS (table_name 'orders_1');
+ CREATE FOREIGN TABLE f_products_0 PARTITION OF products FOR VALUES FROM (1) TO
+ (1001) SERVER loopback OPTIONS (table_name 'products_0');
+ CREATE FOREIGN TABLE f_products_1 PARTITION OF products FOR VALUES FROM (1001)
+ TO (2001) SERVER loopback OPTIONS (table_name 'products_1');
+ ALTER SERVER loopback OPTIONS (ADD use_remote_estimate 'true');
+ ALTER SERVER loopback2 OPTIONS (ADD use_remote_estimate 'true');
+ ANALYZE;
+ SET enable_agg_pushdown TO on;
+ SET enable_partition_wise_join TO on;
+ -- Perform the aggregation on partitions of "orders" , join the result to the
+ -- "customers" local table and finalize the aggregation.
+ EXPLAIN (VERBOSE on, COSTS off)
+ SELECT c.name, count(*)
+ FROM customers AS c, orders AS o
+ WHERE c.id = o.customer_id
+ GROUP BY c.id;
+ QUERY PLAN
+ --------------------------------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Output: c.name, count(*), c.id
+ Group Key: c.id
+ -> Nested Loop
+ Output: c.id, c.name, (PARTIAL count(*))
+ Inner Unique: true
+ -> Merge Append
+ Sort Key: o.customer_id
+ -> Foreign Scan
+ Output: o.customer_id, (PARTIAL count(*))
+ Remote SQL: SELECT customer_id, count(*) FROM public.orders_0 GROUP BY customer_id ORDER BY customer_id ASC NULLS LAST
+ -> Foreign Scan
+ Output: o_1.customer_id, (PARTIAL count(*))
+ Remote SQL: SELECT customer_id, count(*) FROM public.orders_1 GROUP BY customer_id ORDER BY customer_id ASC NULLS LAST
+ -> Index Scan using customers_pkey on public.customers c
+ Output: c.id, c.name
+ Index Cond: (c.id = o.customer_id)
+ (17 rows)
+
+ -- Perform partition-wise join of "orders" and "products", aggregate the
+ -- result at partition level, append the outputs of partitions and join it to
+ -- the "customers" local table.
+ EXPLAIN (VERBOSE on, COSTS off)
+ SELECT p.id, c.name, sum(p.price)
+ FROM customers AS c, orders AS o, products AS p
+ WHERE c.id = o.customer_id AND p.id = o.product_id
+ GROUP BY p.id, c.id
+ ORDER BY 3;
+ QUERY PLAN
+ -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Sort
+ Output: p.id, c.name, (PARTIAL sum(p.price)), c.id
+ Sort Key: (PARTIAL sum(p.price))
+ -> Result
+ Output: p.id, c.name, (PARTIAL sum(p.price)), c.id
+ -> Hash Join
+ Output: c.id, p.id, c.name, (PARTIAL sum(p.price))
+ Inner Unique: true
+ Hash Cond: (o.customer_id = c.id)
+ -> Append
+ -> Foreign Scan
+ Output: o.customer_id, p.id, (PARTIAL sum(p.price))
+ Relations: (public.f_orders_0 o) INNER JOIN (public.f_products_0 p)
+ Remote SQL: SELECT r5.customer_id, r8.id, sum(r8.price) FROM (public.orders_0 r5 INNER JOIN public.products_0 r8 ON (((r5.product_id = r8.id)))) GROUP BY r5.customer_id, r8.id
+ -> Foreign Scan
+ Output: o_1.customer_id, p_1.id, (PARTIAL sum(p_1.price))
+ Relations: (public.f_orders_1 o) INNER JOIN (public.f_products_1 p)
+ Remote SQL: SELECT r6.customer_id, r9.id, sum(r9.price) FROM (public.orders_1 r6 INNER JOIN public.products_1 r9 ON (((r6.product_id = r9.id)))) GROUP BY r6.customer_id, r9.id
+ -> Hash
+ Output: c.id, c.name
+ -> Seq Scan on public.customers c
+ Output: c.id, c.name
+ (22 rows)
+
+ -- The final aggregation is not necessary if the aggregated output of
+ -- partitions contains distinct set of grouping keys. Moreover, skip sort of
+ -- the result if the output of per-partition queries has the desired ordering
+ -- and so the sets can be merge-appended.
+ EXPLAIN (VERBOSE on, COSTS off)
+ SELECT p.id, sum(p.price)
+ FROM orders AS o, products AS p
+ WHERE p.id = o.product_id
+ GROUP BY p.id
+ ORDER BY 2 DESC;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Result
+ Output: p.id, (PARTIAL sum(p.price))
+ -> Merge Append
+ Sort Key: (PARTIAL sum(p.price)) DESC
+ -> Foreign Scan
+ Output: p.id, (PARTIAL sum(p.price))
+ Relations: (public.f_orders_0 o) INNER JOIN (public.f_products_0 p)
+ Remote SQL: SELECT r7.id, sum(r7.price) FROM (public.orders_0 r4 INNER JOIN public.products_0 r7 ON (((r4.product_id = r7.id)))) GROUP BY r7.id ORDER BY sum(r7.price) DESC NULLS FIRST
+ -> Foreign Scan
+ Output: p_1.id, (PARTIAL sum(p_1.price))
+ Relations: (public.f_orders_1 o) INNER JOIN (public.f_products_1 p)
+ Remote SQL: SELECT r8.id, sum(r8.price) FROM (public.orders_1 r5 INNER JOIN public.products_1 r8 ON (((r5.product_id = r8.id)))) GROUP BY r8.id ORDER BY sum(r8.price) DESC NULLS FIRST
+ (12 rows)
+
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
new file mode 100644
index 459c6b9..6085879
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
*************** PG_FUNCTION_INFO_V1(postgres_fdw_handler
*** 273,282 ****
*/
static void postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static void postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
static ForeignScan *postgresGetForeignPlan(PlannerInfo *root,
RelOptInfo *foreignrel,
Oid foreigntableid,
--- 273,283 ----
*/
static void postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid, bool grouped);
static void postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
static ForeignScan *postgresGetForeignPlan(PlannerInfo *root,
RelOptInfo *foreignrel,
Oid foreigntableid,
*************** static void postgresGetForeignJoinPaths(
*** 341,347 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static bool postgresRecheckForeignScan(ForeignScanState *node,
TupleTableSlot *slot);
static void postgresGetForeignUpperPaths(PlannerInfo *root,
--- 342,349 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped);
static bool postgresRecheckForeignScan(ForeignScanState *node,
TupleTableSlot *slot);
static void postgresGetForeignUpperPaths(PlannerInfo *root,
*************** static void postgresGetForeignUpperPaths
*** 352,363 ****
/*
* Helper functions
*/
static void estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *baserel,
List *join_conds,
List *pathkeys,
! double *p_rows, int *p_width,
! Cost *p_startup_cost, Cost *p_total_cost);
static void get_remote_estimate(const char *sql,
PGconn *conn,
double *rows,
--- 354,368 ----
/*
* Helper functions
*/
+ static void init_pgfdw_baserel_info(PlannerInfo *root, RelOptInfo *rel,
+ Oid foreigntableid, bool grouped);
static void estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *baserel,
List *join_conds,
+ List *order_by_exprs,
List *pathkeys,
! EstimateInfo * estimates,
! bool grouped);
static void get_remote_estimate(const char *sql,
PGconn *conn,
double *rows,
*************** static HeapTuple make_tuple_from_result_
*** 404,416 ****
static void conversion_error_callback(void *arg);
static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
! RelOptInfo *rel);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
static void add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path);
static void add_foreign_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
RelOptInfo *grouped_rel);
--- 409,426 ----
static void conversion_error_callback(void *arg);
static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra, bool grouped);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel);
+ static List *create_remote_agg_tlist(PlannerInfo *root,
+ RelOptInfo *grouped_rel,
+ PathTarget *grouping_target);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
! RelOptInfo *rel, bool grouped);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
static void add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path, bool grouped);
! static List *get_order_by_expressions(Relids relids, List *pathkeys,
! List *fdw_scan_tlist);
static void add_foreign_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
RelOptInfo *grouped_rel);
*************** postgres_fdw_handler(PG_FUNCTION_ARGS)
*** 485,567 ****
static void
postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
PgFdwRelationInfo *fpinfo;
! ListCell *lc;
! RangeTblEntry *rte = planner_rt_fetch(baserel->relid, root);
! const char *namespace;
! const char *relname;
! const char *refname;
/*
* We use PgFdwRelationInfo to pass various information to subsequent
* functions.
*/
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! baserel->fdw_private = (void *) fpinfo;
!
! /* Base foreign tables need to be pushed down always. */
! fpinfo->pushdown_safe = true;
!
! /* Look up foreign-table catalog info. */
! fpinfo->table = GetForeignTable(foreigntableid);
! fpinfo->server = GetForeignServer(fpinfo->table->serverid);
!
! /*
! * Extract user-settable option values. Note that per-table setting of
! * use_remote_estimate overrides per-server setting.
! */
! fpinfo->use_remote_estimate = false;
! fpinfo->fdw_startup_cost = DEFAULT_FDW_STARTUP_COST;
! fpinfo->fdw_tuple_cost = DEFAULT_FDW_TUPLE_COST;
! fpinfo->shippable_extensions = NIL;
! fpinfo->fetch_size = 100;
!
! apply_server_options(fpinfo);
! apply_table_options(fpinfo);
!
! /*
! * If the table or the server is configured to use remote estimates,
! * identify which user to do remote access as during planning. This
! * should match what ExecCheckRTEPerms() does. If we fail due to lack of
! * permissions, the query would have failed at runtime anyway.
! */
! if (fpinfo->use_remote_estimate)
{
! Oid userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
!
! fpinfo->user = GetUserMapping(userid, fpinfo->server->serverid);
}
- else
- fpinfo->user = NULL;
! /*
! * Identify which baserestrictinfo clauses can be sent to the remote
! * server and which can't.
! */
! classifyConditions(root, baserel, baserel->baserestrictinfo,
! &fpinfo->remote_conds, &fpinfo->local_conds);
! /*
! * Identify which attributes will need to be retrieved from the remote
! * server. These include all attrs needed for joins or final output, plus
! * all attrs used in the local_conds. (Note: if we end up using a
! * parameterized scan, it's possible that some of the join clauses will be
! * sent to the remote and thus we wouldn't really need to retrieve the
! * columns used in them. Doesn't seem worth detecting that case though.)
! */
! fpinfo->attrs_used = NULL;
! pull_varattnos((Node *) baserel->reltarget->exprs, baserel->relid,
! &fpinfo->attrs_used);
! foreach(lc, fpinfo->local_conds)
{
! RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
! pull_varattnos((Node *) rinfo->clause, baserel->relid,
! &fpinfo->attrs_used);
}
/*
* Compute the selectivity and cost of the local_conds, so we don't have
* to do it over again for each path. The best we can do for these
--- 495,545 ----
static void
postgresGetForeignRelSize(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
PgFdwRelationInfo *fpinfo;
! EstimateInfo *estimates;
/*
* We use PgFdwRelationInfo to pass various information to subsequent
* functions.
+ *
+ * This function can be called twice: first time for plain scan, second
+ * time for the grouped one. fpinfo should only be initialized once.
*/
! if (baserel->fdw_private == NULL)
{
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! baserel->fdw_private = fpinfo;
! init_pgfdw_baserel_info(root, baserel, foreigntableid, grouped);
}
! fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
! if (grouped)
{
! /*
! * baserestrictinfo must be evaluated below the aggregation, so all
! * its expressions must be remote.
! */
! if (fpinfo->local_conds != NIL)
! return;
! /*
! * Given that there are no local_conds and remote_conds are evaluated
! * on the foreign server, baserel->gpi->target is the only thing left
! * to check.
! *
! * XXX Do we need to verify that the target contains no
! * PlaceHolderVars?
! */
! if (!foreign_grouping_ok(root, baserel))
! return;
}
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+
/*
* Compute the selectivity and cost of the local_conds, so we don't have
* to do it over again for each path. The best we can do for these
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 580,587 ****
* when they are set to some sensible costs during one (usually the first)
* of the calls to estimate_path_cost_size().
*/
! fpinfo->rel_startup_cost = -1;
! fpinfo->rel_total_cost = -1;
/*
* If the table or the server is configured to use remote estimates,
--- 558,565 ----
* when they are set to some sensible costs during one (usually the first)
* of the calls to estimate_path_cost_size().
*/
! estimates->rel_startup_cost = -1;
! estimates->rel_total_cost = -1;
/*
* If the table or the server is configured to use remote estimates,
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 597,609 ****
* values in fpinfo so we don't need to do it again to generate the
* basic foreign path.
*/
! estimate_path_cost_size(root, baserel, NIL, NIL,
! &fpinfo->rows, &fpinfo->width,
! &fpinfo->startup_cost, &fpinfo->total_cost);
/* Report estimated baserel size to planner. */
! baserel->rows = fpinfo->rows;
! baserel->reltarget->width = fpinfo->width;
}
else
{
--- 575,591 ----
* values in fpinfo so we don't need to do it again to generate the
* basic foreign path.
*/
! estimate_path_cost_size(root, baserel, NIL, NIL, NIL, estimates,
! grouped);
/* Report estimated baserel size to planner. */
!
! /*
! * TODO If grouped, make sure baserel->gpi exists and store the values
! * there.
! */
! baserel->rows = estimates->rows;
! baserel->reltarget->width = estimates->width;
}
else
{
*************** postgresGetForeignRelSize(PlannerInfo *r
*** 628,661 ****
set_baserel_size_estimates(root, baserel);
/* Fill in basically-bogus cost estimates for use later. */
! estimate_path_cost_size(root, baserel, NIL, NIL,
! &fpinfo->rows, &fpinfo->width,
! &fpinfo->startup_cost, &fpinfo->total_cost);
}
-
- /*
- * Set the name of relation in fpinfo, while we are constructing it here.
- * It will be used to build the string describing the join relation in
- * EXPLAIN output. We can't know whether VERBOSE option is specified or
- * not, so always schema-qualify the foreign table name.
- */
- fpinfo->relation_name = makeStringInfo();
- namespace = get_namespace_name(get_rel_namespace(foreigntableid));
- relname = get_rel_name(foreigntableid);
- refname = rte->eref->aliasname;
- appendStringInfo(fpinfo->relation_name, "%s.%s",
- quote_identifier(namespace),
- quote_identifier(relname));
- if (*refname && strcmp(refname, relname) != 0)
- appendStringInfo(fpinfo->relation_name, " %s",
- quote_identifier(rte->eref->aliasname));
-
- /* No outer and inner relations. */
- fpinfo->make_outerrel_subquery = false;
- fpinfo->make_innerrel_subquery = false;
- fpinfo->lower_subquery_rels = NULL;
- /* Set the relation index. */
- fpinfo->relation_index = baserel->relid;
}
/*
--- 610,618 ----
set_baserel_size_estimates(root, baserel);
/* Fill in basically-bogus cost estimates for use later. */
! estimate_path_cost_size(root, baserel, NIL, NIL, NIL, estimates,
! grouped);
}
}
/*
*************** get_useful_ecs_for_relation(PlannerInfo
*** 775,781 ****
* to figure out which pathkeys to consider.
*/
static List *
! get_useful_pathkeys_for_relation(PlannerInfo *root, RelOptInfo *rel)
{
List *useful_pathkeys_list = NIL;
List *useful_eclass_list;
--- 732,739 ----
* to figure out which pathkeys to consider.
*/
static List *
! get_useful_pathkeys_for_relation(PlannerInfo *root, RelOptInfo *rel,
! bool grouped)
{
List *useful_pathkeys_list = NIL;
List *useful_eclass_list;
*************** get_useful_pathkeys_for_relation(Planner
*** 790,801 ****
if (root->query_pathkeys)
{
bool query_pathkeys_ok = true;
foreach(lc, root->query_pathkeys)
{
! PathKey *pathkey = (PathKey *) lfirst(lc);
! EquivalenceClass *pathkey_ec = pathkey->pk_eclass;
! Expr *em_expr;
/*
* The planner and executor don't have any clever strategy for
--- 748,762 ----
if (root->query_pathkeys)
{
bool query_pathkeys_ok = true;
+ bool sort_pathkeys_ok = true;
+ PathKey *pathkey;
+ EquivalenceClass *pathkey_ec;
+ Expr *em_expr;
foreach(lc, root->query_pathkeys)
{
! pathkey = lfirst_node(PathKey, lc);
! pathkey_ec = pathkey->pk_eclass;
/*
* The planner and executor don't have any clever strategy for
*************** get_useful_pathkeys_for_relation(Planner
*** 815,823 ****
break;
}
}
-
if (query_pathkeys_ok)
useful_pathkeys_list = list_make1(list_copy(root->query_pathkeys));
}
/*
--- 776,807 ----
break;
}
}
if (query_pathkeys_ok)
useful_pathkeys_list = list_make1(list_copy(root->query_pathkeys));
+
+ /*
+ * If the path is grouped, then it can be sorted by aggregates. These
+ * are only present in sort_pathkeys.
+ */
+ if (grouped && root->sort_pathkeys != root->query_pathkeys)
+ {
+ foreach(lc, root->sort_pathkeys)
+ {
+ pathkey = lfirst_node(PathKey, lc);
+ pathkey_ec = pathkey->pk_eclass;
+
+ if (pathkey_ec->ec_has_volatile ||
+ !(em_expr = find_em_expr_for_rel(pathkey_ec, rel)) ||
+ !is_foreign_expr(root, rel, em_expr))
+ {
+ sort_pathkeys_ok = false;
+ break;
+ }
+ }
+ if (sort_pathkeys_ok)
+ useful_pathkeys_list = lappend(useful_pathkeys_list,
+ list_copy(root->sort_pathkeys));
+ }
}
/*
*************** get_useful_pathkeys_for_relation(Planner
*** 884,895 ****
static void
postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
ForeignPath *path;
List *ppi_list;
ListCell *lc;
/*
* Create simplest ForeignScan path node and add it to baserel. This path
--- 868,966 ----
static void
postgresGetForeignPaths(PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) baserel->fdw_private;
ForeignPath *path;
+ PathTarget *target = NULL;
List *ppi_list;
ListCell *lc;
+ List *fdw_private;
+ EstimateInfo *estimates;
+
+ if (grouped)
+ {
+ /*
+ * The target must contain aggregates.
+ */
+ Assert(baserel->gpi != NULL);
+ target = baserel->gpi->target;
+
+ /*
+ * See postgresGetForeignRelSize().
+ */
+ if (fpinfo->local_conds != NIL)
+ return;
+
+ /*
+ * remote_conds will become fdw_recheck_quals for base relation, see
+ * postgresGetForeignPlan(). Check if the grouping target contains all
+ * vars necessary to evaluate the recheck quals.
+ */
+ if (fpinfo->remote_conds != NIL && IS_SIMPLE_REL(baserel))
+ {
+ foreach(lc, fpinfo->remote_conds)
+ {
+ RestrictInfo *ri = lfirst_node(RestrictInfo, lc);
+ Expr *clause = ri->clause;
+ List *vars = pull_var_clause((Node *) clause, 0);
+ ListCell *lc2;
+
+ /*
+ * Each var must exist in the grouping target.
+ */
+ foreach(lc2, vars)
+ {
+ ListCell *lc3;
+ bool found = false;
+ Var *var = lfirst_node(Var, lc2);
+
+ foreach(lc3, target->exprs)
+ {
+ Expr *gexpr = (Expr *) lfirst(lc3);
+
+ if (IsA(gexpr, Var))
+ {
+ Var *gvar = castNode(Var, gexpr);
+
+ if (gvar->varno == var->varno &&
+ gvar->varattno == var->varattno &&
+ gvar->varlevelsup == var->varlevelsup)
+ {
+ found = true;
+ break;
+ }
+ }
+ }
+ if (!found)
+ break;
+ }
+ list_free(vars);
+
+ /*
+ * Haven't checked all vars with success?
+ */
+ if (lc2 != NULL)
+ return;
+ }
+ return;
+ }
+
+ /*
+ * Wasn't it possible to construct remote query?
+ */
+ if (fpinfo->grouped_tlist == NIL)
+ return;
+ }
+
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+
+ /*
+ * Add information to the paths whether they are grouped or not. The 2nd
+ * element tells there's no ORDER BY clause.
+ */
+ fdw_private = list_make2(makeInteger(grouped ? 1 : 0), NIL);
/*
* Create simplest ForeignScan path node and add it to baserel. This path
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 899,916 ****
* to estimate cost and size of this path.
*/
path = create_foreignscan_path(root, baserel,
! NULL, /* default pathtarget */
! fpinfo->rows,
! fpinfo->startup_cost,
! fpinfo->total_cost,
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
/* Add paths with pathkeys */
! add_paths_with_pathkeys_for_rel(root, baserel, NULL);
/*
* If we're not using remote estimates, stop here. We have no way to
--- 970,995 ----
* to estimate cost and size of this path.
*/
path = create_foreignscan_path(root, baserel,
! target,
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no outer rel either */
NULL, /* no extra plan */
! fdw_private);
!
! /*
! * Grouped path essentially produces an unique set of grouping
! * keys.
! */
! if (grouped)
! make_uniquekeys_for_agg_path(root, (Path *) path);
!
! add_path(baserel, (Path *) path, grouped);
/* Add paths with pathkeys */
! add_paths_with_pathkeys_for_rel(root, baserel, NULL, grouped);
/*
* If we're not using remote estimates, stop here. We have no way to
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 921,926 ****
--- 1000,1013 ----
return;
/*
+ * Grouped parameterized path would lead to repeated aggregation, which is
+ * not too interesting. Since the rest deals only with parameterized
+ * paths, return now.
+ */
+ if (grouped)
+ return;
+
+ /*
* Thumb through all join clauses for the rel to identify which outer
* relations could supply one or more safe-to-send-to-remote join clauses.
* We'll build a parameterized path for each such outer relation.
*************** postgresGetForeignPaths(PlannerInfo *roo
*** 1052,1085 ****
foreach(lc, ppi_list)
{
ParamPathInfo *param_info = (ParamPathInfo *) lfirst(lc);
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
/* Get a cost estimate from the remote */
estimate_path_cost_size(root, baserel,
! param_info->ppi_clauses, NIL,
! &rows, &width,
! &startup_cost, &total_cost);
/*
* ppi_rows currently won't get looked at by anything, but still we
* may as well ensure that it matches our idea of the rowcount.
*/
! param_info->ppi_rows = rows;
/* Make the path */
path = create_foreignscan_path(root, baserel,
NULL, /* default pathtarget */
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
param_info->ppi_req_outer,
NULL,
! NIL); /* no fdw_private list */
! add_path(baserel, (Path *) path, false);
}
}
--- 1139,1170 ----
foreach(lc, ppi_list)
{
ParamPathInfo *param_info = (ParamPathInfo *) lfirst(lc);
! EstimateInfo est_param;
/* Get a cost estimate from the remote */
+ memset(&est_param, 0, sizeof(EstimateInfo));
estimate_path_cost_size(root, baserel,
! param_info->ppi_clauses, NIL, NIL,
! &est_param, grouped);
/*
* ppi_rows currently won't get looked at by anything, but still we
* may as well ensure that it matches our idea of the rowcount.
*/
! param_info->ppi_rows = est_param.rows;
/* Make the path */
path = create_foreignscan_path(root, baserel,
NULL, /* default pathtarget */
! est_param.rows,
! est_param.startup_cost,
! est_param.total_cost,
NIL, /* no pathkeys */
param_info->ppi_req_outer,
NULL,
! fdw_private);
!
! add_path(baserel, (Path *) path, grouped);
}
}
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1107,1119 ****
List *retrieved_attrs;
StringInfoData sql;
ListCell *lc;
if (IS_SIMPLE_REL(foreignrel))
{
! /*
! * For base relations, set scan_relid as the relid of the relation.
! */
! scan_relid = foreignrel->relid;
/*
* In a base-relation scan, we must apply the given scan_clauses.
--- 1192,1235 ----
List *retrieved_attrs;
StringInfoData sql;
ListCell *lc;
+ Value *grouped_val;
+ List *order_by_exprs;
+ bool grouped;
+
+ /*
+ * Retrieve the useful information from fdw_private.
+ */
+ Assert(list_length(best_path->fdw_private) == 2);
+ grouped_val = (Value *) linitial(best_path->fdw_private);
+ Assert(IsA(grouped_val, Integer));
+ grouped = intVal(grouped_val) == 1;
+ order_by_exprs = (List *) lsecond(best_path->fdw_private);
if (IS_SIMPLE_REL(foreignrel))
{
! if (!grouped)
! {
! /*
! * For base relations, set scan_relid as the relid of the
! * relation.
! */
! scan_relid = foreignrel->relid;
! }
! else
! {
! /*
! * XXX Something seems to be inconsistent here: deparseSelectSql()
! * would create the targetlist from foreignrel->gpi->target, but
! * not appendGroupByClause(). Try to refactor.
! */
! fdw_scan_tlist = fpinfo->grouped_tlist;
!
! /*
! * The scan tuple descriptor should be constructed from
! * fdw_scan_tlist rather than from that of the remote relation.
! */
! scan_relid = 0;
! }
/*
* In a base-relation scan, we must apply the given scan_clauses.
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1160,1165 ****
--- 1276,1283 ----
}
else
{
+ List *remote_src, *local_src;
+
/*
* Join relation or upper relation - set scan_relid to 0.
*/
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1176,1183 ****
* Instead we get the conditions to apply from the fdw_private
* structure.
*/
! remote_exprs = extract_actual_clauses(fpinfo->remote_conds, false);
! local_exprs = extract_actual_clauses(fpinfo->local_conds, false);
/*
* We leave fdw_recheck_quals empty in this case, since we never need
--- 1294,1313 ----
* Instead we get the conditions to apply from the fdw_private
* structure.
*/
! if (IS_UPPER_REL(foreignrel) &&
! (fpinfo->remote_conds_upper || fpinfo->local_conds_upper))
! {
! remote_src = fpinfo->remote_conds_upper;
! local_src = fpinfo->local_conds_upper;
! }
! else
! {
! remote_src = fpinfo->remote_conds;
! local_src = fpinfo->local_conds;
! }
!
! remote_exprs = extract_actual_clauses(remote_src, false);
! local_exprs = extract_actual_clauses(local_src, false);
/*
* We leave fdw_recheck_quals empty in this case, since we never need
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1191,1197 ****
*/
/* Build the list of columns to be fetched from the foreign server. */
! fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
/*
* Ensure that the outer plan produces a tuple whose descriptor
--- 1321,1330 ----
*/
/* Build the list of columns to be fetched from the foreign server. */
! if (!grouped)
! fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
! else
! fdw_scan_tlist = fpinfo->grouped_tlist;
/*
* Ensure that the outer plan produces a tuple whose descriptor
*************** postgresGetForeignPlan(PlannerInfo *root
*** 1238,1245 ****
*/
initStringInfo(&sql);
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
! remote_exprs, best_path->path.pathkeys,
! false, &retrieved_attrs, ¶ms_list);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
--- 1371,1380 ----
*/
initStringInfo(&sql);
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
! remote_exprs,
! order_by_exprs, best_path->path.pathkeys,
! false, &retrieved_attrs, ¶ms_list,
! grouped);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
*************** postgresExplainDirectModify(ForeignScanS
*** 2481,2486 ****
--- 2616,2739 ----
}
}
+ /*
+ * init_pgfdw_rel_info
+ * Initialize PgFdwRelationInfo and assign it to the base relation. That
+ * includes estimates, as well as separation of local expressions from
+ * remote ones.
+ */
+ static void
+ init_pgfdw_baserel_info(PlannerInfo *root, RelOptInfo *rel,
+ Oid foreigntableid, bool grouped)
+
+ {
+ const char *namespace;
+ const char *relname;
+ const char *refname;
+ PgFdwRelationInfo *fpinfo;
+ ListCell *lc;
+ RangeTblEntry *rte;
+
+ rte = planner_rt_fetch(rel->relid, root);
+ fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
+
+ /* Base foreign tables need to be pushed down always. */
+ fpinfo->pushdown_safe = true;
+
+ /* Look up foreign-table catalog info. */
+ fpinfo->table = GetForeignTable(foreigntableid);
+ fpinfo->server = GetForeignServer(fpinfo->table->serverid);
+
+ /*
+ * Extract user-settable option values. Note that per-table setting of
+ * use_remote_estimate overrides per-server setting.
+ */
+ fpinfo->use_remote_estimate = false;
+ fpinfo->fdw_startup_cost = DEFAULT_FDW_STARTUP_COST;
+ fpinfo->fdw_tuple_cost = DEFAULT_FDW_TUPLE_COST;
+ fpinfo->shippable_extensions = NIL;
+ fpinfo->fetch_size = 100;
+
+ apply_server_options(fpinfo);
+ apply_table_options(fpinfo);
+
+ /*
+ * If the table or the server is configured to use remote estimates,
+ * identify which user to do remote access as during planning. This
+ * should match what ExecCheckRTEPerms() does. If we fail due to lack of
+ * permissions, the query would have failed at runtime anyway.
+ */
+ if (fpinfo->use_remote_estimate)
+ {
+ Oid userid = rte->checkAsUser ? rte->checkAsUser :
+ GetUserId();
+
+ fpinfo->user = GetUserMapping(userid, fpinfo->server->serverid);
+ }
+ else
+ fpinfo->user = NULL;
+
+ /*
+ * Identify which baserestrictinfo clauses can be sent to the remote
+ * server and which can't.
+ */
+ classifyConditions(root, rel, rel->baserestrictinfo,
+ &fpinfo->remote_conds, &fpinfo->local_conds);
+
+ /*
+ * Identify which attributes will need to be retrieved from the remote
+ * server. These include all attrs needed for joins or final output, plus
+ * all attrs used in the local_conds. (Note: if we end up using a
+ * parameterized scan, it's possible that some of the join clauses will be
+ * sent to the remote and thus we wouldn't really need to retrieve the
+ * columns used in them. Doesn't seem worth detecting that case though.)
+ */
+ fpinfo->attrs_used = NULL;
+ if (!grouped)
+ {
+ pull_varattnos((Node *) rel->reltarget->exprs, rel->relid,
+ &fpinfo->attrs_used);
+ foreach(lc, fpinfo->local_conds)
+ {
+ RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
+
+ pull_varattnos((Node *) rinfo->clause, rel->relid,
+ &fpinfo->attrs_used);
+ }
+ }
+ else
+ {
+ /*
+ * deparseExplicitTargetList() is used for the grouped relation, thus
+ * no need to retrieve attrs_used.
+ */
+ }
+
+ /*
+ * Set the name of relation in fpinfo, while we are constructing it here.
+ * It will be used to build the string describing the join relation in
+ * EXPLAIN output. We can't know whether VERBOSE option is specified or
+ * not, so always schema-qualify the foreign table name.
+ */
+ fpinfo->relation_name = makeStringInfo();
+ namespace = get_namespace_name(get_rel_namespace(foreigntableid));
+ relname = get_rel_name(foreigntableid);
+ refname = rte->eref->aliasname;
+ appendStringInfo(fpinfo->relation_name, "%s.%s",
+ quote_identifier(namespace),
+ quote_identifier(relname));
+ if (*refname && strcmp(refname, relname) != 0)
+ appendStringInfo(fpinfo->relation_name, " %s",
+ quote_identifier(rte->eref->aliasname));
+
+ /* No outer and inner relations. */
+ fpinfo->make_outerrel_subquery = false;
+ fpinfo->make_innerrel_subquery = false;
+ fpinfo->lower_subquery_rels = NULL;
+ /* Set the relation index. */
+ fpinfo->relation_index = rel->relid;
+ }
+
/*
* estimate_path_cost_size
*************** static void
*** 2498,2506 ****
estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *foreignrel,
List *param_join_conds,
List *pathkeys,
! double *p_rows, int *p_width,
! Cost *p_startup_cost, Cost *p_total_cost)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
double rows;
--- 2751,2760 ----
estimate_path_cost_size(PlannerInfo *root,
RelOptInfo *foreignrel,
List *param_join_conds,
+ List *order_by_exprs,
List *pathkeys,
! EstimateInfo * estimates,
! bool grouped)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
double rows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2539,2546 ****
&remote_param_join_conds, &local_param_join_conds);
/* Build the list of columns to be fetched from the foreign server. */
! if (IS_JOIN_REL(foreignrel) || IS_UPPER_REL(foreignrel))
fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
else
fdw_scan_tlist = NIL;
--- 2793,2805 ----
&remote_param_join_conds, &local_param_join_conds);
/* Build the list of columns to be fetched from the foreign server. */
! if ((IS_JOIN_REL(foreignrel) && !grouped) || IS_UPPER_REL(foreignrel))
fdw_scan_tlist = build_tlist_to_deparse(foreignrel);
+ else if (grouped)
+ {
+ Assert(fpinfo->grouped_tlist != NIL);
+ fdw_scan_tlist = fpinfo->grouped_tlist;
+ }
else
fdw_scan_tlist = NIL;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2560,2567 ****
initStringInfo(&sql);
appendStringInfoString(&sql, "EXPLAIN ");
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
! remote_conds, pathkeys, false,
! &retrieved_attrs, NULL);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
--- 2819,2826 ----
initStringInfo(&sql);
appendStringInfoString(&sql, "EXPLAIN ");
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
! remote_conds, order_by_exprs, pathkeys, false,
! &retrieved_attrs, NULL, grouped);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2615,2629 ****
* bare scan each time. Instead, use the costs if we have cached them
* already.
*/
! if (fpinfo->rel_startup_cost > 0 && fpinfo->rel_total_cost > 0)
{
! startup_cost = fpinfo->rel_startup_cost;
! run_cost = fpinfo->rel_total_cost - fpinfo->rel_startup_cost;
}
else if (IS_JOIN_REL(foreignrel))
{
! PgFdwRelationInfo *fpinfo_i;
! PgFdwRelationInfo *fpinfo_o;
QualCost join_cost;
QualCost remote_conds_cost;
double nrows;
--- 2874,2890 ----
* bare scan each time. Instead, use the costs if we have cached them
* already.
*/
! if (estimates->rel_startup_cost > 0 && estimates->rel_total_cost > 0)
{
! startup_cost = estimates->rel_startup_cost;
! run_cost = estimates->rel_total_cost - estimates->rel_startup_cost;
}
else if (IS_JOIN_REL(foreignrel))
{
! PgFdwRelationInfo *fpinfo_i,
! *fpinfo_o;
! EstimateInfo *est_i,
! *est_o;
QualCost join_cost;
QualCost remote_conds_cost;
double nrows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2632,2641 ****
Assert(fpinfo->innerrel && fpinfo->outerrel);
fpinfo_i = (PgFdwRelationInfo *) fpinfo->innerrel->fdw_private;
fpinfo_o = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/* Estimate of number of rows in cross product */
! nrows = fpinfo_i->rows * fpinfo_o->rows;
/* Clamp retrieved rows estimate to at most size of cross product */
retrieved_rows = Min(retrieved_rows, nrows);
--- 2893,2904 ----
Assert(fpinfo->innerrel && fpinfo->outerrel);
fpinfo_i = (PgFdwRelationInfo *) fpinfo->innerrel->fdw_private;
+ est_i = !grouped ? &fpinfo_i->est_plain : &fpinfo_i->est_grouped;
fpinfo_o = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
+ est_o = !grouped ? &fpinfo_o->est_plain : &fpinfo_o->est_grouped;
/* Estimate of number of rows in cross product */
! nrows = est_i->rows * est_o->rows;
/* Clamp retrieved rows estimate to at most size of cross product */
retrieved_rows = Min(retrieved_rows, nrows);
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2659,2665 ****
* tables) since we do not know what strategy the foreign server
* is going to use.
*/
! startup_cost = fpinfo_i->rel_startup_cost + fpinfo_o->rel_startup_cost;
startup_cost += join_cost.startup;
startup_cost += remote_conds_cost.startup;
startup_cost += fpinfo->local_conds_cost.startup;
--- 2922,2928 ----
* tables) since we do not know what strategy the foreign server
* is going to use.
*/
! startup_cost = est_i->rel_startup_cost + est_o->rel_startup_cost;
startup_cost += join_cost.startup;
startup_cost += remote_conds_cost.startup;
startup_cost += fpinfo->local_conds_cost.startup;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2679,2695 ****
* 4. Run time cost of applying nonpushable other clauses locally
* on the result fetched from the foreign server.
*/
! run_cost = fpinfo_i->rel_total_cost - fpinfo_i->rel_startup_cost;
! run_cost += fpinfo_o->rel_total_cost - fpinfo_o->rel_startup_cost;
run_cost += nrows * join_cost.per_tuple;
nrows = clamp_row_est(nrows * fpinfo->joinclause_sel);
run_cost += nrows * remote_conds_cost.per_tuple;
run_cost += fpinfo->local_conds_cost.per_tuple * retrieved_rows;
}
! else if (IS_UPPER_REL(foreignrel))
{
PgFdwRelationInfo *ofpinfo;
! PathTarget *ptarget = root->upper_targets[UPPERREL_GROUP_AGG];
AggClauseCosts aggcosts;
double input_rows;
int numGroupCols;
--- 2942,2970 ----
* 4. Run time cost of applying nonpushable other clauses locally
* on the result fetched from the foreign server.
*/
! run_cost = est_i->rel_total_cost - est_i->rel_startup_cost;
! run_cost += est_o->rel_total_cost - est_o->rel_startup_cost;
run_cost += nrows * join_cost.per_tuple;
nrows = clamp_row_est(nrows * fpinfo->joinclause_sel);
run_cost += nrows * remote_conds_cost.per_tuple;
run_cost += fpinfo->local_conds_cost.per_tuple * retrieved_rows;
}
!
! /*
! * TODO Consider a separate branch for
! *
! * (IS_SIMPLE_REL(foreignrel) && * grouped)
! *
! * and use foreignrel->gpi->target in it instead of
! * fpinfo->grouped_tlist.
! */
! else if (IS_UPPER_REL(foreignrel) ||
! (IS_SIMPLE_REL(foreignrel) && grouped)
! )
{
PgFdwRelationInfo *ofpinfo;
! EstimateInfo *est;
! PathTarget *ptarget;
AggClauseCosts aggcosts;
double input_rows;
int numGroupCols;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2707,2717 ****
* considering remote and local conditions for costing.
*/
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/* Get rows and width from input rel */
! input_rows = ofpinfo->rows;
! width = ofpinfo->width;
/* Collect statistics about aggregates for estimating costs. */
MemSet(&aggcosts, 0, sizeof(AggClauseCosts));
--- 2982,3001 ----
* considering remote and local conditions for costing.
*/
! if (IS_UPPER_REL(foreignrel))
! {
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
! est = &ofpinfo->est_plain;
! }
! else
! {
! ofpinfo = fpinfo;
! est = &ofpinfo->est_grouped;
! }
/* Get rows and width from input rel */
! input_rows = est->rows;
! width = est->width;
/* Collect statistics about aggregates for estimating costs. */
MemSet(&aggcosts, 0, sizeof(AggClauseCosts));
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2736,2741 ****
--- 3020,3033 ----
*/
rows = retrieved_rows = numGroups;
+ if (IS_UPPER_REL(foreignrel))
+ ptarget = root->upper_targets[UPPERREL_GROUP_AGG];
+ else
+ {
+ Assert(foreignrel->gpi != NULL);
+ ptarget = foreignrel->gpi->target;
+ }
+
/*-----
* Startup cost includes:
* 1. Startup cost for underneath input * relation
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2743,2749 ****
* 3. Startup cost for PathTarget eval
*-----
*/
! startup_cost = ofpinfo->rel_startup_cost;
startup_cost += aggcosts.transCost.startup;
startup_cost += aggcosts.transCost.per_tuple * input_rows;
startup_cost += (cpu_operator_cost * numGroupCols) * input_rows;
--- 3035,3041 ----
* 3. Startup cost for PathTarget eval
*-----
*/
! startup_cost = est->rel_startup_cost;
startup_cost += aggcosts.transCost.startup;
startup_cost += aggcosts.transCost.per_tuple * input_rows;
startup_cost += (cpu_operator_cost * numGroupCols) * input_rows;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2756,2762 ****
* 3. PathTarget eval cost for each output row
*-----
*/
! run_cost = ofpinfo->rel_total_cost - ofpinfo->rel_startup_cost;
run_cost += aggcosts.finalCost * numGroups;
run_cost += cpu_tuple_cost * numGroups;
run_cost += ptarget->cost.per_tuple * numGroups;
--- 3048,3054 ----
* 3. PathTarget eval cost for each output row
*-----
*/
! run_cost = est->rel_total_cost - est->rel_startup_cost;
run_cost += aggcosts.finalCost * numGroups;
run_cost += cpu_tuple_cost * numGroups;
run_cost += ptarget->cost.per_tuple * numGroups;
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2809,2816 ****
*/
if (pathkeys == NIL && param_join_conds == NIL)
{
! fpinfo->rel_startup_cost = startup_cost;
! fpinfo->rel_total_cost = total_cost;
}
/*
--- 3101,3108 ----
*/
if (pathkeys == NIL && param_join_conds == NIL)
{
! estimates->rel_startup_cost = startup_cost;
! estimates->rel_total_cost = total_cost;
}
/*
*************** estimate_path_cost_size(PlannerInfo *roo
*** 2825,2834 ****
total_cost += cpu_tuple_cost * retrieved_rows;
/* Return results. */
! *p_rows = rows;
! *p_width = width;
! *p_startup_cost = startup_cost;
! *p_total_cost = total_cost;
}
/*
--- 3117,3126 ----
total_cost += cpu_tuple_cost * retrieved_rows;
/* Return results. */
! estimates->rows = rows;
! estimates->width = width;
! estimates->startup_cost = startup_cost;
! estimates->total_cost = total_cost;
}
/*
*************** postgresImportForeignSchema(ImportForeig
*** 4057,4075 ****
* Assess whether the join between inner and outer relations can be pushed down
* to the foreign server. As a side effect, save information we obtain in this
* function to PgFdwRelationInfo passed in.
*/
static bool
foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra)
{
! PgFdwRelationInfo *fpinfo;
! PgFdwRelationInfo *fpinfo_o;
! PgFdwRelationInfo *fpinfo_i;
ListCell *lc;
List *joinclauses;
/*
* We support pushing down INNER, LEFT, RIGHT and FULL OUTER joins.
* Constructing queries representing SEMI and ANTI joins is hard, hence
* not considered right now.
--- 4349,4378 ----
* Assess whether the join between inner and outer relations can be pushed down
* to the foreign server. As a side effect, save information we obtain in this
* function to PgFdwRelationInfo passed in.
+ *
+ * Caller expects that the "grouped" argument does not affect the result, so
+ * it's enough to call the function only once per relation. XXX As that
+ * argument is only used in Assert() statement, shouldn't it be removed?
*/
static bool
foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinPathExtraData *extra, bool grouped)
{
! PgFdwRelationInfo *fpinfo,
! *fpinfo_o,
! *fpinfo_i;
ListCell *lc;
List *joinclauses;
/*
+ * TODO
+ *
+ * Put the (join-relevant) initial checks of foreign_grouping_ok into a
+ * function and call it here.
+ */
+
+ /*
* We support pushing down INNER, LEFT, RIGHT and FULL OUTER joins.
* Constructing queries representing SEMI and ANTI joins is hard, hence
* not considered right now.
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4139,4145 ****
--- 4442,4458 ----
if (is_remote_clause)
fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
else
+ {
+ /*
+ * We could return in the grouped case because all clauses
+ * must be evaluated below the grouping plan (Thus if the
+ * grouping is remote, those clauses must be remote too.)
+ * However the same PgFdwRelationInfo is used for both grouped
+ * and non-grouped case and we don't want to force caller to
+ * process the non-grouped case first.
+ */
fpinfo->local_conds = lappend(fpinfo->local_conds, rinfo);
+ }
}
}
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4158,4166 ****
PlaceHolderInfo *phinfo = lfirst(lc);
Relids relids = joinrel->relids;
! if (bms_is_subset(phinfo->ph_eval_at, relids) &&
! bms_nonempty_difference(relids, phinfo->ph_eval_at))
! return false;
}
/* Save the join clauses, for later use. */
--- 4471,4496 ----
PlaceHolderInfo *phinfo = lfirst(lc);
Relids relids = joinrel->relids;
! if (bms_is_subset(phinfo->ph_eval_at, relids))
! {
! if (bms_nonempty_difference(relids, phinfo->ph_eval_at))
! {
! #ifdef USE_ASSERT_CHECKING
! if (grouped)
! {
! /*
! * Grouped join shouldn't have PlaceHolderVar even in its
! * own targetlist. It'd mean that the output of partial
! * aggregation can be set to NULL before it gets to the
! * input of the final aggregation.
! */
! Assert(false);
! }
! #endif /* USE_ASSERT_CHECKING */
!
! return false;
! }
! }
}
/* Save the join clauses, for later use. */
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4283,4296 ****
fpinfo->user = NULL;
/*
- * Set cached relation costs to some negative value, so that we can detect
- * when they are set to some sensible costs, during one (usually the
- * first) of the calls to estimate_path_cost_size().
- */
- fpinfo->rel_startup_cost = -1;
- fpinfo->rel_total_cost = -1;
-
- /*
* Set the string describing this join relation to be used in EXPLAIN
* output of corresponding ForeignScan.
*/
--- 4613,4618 ----
*************** foreign_join_ok(PlannerInfo *root, RelOp
*** 4315,4354 ****
static void
add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path)
{
List *useful_pathkeys_list = NIL; /* List of all pathkeys */
! ListCell *lc;
! useful_pathkeys_list = get_useful_pathkeys_for_relation(root, rel);
/* Create one path for each set of pathkeys we found above. */
! foreach(lc, useful_pathkeys_list)
{
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
! List *useful_pathkeys = lfirst(lc);
! estimate_path_cost_size(root, rel, NIL, useful_pathkeys,
! &rows, &width, &startup_cost, &total_cost);
! add_path(rel, (Path *)
! create_foreignscan_path(root, rel,
! NULL,
! rows,
! startup_cost,
! total_cost,
! useful_pathkeys,
! NULL,
! epq_path,
! NIL),
! false);
}
}
/*
* Parse options from foreign server and apply them to fpinfo.
*
* New options might also require tweaking merge_fdw_options().
--- 4637,4858 ----
static void
add_paths_with_pathkeys_for_rel(PlannerInfo *root, RelOptInfo *rel,
! Path *epq_path, bool grouped)
{
List *useful_pathkeys_list = NIL; /* List of all pathkeys */
! ListCell *l;
! PathTarget *target = NULL;
! PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
! List *fdw_scan_tlist = NIL;
! useful_pathkeys_list = get_useful_pathkeys_for_relation(root, rel,
! grouped);
!
! /*
! * If the path should be grouped, not all the ECs are necessarily present
! * in the target: the EC member applicable to the parent relation can in
! * fact be an aggregate argument which does not otherwise appear in the
! * query output.
! */
! if (grouped)
! fdw_scan_tlist = fpinfo->grouped_tlist;
!
! /*
! * Grouped foreign scan should emit tuples according to the grouped
! * target, i.e. the one containing GroupedVars.
! */
! if (grouped)
! {
! Assert(rel->gpi != NULL);
! target = rel->gpi->target;
! }
/* Create one path for each set of pathkeys we found above. */
! foreach(l, useful_pathkeys_list)
{
! List *useful_pathkeys = lfirst(l);
! EstimateInfo estimates;
! List *order_by_exprs = NIL;
! List *fdw_private;
! Path *path;
! /*
! * Perform preprocessing specific to grouped paths or reject the
! * pathkeys if they cannot be used on the remote database.
! */
! order_by_exprs = get_order_by_expressions(rel->relids,
! useful_pathkeys,
! fdw_scan_tlist);
! /*
! * Skip these pathkeys if the remote database cannot perform the
! * appropriate sort.
! */
! if (order_by_exprs == NIL)
! continue;
!
! /*
! * The cached values are not useful for join path.
! */
! estimates.rel_startup_cost = -1;
! estimates.rel_total_cost = -1;
!
! estimate_path_cost_size(root, rel, NIL, order_by_exprs,
! useful_pathkeys, &estimates, grouped);
!
! /*
! * Add information to the paths whether they are grouped or not. And if it
! * is, pass it the expressions for the ORDER BY clause.
! */
! fdw_private = list_make2(makeInteger(grouped ? 1 : 0),
! order_by_exprs);
!
! path = (Path *) create_foreignscan_path(root, rel,
! target,
! estimates.rows,
! estimates.startup_cost,
! estimates.total_cost,
! useful_pathkeys,
! NULL,
! epq_path,
! fdw_private);
!
! /*
! * Grouped path essentially produces an unique set of grouping
! * keys.
! */
! if (grouped)
! make_uniquekeys_for_agg_path(root, path);
!
! add_path(rel, path, grouped);
}
}
/*
+ * Create a list of expressions to form ORDER BY clause on the remote
+ * database. Pass fdw_scan_tlist if the pathkeys possibly contain expressions
+ * not present in the targetlist. Such expressions will be ignored.
+ */
+ static List *
+ get_order_by_expressions(Relids relids, List *pathkeys, List *fdw_scan_tlist)
+ {
+ ListCell *l1;
+ List *result = NIL;
+
+ foreach(l1, pathkeys)
+ {
+ ListCell *l2;
+ PathKey *pathkey = lfirst_node(PathKey, l1);
+ Expr *em_expr = NULL;
+ Aggref *aggref_copy = NULL;
+
+ /*
+ * Since the targetlist can contain EC-derived expressions, extra
+ * effort is needed to ensure that ORDER BY clause references
+ * expression actually contained in the targetlist.
+ */
+ foreach(l2, pathkey->pk_eclass->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+
+ if (bms_is_subset(em->em_relids, relids))
+ {
+ ListCell *l3;
+
+ if (fdw_scan_tlist)
+ {
+ /*
+ * The EM can be derived from an expression present in the
+ * targetlist, but is not necessarily in the list
+ * itself. However the ORDER BY clause used on the remote
+ * server can only contain the "real" targetlist
+ * expression.
+ */
+ foreach(l3, fdw_scan_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, l3);
+ Expr *expr = te->expr;
+
+ /*
+ * Even if the query has Aggref nested in a generic
+ * expression, fdw_scan_tlist should only contain
+ * separate entry for the Aggref itself. Thus we don't
+ * need to check the expressions recursively.
+ */
+ if (IsA(expr, Aggref) && IsA(em->em_expr, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, expr);
+ Aggref *aggref_em;
+
+ /*
+ * It makes no sense to sort by an aggregate if it
+ * requires finalization. First, the finalized
+ * values might end up sorted in a different
+ * order, second, the transient state might not be
+ * sortable at all.
+ */
+ if (aggref->aggfinalfn != InvalidOid)
+ {
+ list_free(result);
+ return NIL;
+ }
+
+ aggref_em = (Aggref *) em->em_expr;
+
+ /*
+ * The EC members are simple aggregates, so adjust
+ * the target expression to make match
+ * possible. Flat copy is sufficient for this
+ * purpose.
+ */
+ Assert(aggref_em->aggsplit == AGGSPLIT_SIMPLE);
+ Assert(aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ aggref_copy = makeNode(Aggref);
+ memcpy(aggref_copy, aggref, sizeof(Aggref));
+ aggref_copy->aggsplit = AGGSPLIT_SIMPLE;
+ aggref_copy->aggtype = aggref_em->aggtype;
+ expr = (Expr *) aggref_copy;
+ }
+
+ if (equal(expr, em->em_expr))
+ {
+ em_expr = em->em_expr;
+ break;
+ }
+ }
+ }
+ else
+ em_expr = em->em_expr;
+ }
+
+ /*
+ * Target expression found for the current pathkey?
+ */
+ if (em_expr != NULL)
+ break;
+ }
+
+ /*
+ * A single missing expression makes the pathkey useless. This is
+ * specific to grouped relation because it can only emit grouping
+ * expressions and aggregates.
+ */
+ if (em_expr == NULL)
+ {
+ list_free(result);
+ return NIL;
+ }
+
+ result = lappend(result, em_expr);
+
+ if (aggref_copy)
+ pfree(aggref_copy);
+ }
+
+ return result;
+ }
+
+ /*
* Parse options from foreign server and apply them to fpinfo.
*
* New options might also require tweaking merge_fdw_options().
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4462,4496 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
PgFdwRelationInfo *fpinfo;
ForeignPath *joinpath;
- double rows;
- int width;
- Cost startup_cost;
- Cost total_cost;
Path *epq_path; /* Path to create plan to be executed when
* EvalPlanQual gets triggered. */
! /*
! * Skip if this join combination has been considered already.
! */
! if (joinrel->fdw_private)
! return;
! /*
! * Create unfinished PgFdwRelationInfo entry which is used to indicate
! * that the join relation is already considered, so that we won't waste
! * time in judging safety of join pushdown and adding the same paths again
! * if found safe. Once we know that this join can be pushed down, we fill
! * the entry.
! */
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! fpinfo->pushdown_safe = false;
! joinrel->fdw_private = fpinfo;
! /* attrs_used is only for base relations. */
! fpinfo->attrs_used = NULL;
/*
* If there is a possibility that EvalPlanQual will be executed, we need
--- 4966,5016 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
PgFdwRelationInfo *fpinfo;
+ EstimateInfo *estimates;
ForeignPath *joinpath;
Path *epq_path; /* Path to create plan to be executed when
* EvalPlanQual gets triggered. */
+ bool first_time;
+ PathTarget *target = NULL;
+ List *fdw_private;
! if (joinrel->fdw_private == NULL)
! {
! /*
! * First time through.
! *
! * Create unfinished PgFdwRelationInfo entry which is used to indicate
! * that the join relation is already considered, so that we won't
! * waste time in judging safety of join pushdown and adding the same
! * paths again if found safe. Once we know that this join can be
! * pushed down, we fill the entry.
! */
! fpinfo = (PgFdwRelationInfo *) palloc0(sizeof(PgFdwRelationInfo));
! fpinfo->pushdown_safe = false;
! joinrel->fdw_private = fpinfo;
! /* attrs_used is only for base relations. */
! fpinfo->attrs_used = NULL;
! first_time = true;
! }
! else
! {
! fpinfo = (PgFdwRelationInfo *) joinrel->fdw_private;
! first_time = false;
!
! if (!fpinfo->pushdown_safe)
! {
! /*
! * The function was already called but some test failed. No reason
! * to see that failure again.
! */
! return;
! }
! }
/*
* If there is a possibility that EvalPlanQual will be executed, we need
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4501,4527 ****
* dominate the only suitable local path available. We also do it before
* calling foreign_join_ok(), since that function updates fpinfo and marks
* it as pushable if the join is found to be pushable.
*/
! if (root->parse->commandType == CMD_DELETE ||
! root->parse->commandType == CMD_UPDATE ||
! root->rowMarks)
{
! epq_path = GetExistingLocalJoinPath(joinrel);
! if (!epq_path)
{
! elog(DEBUG3, "could not push down foreign join because a local path suitable for EPQ checks was not found");
return;
}
}
- else
- epq_path = NULL;
! if (!foreign_join_ok(root, joinrel, jointype, outerrel, innerrel, extra))
! {
! /* Free path required for EPQ if we copied one; we don't need it now */
! if (epq_path)
! pfree(epq_path);
return;
}
/*
--- 5021,5098 ----
* dominate the only suitable local path available. We also do it before
* calling foreign_join_ok(), since that function updates fpinfo and marks
* it as pushable if the join is found to be pushable.
+ *
+ * The following tests should only be performed once --- repeated, the are
+ * supposed to yield the same result. (If anything failed initially,
+ * pushdown_safe should have caused return above.)
*/
! if (first_time)
{
! if (root->parse->commandType == CMD_DELETE ||
! root->parse->commandType == CMD_UPDATE ||
! root->rowMarks)
{
! epq_path = GetExistingLocalJoinPath(joinrel);
! if (!epq_path)
! {
! elog(DEBUG3, "could not push down foreign join because a local path suitable for EPQ checks was not found");
! return;
! }
! }
! else
! epq_path = NULL;
!
! if (!foreign_join_ok(root, joinrel, jointype, outerrel, innerrel, extra,
! grouped))
! {
! /*
! * Free path required for EPQ if we copied one; we don't need it
! * now
! */
! if (epq_path)
! pfree(epq_path);
return;
}
}
! /*
! * Local expressions would be evaluated above the grouping plan, which
! * will be remote.
! */
! if (grouped && fpinfo->local_conds != NIL)
return;
+
+ /*
+ * This function should not be called repeatedly wit the same value of
+ * "grouped", but if it happened, make sure the "upper conditions" do not
+ * get duplicated or used at all if !grouped.
+ */
+ if (fpinfo->remote_conds_upper)
+ {
+ list_free(fpinfo->remote_conds_upper);
+ fpinfo->remote_conds_upper = NIL;
+ }
+ if (fpinfo->local_conds_upper)
+ {
+ list_free(fpinfo->local_conds_upper);
+ fpinfo->local_conds_upper = NIL;
+ }
+
+ if (grouped)
+ {
+ /*
+ * Create targelist to be used for the remote grouping.
+ */
+ Assert(joinrel->gpi != NULL);
+ fpinfo->grouped_tlist = create_remote_agg_tlist(root, joinrel,
+ joinrel->gpi->target);
+
+ /*
+ * Couldn't push any GROUP BY expression or aggregate to the remote
+ * server?
+ */
+ if (fpinfo->grouped_tlist == NIL)
+ return;
}
/*
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4548,4563 ****
0, fpinfo->jointype,
extra->sjinfo);
/* Estimate costs for bare join relation */
! estimate_path_cost_size(root, joinrel, NIL, NIL, &rows,
! &width, &startup_cost, &total_cost);
/* Now update this information in the joinrel */
! joinrel->rows = rows;
! joinrel->reltarget->width = width;
! fpinfo->rows = rows;
! fpinfo->width = width;
! fpinfo->startup_cost = startup_cost;
! fpinfo->total_cost = total_cost;
/*
* Create a new join path and add it to the joinrel which represents a
--- 5119,5163 ----
0, fpinfo->jointype,
extra->sjinfo);
+ /*
+ * Set cached relation costs to some negative value, so that we can detect
+ * when they are set to some sensible costs, during one (usually the
+ * first) of the calls to estimate_path_cost_size().
+ */
+ estimates = !grouped ? &fpinfo->est_plain : &fpinfo->est_grouped;
+ estimates->rel_startup_cost = -1;
+ estimates->rel_total_cost = -1;
+
/* Estimate costs for bare join relation */
!
! /*
! * TODO Include remote_conds_upper and local_conds_upper into the
! * estimates.
! */
! estimate_path_cost_size(root, joinrel, NIL, NIL, NIL, estimates, grouped);
/* Now update this information in the joinrel */
!
! /*
! * TODO If grouped, make sure baserel->gpi exists and store the values
! * there.
! */
! joinrel->rows = estimates->rows;
! joinrel->reltarget->width = estimates->width;
!
! if (grouped)
! {
! /*
! * The target must contain aggregates.
! */
! Assert(joinrel->gpi != NULL);
! target = joinrel->gpi->target;
! }
!
! /*
! * Add information to the paths whether they are grouped or not. Pass NIL
! * for ordering expressions.
! */
! fdw_private = list_make2(makeInteger(grouped ? 1 : 0), NIL);
/*
* Create a new join path and add it to the joinrel which represents a
*************** postgresGetForeignJoinPaths(PlannerInfo
*** 4565,4584 ****
*/
joinpath = create_foreignscan_path(root,
joinrel,
! NULL, /* default pathtarget */
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
epq_path,
! NIL); /* no fdw_private */
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, false);
/* Consider pathkeys for the join relation */
! add_paths_with_pathkeys_for_rel(root, joinrel, epq_path);
/* XXX Consider parameterized paths for the join relation */
}
--- 5165,5191 ----
*/
joinpath = create_foreignscan_path(root,
joinrel,
! target, /* default pathtarget */
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
epq_path,
! fdw_private); /* no fdw_private */
!
! /*
! * Grouped path essentially produces an unique set of grouping
! * keys.
! */
! if (grouped)
! make_uniquekeys_for_agg_path(root, (Path *) joinpath);
/* Add generated path into joinrel by add_path(). */
! add_path(joinrel, (Path *) joinpath, grouped);
/* Consider pathkeys for the join relation */
! add_paths_with_pathkeys_for_rel(root, joinrel, epq_path, grouped);
/* XXX Consider parameterized paths for the join relation */
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4594,4618 ****
Query *query = root->parse;
PathTarget *grouping_target;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
! PgFdwRelationInfo *ofpinfo;
! List *aggvars;
! ListCell *lc;
! int i;
! List *tlist = NIL;
/* Grouping Sets are not pushable */
if (query->groupingSets)
return false;
/* Get the fpinfo of the underlying scan relation. */
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/*
* If underneath input relation has any local conditions, those conditions
* are required to be applied before performing aggregation. Hence the
* aggregate cannot be pushed down.
*/
! if (ofpinfo->local_conds)
return false;
/*
--- 5201,5223 ----
Query *query = root->parse;
PathTarget *grouping_target;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
! PgFdwRelationInfo *ofpinfo = NULL;
/* Grouping Sets are not pushable */
if (query->groupingSets)
return false;
/* Get the fpinfo of the underlying scan relation. */
! if (IS_UPPER_REL(grouped_rel))
! ofpinfo = (PgFdwRelationInfo *) fpinfo->outerrel->fdw_private;
/*
* If underneath input relation has any local conditions, those conditions
* are required to be applied before performing aggregation. Hence the
* aggregate cannot be pushed down.
*/
! if ((IS_UPPER_REL(grouped_rel) && ofpinfo->local_conds) ||
! fpinfo->local_conds)
return false;
/*
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4623,4637 ****
* different from those in the plan's targetlist. Use a copy of path
* target to record the new sortgrouprefs.
*/
! grouping_target = copy_pathtarget(root->upper_targets[UPPERREL_GROUP_AGG]);
/*
! * Evaluate grouping targets and check whether they are safe to push down
! * to the foreign side. All GROUP BY expressions will be part of the
! * grouping target and thus there is no need to evaluate it separately.
! * While doing so, add required expressions into target list which can
! * then be used to pass to foreign server.
*/
i = 0;
foreach(lc, grouping_target->exprs)
{
--- 5228,5296 ----
* different from those in the plan's targetlist. Use a copy of path
* target to record the new sortgrouprefs.
*/
! if (IS_UPPER_REL(grouped_rel))
! grouping_target = root->upper_targets[UPPERREL_GROUP_AGG];
! else
! {
! /*
! * FDW should not be asked to generate grouped paths if the simple or
! * join relation has no relevant target.
! */
! Assert(grouped_rel->gpi != NULL);
! Assert(grouped_rel->gpi->target != NULL);
!
! grouping_target = grouped_rel->gpi->target;
! }
/*
! * Create targetlist to be sent to the remote server.
*/
+ fpinfo->grouped_tlist = create_remote_agg_tlist(root, grouped_rel,
+ grouping_target);
+ if (fpinfo->grouped_tlist == NIL)
+ return false;
+
+ /* Safe to pushdown */
+ fpinfo->pushdown_safe = true;
+
+ /*
+ * Set the string describing this grouped relation to be used in EXPLAIN
+ * output of corresponding ForeignScan.
+ */
+ if (ofpinfo != NULL)
+ {
+ fpinfo->relation_name = makeStringInfo();
+ appendStringInfo(fpinfo->relation_name, "Aggregate on (%s)",
+ ofpinfo->relation_name->data);
+ }
+
+ return true;
+ }
+
+ /*
+ * Evaluate grouping targets and check whether they are safe to push down to
+ * the foreign side. All GROUP BY expressions will be part of the grouping
+ * target and thus there is no need to evaluate it separately. While doing
+ * so, add required expressions into target list which can then be used to
+ * pass to foreign server.
+ *
+ * NIL is returned if the remote aggregation appears to be impossible.
+ *
+ * Note that the function can add items to both remote_conds_upper and
+ * local_conds_upper lists of PgFdwRelationInfo.
+ */
+ static List *
+ create_remote_agg_tlist(PlannerInfo *root, RelOptInfo *grouped_rel,
+ PathTarget *grouping_target)
+ {
+ List *tlist = NIL;
+ ListCell *lc;
+ int i;
+ Query *query = root->parse;
+ PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
+
+ grouping_target = copy_pathtarget(grouping_target);
+
i = 0;
foreach(lc, grouping_target->exprs)
{
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4647,4659 ****
* push down aggregation to the foreign server.
*/
if (!is_foreign_expr(root, grouped_rel, expr))
! return false;
/* Pushable, add to tlist */
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
else
{
/* Check entire expression whether it is pushable or not */
if (is_foreign_expr(root, grouped_rel, expr))
{
--- 5306,5339 ----
* push down aggregation to the foreign server.
*/
if (!is_foreign_expr(root, grouped_rel, expr))
! return NIL;
/* Pushable, add to tlist */
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
else
{
+ /*
+ * Aggregate can sometimes be wrapped by GroupedVar.
+ *
+ */
+ if ((IS_SIMPLE_REL(grouped_rel) || IS_JOIN_REL(grouped_rel)) &&
+ IsA(expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, expr);
+
+ /*
+ * GroupedVar is currently used only to handle Aggrefs.
+ *
+ * XXX These may include aggregates that appear in the HAVING
+ * clause. Although the HAVING clause is currently not sent to
+ * the remote server (see deparseSelectStmtForRel), we still
+ * need to send the aggregates the HAVING clause contains.
+ */
+ Assert(IsA(gvar->agg_partial, Aggref));
+ expr = (Expr *) gvar->agg_partial;
+ }
+
/* Check entire expression whether it is pushable or not */
if (is_foreign_expr(root, grouped_rel, expr))
{
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4662,4667 ****
--- 5342,5349 ----
}
else
{
+ List *aggvars;
+
/*
* If we have sortgroupref set, then it means that we have an
* ORDER BY entry pointing to this expression. Since we are
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4670,4681 ****
if (sgref)
grouping_target->sortgrouprefs[i] = 0;
! /* Not matched exactly, pull the var with aggregates then */
aggvars = pull_var_clause((Node *) expr,
PVC_INCLUDE_AGGREGATES);
if (!is_foreign_expr(root, grouped_rel, (Expr *) aggvars))
! return false;
/*
* Add aggregates, if any, into the targetlist. Plain var
--- 5352,5365 ----
if (sgref)
grouping_target->sortgrouprefs[i] = 0;
! /*
! * Not matched exactly, pull the var with aggregates then
! */
aggvars = pull_var_clause((Node *) expr,
PVC_INCLUDE_AGGREGATES);
if (!is_foreign_expr(root, grouped_rel, (Expr *) aggvars))
! return NIL;
/*
* Add aggregates, if any, into the targetlist. Plain var
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4703,4714 ****
/*
* Classify the pushable and non-pushable having clauses and save them in
! * remote_conds and local_conds of the grouped rel's fpinfo.
*/
if (root->hasHavingQual && query->havingQual)
{
- ListCell *lc;
-
foreach(lc, (List *) query->havingQual)
{
Expr *expr = (Expr *) lfirst(lc);
--- 5387,5403 ----
/*
* Classify the pushable and non-pushable having clauses and save them in
! * remote_conds_upper and local_conds_upper of the grouped rel's fpinfo.
! */
!
! /*
! * TODO For non-upper grouped_rel, only pick those expressions for which
! * grouped_rel provides all input vars. (And Assert() that grouped_rel
! * does not emit just some of them --- in such a case grouped_rel
! * shouldn't be subject to partial aggregation at all.)
*/
if (root->hasHavingQual && query->havingQual)
{
foreach(lc, (List *) query->havingQual)
{
Expr *expr = (Expr *) lfirst(lc);
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4728,4736 ****
NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
! fpinfo->remote_conds = lappend(fpinfo->remote_conds, rinfo);
else
! fpinfo->local_conds = lappend(fpinfo->local_conds, rinfo);
}
}
--- 5417,5427 ----
NULL,
NULL);
if (is_foreign_expr(root, grouped_rel, expr))
! fpinfo->remote_conds_upper =
! lappend(fpinfo->remote_conds_upper, rinfo);
else
! fpinfo->local_conds_upper =
! lappend(fpinfo->local_conds_upper, rinfo);
}
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4738,4749 ****
* If there are any local conditions, pull Vars and aggregates from it and
* check whether they are safe to pushdown or not.
*/
! if (fpinfo->local_conds)
{
List *aggvars = NIL;
- ListCell *lc;
! foreach(lc, fpinfo->local_conds)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
--- 5429,5439 ----
* If there are any local conditions, pull Vars and aggregates from it and
* check whether they are safe to pushdown or not.
*/
! if (fpinfo->local_conds_upper)
{
List *aggvars = NIL;
! foreach(lc, fpinfo->local_conds_upper)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4765,4771 ****
if (IsA(expr, Aggref))
{
if (!is_foreign_expr(root, grouped_rel, expr))
! return false;
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
--- 5455,5461 ----
if (IsA(expr, Aggref))
{
if (!is_foreign_expr(root, grouped_rel, expr))
! return NIL;
tlist = add_to_flat_tlist(tlist, list_make1(expr));
}
*************** foreign_grouping_ok(PlannerInfo *root, R
*** 4775,4803 ****
/* Transfer any sortgroupref data to the replacement tlist */
apply_pathtarget_labeling_to_tlist(tlist, grouping_target);
! /* Store generated targetlist */
! fpinfo->grouped_tlist = tlist;
!
! /* Safe to pushdown */
! fpinfo->pushdown_safe = true;
!
! /*
! * Set cached relation costs to some negative value, so that we can detect
! * when they are set to some sensible costs, during one (usually the
! * first) of the calls to estimate_path_cost_size().
! */
! fpinfo->rel_startup_cost = -1;
! fpinfo->rel_total_cost = -1;
!
! /*
! * Set the string describing this grouped relation to be used in EXPLAIN
! * output of corresponding ForeignScan.
! */
! fpinfo->relation_name = makeStringInfo();
! appendStringInfo(fpinfo->relation_name, "Aggregate on (%s)",
! ofpinfo->relation_name->data);
!
! return true;
}
/*
--- 5465,5471 ----
/* Transfer any sortgroupref data to the replacement tlist */
apply_pathtarget_labeling_to_tlist(tlist, grouping_target);
! return tlist;
}
/*
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4846,4857 ****
Query *parse = root->parse;
PgFdwRelationInfo *ifpinfo = input_rel->fdw_private;
PgFdwRelationInfo *fpinfo = grouped_rel->fdw_private;
ForeignPath *grouppath;
PathTarget *grouping_target;
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
/* Nothing to be done, if there is no grouping or aggregation required. */
if (!parse->groupClause && !parse->groupingSets && !parse->hasAggs &&
--- 5514,5523 ----
Query *parse = root->parse;
PgFdwRelationInfo *ifpinfo = input_rel->fdw_private;
PgFdwRelationInfo *fpinfo = grouped_rel->fdw_private;
+ EstimateInfo *estimates;
ForeignPath *grouppath;
PathTarget *grouping_target;
! List *fdw_private;
/* Nothing to be done, if there is no grouping or aggregation required. */
if (!parse->groupClause && !parse->groupingSets && !parse->hasAggs &&
*************** add_foreign_grouping_paths(PlannerInfo *
*** 4877,4902 ****
return;
/* Estimate the cost of push down */
! estimate_path_cost_size(root, grouped_rel, NIL, NIL, &rows,
! &width, &startup_cost, &total_cost);
! /* Now update this information in the fpinfo */
! fpinfo->rows = rows;
! fpinfo->width = width;
! fpinfo->startup_cost = startup_cost;
! fpinfo->total_cost = total_cost;
/* Create and add foreign path to the grouping relation. */
grouppath = create_foreignscan_path(root,
grouped_rel,
grouping_target,
! rows,
! startup_cost,
! total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
NULL,
! NIL); /* no fdw_private */
/* Add generated path into grouped_rel by add_path(). */
add_path(grouped_rel, (Path *) grouppath, false);
--- 5543,5570 ----
return;
/* Estimate the cost of push down */
! estimates = &fpinfo->est_grouped;
! estimate_path_cost_size(root, grouped_rel, NIL, NIL, NIL, estimates,
! true);
! /*
! * The path is considered grouped when it comes to plan creation and
! * deparsing. The 2nd element (ORDER BY expressions) is NIL because
! * UPPERREL_GROUP_AGG does not care about ordering.
! */
! fdw_private = list_make2(makeInteger(1), NIL);
/* Create and add foreign path to the grouping relation. */
grouppath = create_foreignscan_path(root,
grouped_rel,
grouping_target,
! estimates->rows,
! estimates->startup_cost,
! estimates->total_cost,
NIL, /* no pathkeys */
NULL, /* no required_outer */
NULL,
! fdw_private);
/* Add generated path into grouped_rel by add_path(). */
add_path(grouped_rel, (Path *) grouppath, false);
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
new file mode 100644
index 788b003..e9a63a8
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 20,25 ****
--- 20,37 ----
#include "libpq-fe.h"
+ typedef struct EstimateInfo
+ {
+ /* Estimated size and cost for a scan or join. */
+ double rows;
+ int width;
+ Cost startup_cost;
+ Cost total_cost;
+ /* Costs excluding costs for transferring data from the foreign server */
+ Cost rel_startup_cost;
+ Cost rel_total_cost;
+ } EstimateInfo;
+
/*
* FDW-specific planner information kept in RelOptInfo.fdw_private for a
* postgres_fdw foreign table. For a baserel, this struct is created by
*************** typedef struct PgFdwRelationInfo
*** 43,48 ****
--- 55,74 ----
List *remote_conds;
List *local_conds;
+ /*
+ * HAVING clauses.
+ *
+ * As for postgresGetForeignUpperPaths(), remote_conds / local_conds can
+ * be used for the HAVING clause because the remote_conds / local_conds of
+ * the input relation can be used for the WHERE clause. However, separate
+ * fields are needed for postgresGetForeignJoinPaths() because remote join
+ * can evaluate both WHERE and HAVING clause if it's grouped, and it does
+ * not use the concept of upper path. To be consistent, we always store
+ * the HAVING expressions into the following fields.
+ */
+ List *remote_conds_upper;
+ List *local_conds_upper;
+
/* Actual remote restriction clauses for scan (sans RestrictInfos) */
List *final_remote_exprs;
*************** typedef struct PgFdwRelationInfo
*** 56,69 ****
/* Selectivity of join conditions */
Selectivity joinclause_sel;
! /* Estimated size and cost for a scan or join. */
! double rows;
! int width;
! Cost startup_cost;
! Cost total_cost;
! /* Costs excluding costs for transferring data from the foreign server */
! Cost rel_startup_cost;
! Cost rel_total_cost;
/* Options extracted from catalogs. */
bool use_remote_estimate;
--- 82,90 ----
/* Selectivity of join conditions */
Selectivity joinclause_sel;
! /* Estimates for both plain and grouped relation. */
! EstimateInfo est_plain;
! EstimateInfo est_grouped;
/* Options extracted from catalogs. */
bool use_remote_estimate;
*************** typedef struct PgFdwRelationInfo
*** 92,98 ****
/* joinclauses contains only JOIN/ON conditions for an outer join */
List *joinclauses; /* List of RestrictInfo */
! /* Grouping information */
List *grouped_tlist;
/* Subquery information */
--- 113,121 ----
/* joinclauses contains only JOIN/ON conditions for an outer join */
List *joinclauses; /* List of RestrictInfo */
! /*
! * The targetlist used to construct the remote query.
! */
List *grouped_tlist;
/* Subquery information */
*************** extern unsigned int GetCursorNumber(PGco
*** 121,126 ****
--- 144,150 ----
extern unsigned int GetPrepStmtNumber(PGconn *conn);
extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
+ extern void pgfdw_send_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
*************** extern Expr *find_em_expr_for_rel(Equiva
*** 174,181 ****
extern List *build_tlist_to_deparse(RelOptInfo *foreignrel);
extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
! List *remote_conds, List *pathkeys, bool is_subquery,
! List **retrieved_attrs, List **params_list);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
--- 198,207 ----
extern List *build_tlist_to_deparse(RelOptInfo *foreignrel);
extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
! List *remote_conds, List *order_by_exprs,
! List *pathkeys, bool is_subquery,
! List **retrieved_attrs, List **params_list,
! bool grouping);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
new file mode 100644
index 3c3c5c7..3fc974f
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
*************** SELECT t1.a,t1.b FROM fprt1 t1, LATERAL
*** 1835,1837 ****
--- 1835,1902 ----
SELECT t1.a,t1.b FROM fprt1 t1, LATERAL (SELECT t2.a, t2.b FROM fprt2 t2 WHERE t1.a = t2.b AND t1.b = t2.a) q WHERE t1.a%25 = 0 ORDER BY 1,2;
RESET enable_partition_wise_join;
+
+ -- ===================================================================
+ -- test aggregation of base relation or join
+ -- ===================================================================
+ CREATE TABLE customers(id int primary key, name varchar);
+ INSERT INTO customers VALUES (0, 'me'), (1, 'you'), (2, 'he'), (3, 'she');
+ CREATE TABLE products(id int, price float4) PARTITION BY RANGE (id);
+ CREATE TABLE products_0 (LIKE products);
+ INSERT INTO products_0(id, price) SELECT g.i, trunc(100 * random()) /
+ 10 + 1 FROM generate_series(1, 1000) AS g(i);
+ CREATE TABLE products_1 (LIKE products);
+ INSERT INTO products_1(id, price) SELECT g.i, trunc(100 * random()) /
+ 10 + 1 FROM generate_series(1001, 2000) AS g(i);
+ CREATE TABLE orders(customer_id int, product_id int) PARTITION BY RANGE
+ (product_id);
+ CREATE TABLE orders_0 (LIKE orders);
+ INSERT INTO orders_0(customer_id, product_id) SELECT trunc(2 * random()),
+ trunc(1000 * random()) FROM generate_series(1, 5000);
+ CREATE TABLE orders_1 (LIKE orders);
+ INSERT INTO orders_1(customer_id, product_id)
+ SELECT trunc(2 * random()) + 2, trunc(1000 * random()) + 1001 FROM
+ generate_series(1, 5000);
+ CREATE FOREIGN TABLE f_orders_0 PARTITION OF orders FOR VALUES FROM (1) TO
+ (1001) SERVER loopback OPTIONS (table_name 'orders_0');
+ CREATE FOREIGN TABLE f_orders_1 PARTITION OF orders FOR VALUES FROM (1001) TO
+ (2001) SERVER loopback OPTIONS (table_name 'orders_1');
+ CREATE FOREIGN TABLE f_products_0 PARTITION OF products FOR VALUES FROM (1) TO
+ (1001) SERVER loopback OPTIONS (table_name 'products_0');
+ CREATE FOREIGN TABLE f_products_1 PARTITION OF products FOR VALUES FROM (1001)
+ TO (2001) SERVER loopback OPTIONS (table_name 'products_1');
+ ALTER SERVER loopback OPTIONS (ADD use_remote_estimate 'true');
+ ALTER SERVER loopback2 OPTIONS (ADD use_remote_estimate 'true');
+ ANALYZE;
+
+ SET enable_agg_pushdown TO on;
+ SET enable_partition_wise_join TO on;
+
+ -- Perform the aggregation on partitions of "orders" , join the result to the
+ -- "customers" local table and finalize the aggregation.
+ EXPLAIN (VERBOSE on, COSTS off)
+ SELECT c.name, count(*)
+ FROM customers AS c, orders AS o
+ WHERE c.id = o.customer_id
+ GROUP BY c.id;
+
+ -- Perform partition-wise join of "orders" and "products", aggregate the
+ -- result at partition level, append the outputs of partitions and join it to
+ -- the "customers" local table.
+ EXPLAIN (VERBOSE on, COSTS off)
+ SELECT p.id, c.name, sum(p.price)
+ FROM customers AS c, orders AS o, products AS p
+ WHERE c.id = o.customer_id AND p.id = o.product_id
+ GROUP BY p.id, c.id
+ ORDER BY 3;
+
+ -- The final aggregation is not necessary if the aggregated output of
+ -- partitions contains distinct set of grouping keys. Moreover, skip sort of
+ -- the result if the output of per-partition queries has the desired ordering
+ -- and so the sets can be merge-appended.
+ EXPLAIN (VERBOSE on, COSTS off)
+ SELECT p.id, sum(p.price)
+ FROM orders AS o, products AS p
+ WHERE p.id = o.product_id
+ GROUP BY p.id
+ ORDER BY 2 DESC;
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index f09647e..734940a
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, ExprState *s
*** 809,815 ****
* there.)
*/
if (IsA(gvar->gvexpr, Aggref))
-
ExecInitExprRec((Expr *) gvar->agg_partial, state,
resv, resnull);
else
--- 809,814 ----
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index f95c913..3b709a3
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 986,996 ****
static void
set_foreign_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
/* Mark rel with estimated output rows, width, etc */
! set_foreign_size_estimates(root, rel);
! /* Let FDW adjust the size estimates, if it can */
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid);
/* ... but do not let it set the rows estimate to zero */
rel->rows = clamp_row_est(rel->rows);
--- 986,1014 ----
static void
set_foreign_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
+ Relids required_outer = rel->lateral_relids;
+
/* Mark rel with estimated output rows, width, etc */
! set_foreign_size_estimates(root, rel, false);
! /*
! * Let FDW adjust the size estimates, if it can.
! *
! * For the grouped relation it only makes sense if set_foreign_pathlist is
! * supposed to handle it, see below.
! */
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid, false);
! if (rel->gpi != NULL && rel->gpi->target != NULL &&
! required_outer == NULL)
! {
! rel->fdwroutine->GetForeignRelSize(root, rel, rte->relid, true);
!
! /*
! * XXX Should cost_qual_eval be called separate so we don't repeat it
! * here for essentially identical baserestrictinfo?
! */
! set_foreign_size_estimates(root, rel, true);
! }
/* ... but do not let it set the rows estimate to zero */
rel->rows = clamp_row_est(rel->rows);
*************** set_foreign_size(PlannerInfo *root, RelO
*** 1003,1010 ****
static void
set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
/* Call the FDW's GetForeignPaths function to generate path(s) */
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid);
}
/*
--- 1021,1038 ----
static void
set_foreign_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
+ Relids required_outer = rel->lateral_relids;
+
/* Call the FDW's GetForeignPaths function to generate path(s) */
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid, false);
!
! /*
! * Create grouped paths if grouped target exists. Do nothing if outer
! * relations are needed --- that could cause parameterized (and therefore
! * repeated) grouping.
! */
! if (rel->gpi != NULL && rel->gpi->target && required_outer == NULL)
! rel->fdwroutine->GetForeignPaths(root, rel, rte->relid, true);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1586,1591 ****
--- 1614,1620 ----
foreach(l, live_childrels)
{
RelOptInfo *childrel = lfirst(l);
+ List *childpaths;
ListCell *lcp;
Path *cheapest_partial_path = NULL;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1698,1704 ****
* heuristic to indicate which sort orderings and parameterizations we
* should build Append and MergeAppend paths for.
*/
! foreach(lcp, childrel->pathlist)
{
Path *childpath = (Path *) lfirst(lcp);
List *childkeys = childpath->pathkeys;
--- 1727,1743 ----
* heuristic to indicate which sort orderings and parameterizations we
* should build Append and MergeAppend paths for.
*/
! childpaths = childrel->pathlist;
!
! /*
! * Extra orderings may be available for grouped paths, i.e. ordered by
! * aggregate. (At least ForeignPath can generate these.)
! */
! if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
! childpaths = list_concat(list_copy(childpaths),
! childrel->gpi->pathlist);
!
! foreach(lcp, childpaths)
{
Path *childpath = (Path *) lfirst(lcp);
List *childkeys = childpath->pathkeys;
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index 9f489c3..f28f5fc
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
*************** set_namedtuplestore_size_estimates(Plann
*** 5037,5043 ****
* already.
*/
void
! set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel)
{
/* Should only be applied to base relations */
Assert(rel->relid > 0);
--- 5037,5043 ----
* already.
*/
void
! set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
/* Should only be applied to base relations */
Assert(rel->relid > 0);
*************** set_foreign_size_estimates(PlannerInfo *
*** 5046,5052 ****
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel, rel->reltarget, false);
}
--- 5046,5064 ----
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! if (!grouped)
! set_rel_width(root, rel, rel->reltarget, false);
! else
! {
! Assert(rel->gpi != NULL);
!
! /*
! * If the relation can produce grouped path, estimate width and costs
! * of the corresponding target.
! */
! if (rel->gpi->target)
! set_rel_width(root, rel, rel->gpi->target, true);
! }
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 883d8a1..a678bc2
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 343,351 ****
*/
if (joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
! jointype, &extra);
/*
* 6. Finally, give extensions a chance to manipulate the path list.
--- 343,357 ----
*/
if (joinrel->fdwroutine &&
joinrel->fdwroutine->GetForeignJoinPaths)
+ {
joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
outerrel, innerrel,
! jointype, &extra, false);
! if (joinrel->gpi != NULL && joinrel->gpi->target != NULL)
! joinrel->fdwroutine->GetForeignJoinPaths(root, joinrel,
! outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 6. Finally, give extensions a chance to manipulate the path list.
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index b11afd9..ba63830
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** static Plan *prepare_sort_from_pathkeys(
*** 251,256 ****
--- 251,257 ----
static EquivalenceMember *find_ec_member_for_tle(EquivalenceClass *ec,
TargetEntry *tle,
Relids relids);
+ static TargetEntry *get_aggref_from_tle(TargetEntry *tle);
static Sort *make_sort_from_pathkeys(Plan *lefttree, List *pathkeys,
Relids relids);
static Sort *make_sort_from_groupcols(List *groupcls,
*************** prepare_sort_from_pathkeys(Plan *lefttre
*** 5655,5660 ****
--- 5656,5664 ----
tle = get_tle_by_resno(tlist, reqColIdx[numsortkeys]);
if (tle)
{
+ /* Handle special cases of sorting by aggregate. */
+ tle = get_aggref_from_tle(tle);
+
em = find_ec_member_for_tle(ec, tle, relids);
if (em)
{
*************** prepare_sort_from_pathkeys(Plan *lefttre
*** 5686,5691 ****
--- 5690,5698 ----
foreach(j, tlist)
{
tle = (TargetEntry *) lfirst(j);
+
+ /* Handle special cases of sorting by aggregate. */
+ tle = get_aggref_from_tle(tle);
em = find_ec_member_for_tle(ec, tle, relids);
if (em)
{
*************** find_ec_member_for_tle(EquivalenceClass
*** 5859,5864 ****
--- 5866,5893 ----
}
/*
+ * Get Aggref from target entry if it's wrapped in GroupedVar.
+ */
+ static TargetEntry *
+ get_aggref_from_tle(TargetEntry *tle)
+ {
+ if (IsA(tle->expr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, tle->expr);
+
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, gvar->gvexpr);
+
+ tle = flatCopyTargetEntry(tle);
+ tle->expr = (Expr *) aggref;
+ }
+ }
+
+ return tle;
+ }
+
+ /*
* make_sort_from_pathkeys
* Create sort plan to sort according to given pathkeys
*
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index 502c725..02c6c61
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** set_foreignscan_references(PlannerInfo *
*** 1199,1228 ****
if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0)
{
/*
* Adjust tlist, qual, fdw_exprs, fdw_recheck_quals to reference
! * foreign scan tuple
*/
! indexed_tlist *itlist = build_tlist_index(fscan->fdw_scan_tlist);
!
fscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
! (Node *) fscan->scan.plan.targetlist,
itlist,
INDEX_VAR,
rtoffset);
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
(Node *) fscan->scan.plan.qual,
itlist,
INDEX_VAR,
rtoffset);
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
(Node *) fscan->fdw_exprs,
itlist,
INDEX_VAR,
rtoffset);
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
(Node *) fscan->fdw_recheck_quals,
--- 1199,1242 ----
if (fscan->fdw_scan_tlist != NIL || fscan->scan.scanrelid == 0)
{
+ List *tlist_tmp;
+ indexed_tlist *itlist = build_tlist_index(fscan->fdw_scan_tlist);
+
/*
* Adjust tlist, qual, fdw_exprs, fdw_recheck_quals to reference
! * foreign scan tuple.
! *
! * Since fdw_scan_tlist contains Aggrefs instead of GroupVars (this is
! * convenient for deparsing), the Aggrefs also need to be restored in
! * the referencing lists.
! *
! * XXX Consider thoroughly where tlist_tmp really needs to be used,
! * i.e. which of the lists can / cannot contain aggregates.
*/
! tlist_tmp =
! replace_grouped_vars_with_aggrefs(root,
! fscan->scan.plan.targetlist);
fscan->scan.plan.targetlist = (List *)
fix_upper_expr(root,
! (Node *) tlist_tmp,
itlist,
INDEX_VAR,
rtoffset);
+
fscan->scan.plan.qual = (List *)
fix_upper_expr(root,
(Node *) fscan->scan.plan.qual,
itlist,
INDEX_VAR,
rtoffset);
+
fscan->fdw_exprs = (List *)
fix_upper_expr(root,
(Node *) fscan->fdw_exprs,
itlist,
INDEX_VAR,
rtoffset);
+
fscan->fdw_recheck_quals = (List *)
fix_upper_expr(root,
(Node *) fscan->fdw_recheck_quals,
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
new file mode 100644
index 04e43cc..16d6082
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
*************** struct ExplainState;
*** 26,36 ****
typedef void (*GetForeignRelSize_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
typedef void (*GetForeignPaths_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid);
typedef ForeignScan *(*GetForeignPlan_function) (PlannerInfo *root,
RelOptInfo *baserel,
--- 26,38 ----
typedef void (*GetForeignRelSize_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
typedef void (*GetForeignPaths_function) (PlannerInfo *root,
RelOptInfo *baserel,
! Oid foreigntableid,
! bool grouped);
typedef ForeignScan *(*GetForeignPlan_function) (PlannerInfo *root,
RelOptInfo *baserel,
*************** typedef void (*GetForeignJoinPaths_funct
*** 57,63 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
UpperRelationKind stage,
--- 59,66 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped);
typedef void (*GetForeignUpperPaths_function) (PlannerInfo *root,
UpperRelationKind stage,
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
new file mode 100644
index 12d8134..9956e0f
*** a/src/include/optimizer/cost.h
--- b/src/include/optimizer/cost.h
*************** extern void set_cte_size_estimates(Plann
*** 194,200 ****
double cte_rows);
extern void set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern void set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel);
! extern void set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern PathTarget *set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target);
extern double compute_bitmap_pages(PlannerInfo *root, RelOptInfo *baserel,
Path *bitmapqual, int loop_count, Cost *cost, double *tuple);
--- 194,201 ----
double cte_rows);
extern void set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern void set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel);
! extern void set_foreign_size_estimates(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
extern PathTarget *set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target);
extern double compute_bitmap_pages(PlannerInfo *root, RelOptInfo *baserel,
Path *bitmapqual, int loop_count, Cost *cost, double *tuple);
agg_pushdown_v5/11_readme.diff
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
new file mode 100644
index 1e4084d..3994394
*** a/src/backend/optimizer/README
--- b/src/backend/optimizer/README
*************** plan as possible. Expanding the range o
*** 1076,1081 ****
--- 1076,1116 ----
pushed below the Gather (and costing them accurately) is likely to keep us
busy for a long time to come.
+ Grouped Paths
+ -------------
+
+ If both query semantics and path type allow, an existing path can be subject
+ to partial aggregation (ie aggregation without calling the aggfinalfn
+ function) and stored as a "grouped path" in a separate pathlist. Both base
+ relation and join paths can be turned into a grouped path this way. If the
+ source path was partial (in terms of parallel processing, see above), then the
+ grouped path is called "partial grouped path".
+
+ If a grouped path appears at the top of the join tree, the transient state
+ values are aggregated and the aggfinalfn function is called for each
+ aggregate. Likewise, if the top-level join produces a partial grouped path, we
+ first apply Gather plan on it and then we finalize the aggregation in the same
+ way.
+
+ Besides being a result of aggregation, (partial) grouped path can be created
+ by joining an existing grouped path to an ordinary (non-grouped) path. This
+ technique can result in duplication of grouping keys, which are essentially
+ unique in the output of aggregation. The final aggregation takes these
+ duplicities into account.
+
+ In contrast, join of two grouped paths is not supported as there doesn't seem
+ to be an interesting use case for it. If we want to support this kind of join
+ in the future, additional attention needs to be paid to the duplication of
+ grouping keys mentioned above. For example, if scan of table A is joined to a
+ grouped scan of table B, then the cardinality of grouping keys of table B
+ present in (non-grouped) table A determines how many times does each
+ aggregation transient state value of B appear on the input of the final
+ aggregation path. However if A is grouped too, the number of those key values
+ on the A side can get lower and thus affect the input of the final aggregation
+ path. When compensating for this effect, we'd have to count input values of
+ each group during the aggregation of A and "multiply" the corresponding
+ transient state values of B accordingly.
+
Partition-wise joins
--------------------
A join between two similarly partitioned tables can be broken down into joins
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-01-15 21:15 Robert Haas <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 2 replies; 59+ messages in thread
From: Robert Haas @ 2018-01-15 21:15 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Fri, Dec 22, 2017 at 10:43 AM, Antonin Houska <[email protected]> wrote:
> Michael Paquier <[email protected]> wrote:
>> On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <[email protected]> wrote:
>> > I'm not about to add any other features now. Implementation of the missing
>> > parts (see the TODO comments in the code) is the next step. But what I'd
>> > appreciate most is a feedback on the design. Thanks.
>>
>> I am getting a conflict after applying patch 5 but this did not get
>> any reviews so moved to next CF with waiting on author as status.
>
> Attached is the next version.
I've been a bit confused for a while about what this patch is trying
to do, so I spent some time today looking at it to try to figure it
out. There's a lot I don't understand yet, but it seems like the
general idea is to build, potentially for each relation in the join
tree, not only the regular list of paths but also a list of "grouped"
paths. If the pre-grouped path wins, then we can get a final path
that looks like Finalize Aggregate -> Some Joins -> Partial Aggregate
-> Maybe Some More Joins -> Base Table Scan. In some cases the patch
seems to replace that uppermost Finalize Aggregate with a Result node.
translate_expression_to_rels() looks unsafe. Equivalence members are
known to be equal under the semantics of the relevant operator class,
but that doesn't mean that one can be freely substituted for another.
For example:
rhaas=# create table one (a numeric);
CREATE TABLE
rhaas=# create table two (a numeric);
CREATE TABLE
rhaas=# insert into one values ('0');
INSERT 0 1
rhaas=# insert into two values ('0.00');
INSERT 0 1
rhaas=# select one.a, count(*) from one, two where one.a = two.a group by 1;
a | count
---+-------
0 | 1
(1 row)
rhaas=# select two.a, count(*) from one, two where one.a = two.a group by 1;
a | count
------+-------
0.00 | 1
(1 row)
There are, admittedly, a large number of data types for which such a
substitution would work just fine. numeric will not, citext will not,
but many others are fine. Regrettably, we have no framework in the
system for identifying equality operators which actually test identity
versus some looser notion of equality. Cross-type operators are a
problem, too; if we have foo.x = bar.y in the query, one might be int4
and the other int8, for example, but they can still belong to the same
equivalence class.
Concretely, in your test query "SELECT p.i, avg(c1.v) FROM
agg_pushdown_parent AS p JOIN agg_pushdown_child1 AS c1 ON c1.parent =
p.i GROUP BY p.i" you assume that it's OK to do a Partial
HashAggregate over c1.parent rather than p.i. This will be false if,
say, c1.parent is of type citext and p.i is of type text; this will
get grouped together that shouldn't. It will also be false if the
grouping expression is something like GROUP BY length(p.i::text),
because one value could be '0'::numeric and the other '0.00'::numeric.
I can't think of a reason why it would be false if the grouping
expressions are both simple Vars of the same underlying data type, but
I'm a little nervous that I might be wrong even about that case.
Maybe you've handled all of this somehow, but it's not obvious to me
that it has been considered.
Another thing I noticed is that the GroupedPathInfo looks a bit like a
stripped-down RelOptInfo, in that for example it has both a pathlist
and a partial_pathlist. I'm inclined to think that we should build new
RelOptInfos instead. As you have it, there are an awful lot of things
that seem to know about the grouped/ungrouped distinction, many of
which are quite low-level functions like add_path(),
add_paths_to_append_rel(), try_nestloop_path(), etc. I think that
some of this would go away if you had separate RelOptInfos instead of
GroupedPathInfo.
Some compiler noise:
allpaths.c:2794:11: error: variable 'partial_target' is used
uninitialized whenever 'if' condition is false
[-Werror,-Wsometimes-uninitialized]
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
allpaths.c:2798:56: note: uninitialized use occurs here
create_gather_path(root, rel, cheapest_partial_path,
partial_target,
^~~~~~~~~~~~~~
allpaths.c:2794:7: note: remove the 'if' if its condition is always true
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
allpaths.c:2794:11: error: variable 'partial_target' is used
uninitialized whenever '&&' condition is false
[-Werror,-Wsometimes-uninitialized]
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~
allpaths.c:2798:56: note: uninitialized use occurs here
create_gather_path(root, rel, cheapest_partial_path,
partial_target,
^~~~~~~~~~~~~~
allpaths.c:2794:11: note: remove the '&&' if its condition is always true
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~~~~
allpaths.c:2773:28: note: initialize the variable 'partial_target' to
silence this warning
PathTarget *partial_target;
^
= NULL
Core dump running the regression tests:
* frame #0: 0x00007fffe633ad42 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fffe6428457 libsystem_pthread.dylib`pthread_kill + 90
frame #2: 0x00007fffe62a0420 libsystem_c.dylib`abort + 129
frame #3: 0x000000010e3ae90e
postgres`ExceptionalCondition(conditionName=<unavailable>,
errorType=<unavailable>, fileName=<unavailable>,
lineNumber=<unavailable>) at assert.c:54 [opt]
frame #4: 0x000000010e17c23b
postgres`check_list_invariants(list=<unavailable>) at list.c:39 [opt]
frame #5: 0x000000010e17cdff postgres`list_free [inlined]
list_free_private(list=0x00007fe6a483be40, deep='\0') at list.c:1107
[opt]
frame #6: 0x000000010e17cdfa
postgres`list_free(list=0x00007fe6a483be40) at list.c:1135 [opt]
frame #7: 0x000000010e1b365f
postgres`generate_bitmap_or_paths(root=0x00007fe6a68131c8,
rel=0x00007fe6a6815310, clauses=<unavailable>,
other_clauses=<unavailable>, agg_info=0x0000000000000000) at
indxpath.c:1580 [opt]
frame #8: 0x000000010e1b305c
postgres`create_index_paths(root=0x00007fe6a68131c8,
rel=<unavailable>, agg_info=0x0000000000000000) at indxpath.c:334
[opt]
frame #9: 0x000000010e1a7cfa postgres`set_rel_pathlist [inlined]
set_plain_rel_pathlist at allpaths.c:745 [opt]
frame #10: 0x000000010e1a7bc1
postgres`set_rel_pathlist(root=0x00007fe6a68131c8,
rel=0x00007fe6a6815310, rti=1, rte=0x00007fe6a6803270) at
allpaths.c:454 [opt]
frame #11: 0x000000010e1a4910 postgres`make_one_rel at allpaths.c:312 [opt]
frame #12: 0x000000010e1a48cc
postgres`make_one_rel(root=0x00007fe6a68131c8,
joinlist=0x00007fe6a6815810) at allpaths.c:182 [opt]
frame #13: 0x000000010e1cb806
postgres`query_planner(root=0x00007fe6a68131c8, tlist=<unavailable>,
qp_callback=<unavailable>, qp_extra=0x00007fff51ca24d8) at
planmain.c:268 [opt]
frame #14: 0x000000010e1ce7b4
postgres`grouping_planner(root=<unavailable>, inheritance_update='\0',
tuple_fraction=<unavailable>) at planner.c:1789 [opt]
frame #15: 0x000000010e1ccb12
postgres`subquery_planner(glob=<unavailable>,
parse=0x00007fe6a6803158, parent_root=<unavailable>,
hasRecursion=<unavailable>, tuple_fraction=0) at planner.c:901 [opt]
frame #16: 0x000000010e1cba3e
postgres`standard_planner(parse=0x00007fe6a6803158, cursorOptions=256,
boundParams=0x0000000000000000) at planner.c:364 [opt]
frame #17: 0x000000010e291b3b
postgres`pg_plan_query(querytree=0x00007fe6a6803158,
cursorOptions=256, boundParams=0x0000000000000000) at postgres.c:807
[opt]
frame #18: 0x000000010e291c6e
postgres`pg_plan_queries(querytrees=<unavailable>, cursorOptions=256,
boundParams=0x0000000000000000) at postgres.c:873 [opt]
frame #19: 0x000000010e296335
postgres`exec_simple_query(query_string="SELECT p1.oid,
p1.typname\nFROM pg_type as p1\nWHERE p1.typnamespace = 0 OR\n
(p1.typlen <= 0 AND p1.typlen != -1 AND p1.typlen != -2) OR\n
(p1.typtype not in ('b', 'c', 'd', 'e', 'p', 'r')) OR\n NOT
p1.typisdefined OR\n (p1.typalign not in ('c', 's', 'i', 'd')) OR\n
(p1.typstorage not in ('p', 'x', 'e', 'm'));") at postgres.c:1048
[opt]
frame #20: 0x000000010e295099
postgres`PostgresMain(argc=<unavailable>, argv=<unavailable>,
dbname="regression", username=<unavailable>) at postgres.c:0 [opt]
frame #21: 0x000000010e20a910 postgres`PostmasterMain [inlined]
BackendRun at postmaster.c:4412 [opt]
frame #22: 0x000000010e20a741 postgres`PostmasterMain [inlined]
BackendStartup at postmaster.c:4084 [opt]
frame #23: 0x000000010e20a3bd postgres`PostmasterMain at
postmaster.c:1757 [opt]
frame #24: 0x000000010e2098ff
postgres`PostmasterMain(argc=1516050552, argv=<unavailable>) at
postmaster.c:1365 [opt]
frame #25: 0x000000010e177537 postgres`main(argc=<unavailable>,
argv=<unavailable>) at main.c:228 [opt]
frame #26: 0x00007fffe620c235 libdyld.dylib`start + 1
(lldb) p debug_query_string
(const char *) $0 = 0x00007fe6a401e518 "SELECT p1.oid,
p1.typname\nFROM pg_type as p1\nWHERE p1.typnamespace = 0 OR\n
(p1.typlen <= 0 AND p1.typlen != -1 AND p1.typlen != -2) OR\n
(p1.typtype not in ('b', 'c', 'd', 'e', 'p', 'r')) OR\n NOT
p1.typisdefined OR\n (p1.typalign not in ('c', 's', 'i', 'd')) OR\n
(p1.typstorage not in ('p', 'x', 'e', 'm'));"
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-01-26 13:04 Antonin Houska <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2018-01-26 13:04 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
Robert Haas <[email protected]> wrote:
> On Fri, Dec 22, 2017 at 10:43 AM, Antonin Houska <[email protected]> wrote:
> > Michael Paquier <[email protected]> wrote:
> >> On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <[email protected]> wrote:
> >> > I'm not about to add any other features now. Implementation of the missing
> >> > parts (see the TODO comments in the code) is the next step. But what I'd
> >> > appreciate most is a feedback on the design. Thanks.
> >>
> >> I am getting a conflict after applying patch 5 but this did not get
> >> any reviews so moved to next CF with waiting on author as status.
> >
> > Attached is the next version.
>
> I've been a bit confused for a while about what this patch is trying
> to do, so I spent some time today looking at it to try to figure it
> out.
Thanks.
> There's a lot I don't understand yet, but it seems like the general idea is
> to build, potentially for each relation in the join tree, not only the
> regular list of paths but also a list of "grouped" paths. If the
> pre-grouped path wins, then we can get a final path that looks like Finalize
> Aggregate -> Some Joins -> Partial Aggregate -> Maybe Some More Joins ->
> Base Table Scan.
Yes. The regression test output shows some more plans.
> In some cases the patch seems to replace that uppermost Finalize Aggregate
> with a Result node.
This is a feature implemented in 09_avoid_agg_finalization.diff. You can
ignore it for now, it's of lower priority than the preceding parts and needs
more work to be complete.
> translate_expression_to_rels() looks unsafe. Equivalence members are
> known to be equal under the semantics of the relevant operator class,
> but that doesn't mean that one can be freely substituted for another.
> For example:
>
> rhaas=# create table one (a numeric);
> CREATE TABLE
> rhaas=# create table two (a numeric);
> CREATE TABLE
> rhaas=# insert into one values ('0');
> INSERT 0 1
> rhaas=# insert into two values ('0.00');
> INSERT 0 1
> rhaas=# select one.a, count(*) from one, two where one.a = two.a group by 1;
> a | count
> ---+-------
> 0 | 1
> (1 row)
>
> rhaas=# select two.a, count(*) from one, two where one.a = two.a group by 1;
> a | count
> ------+-------
> 0.00 | 1
> (1 row)
>
> There are, admittedly, a large number of data types for which such a
> substitution would work just fine. numeric will not, citext will not,
> but many others are fine. Regrettably, we have no framework in the
> system for identifying equality operators which actually test identity
> versus some looser notion of equality. Cross-type operators are a
> problem, too; if we have foo.x = bar.y in the query, one might be int4
> and the other int8, for example, but they can still belong to the same
> equivalence class.
>
> Concretely, in your test query "SELECT p.i, avg(c1.v) FROM
> agg_pushdown_parent AS p JOIN agg_pushdown_child1 AS c1 ON c1.parent =
> p.i GROUP BY p.i" you assume that it's OK to do a Partial
> HashAggregate over c1.parent rather than p.i. This will be false if,
> say, c1.parent is of type citext and p.i is of type text; this will
> get grouped together that shouldn't.
I've really missed this problem, thanks for bringing it up! Your
considerations made me think of the specifics of the types like numeric or
citext. Although your example does not demonstrate what I consider the core
issue, I believe we eventually think of the same.
Consider this example query (using the tables you defined upthread):
SELECT one.a
FROM one, two
WHERE one.a = two.a AND scale(two.a) > 1
GROUP BY 1
I we first try to aggregate "two", then evaluate WHERE clause and then
finalize the aggregation, the partial aggregation of "two" can put various
binary values of the "a" column (0, 0.0, etc.) into the same group so the
scale of the output (of the partial agregation) will be undefined. Thus the
aggregation push-down will change the input of the WHERE clause.
So one problem is that the grouping expression can be inappropriate for
partial aggregation even if there's no type change during the
translation. What I consider typical for this case is that the equality
operator used to identify groups (SortGroupClause.eqop) can return true even
if binary (stored) values of the inputs differ. The partial aggregation could
only take place if we had a special equality operator which distinguishes the
*binary* values (I don't know yet how to store this operator the catalog ---
in pg_opclass recors for the hash opclasses?)..
On the other hand, if the grouping expression is not a plain Var and there's
no type change during the translation and the expression output type is not of
the "identity-unsafe type" (numeric, citext, etc.), input vars of the
"identity-unsafe type"should not prevent us from using the expression for
partial aggregation: in such a case the grouping keys are computed in the same
way they would be w/o the partial aggregation.
The other problem is which type changes are legal. We should not allow any
type-specific information (such as scale or precision of the numeric type) to
bet lost or ambiguous. In this example
> It will also be false if the grouping expression is something like GROUP BY
> length(p.i::text), because one value could be '0'::numeric and the other
> '0.00'::numeric.
you show that not only numeric -> int is a problem but also int ->
numeric. This is the tricky part.
One of my ideas is to check whether the source and destination types are
binary coercible (i.e. pg_cast(castfunc) = 0) but this might be a misuse of
the binary coercibility. Another idea is to allow only such changes that the
destination type is in the same operator class as the source, and explicitly
enumerate the "safe opclasses". But that would mean that user cannot define
new opclasses within which the translation is possible --- not sure this is a
serious issue.
Perhaps the most consistent solution is that the translation is not performed
if any input variable of the grouping expression should change its data type.
> Another thing I noticed is that the GroupedPathInfo looks a bit like a
> stripped-down RelOptInfo, in that for example it has both a pathlist
> and a partial_pathlist. I'm inclined to think that we should build new
> RelOptInfos instead. As you have it, there are an awful lot of things
> that seem to know about the grouped/ungrouped distinction, many of
> which are quite low-level functions like add_path(),
> add_paths_to_append_rel(), try_nestloop_path(), etc. I think that
> some of this would go away if you had separate RelOptInfos instead of
> GroupedPathInfo.
This was proposed by Ashutosh Bapat upthread. I actually did this in the early
(not published) prototype of the patch and I considered it too invasive for
standard_join_search() and subroutines. IIRC an array like simple_rel_array
had to be introduced for the grouped relation in which the meaning of NULL was
twofold: either the relation is not a base relation or it is base relation but
no grouped relation exists for it. Also I thought there would be too much
duplicate data if the grouped relation had a separate RelOptInfo. Now that I'm
done with one approach I can consider the original approach aggain.
> Some compiler noise:
> Core dump running the regression tests:
I'll fix these, thanks.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-01-26 13:58 Robert Haas <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Robert Haas @ 2018-01-26 13:58 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Fri, Jan 26, 2018 at 8:04 AM, Antonin Houska <[email protected]> wrote:
> So one problem is that the grouping expression can be inappropriate for
> partial aggregation even if there's no type change during the
> translation. What I consider typical for this case is that the equality
> operator used to identify groups (SortGroupClause.eqop) can return true even
> if binary (stored) values of the inputs differ. The partial aggregation could
> only take place if we had a special equality operator which distinguishes the
> *binary* values (I don't know yet how to store this operator the catalog ---
> in pg_opclass recors for the hash opclasses?)..
We don't have an operator that tests for binary equality, but it's
certainly testable from C; see datumIsEqual. I'm not sure how much
this really helps, though. I think it would be safe to build an
initial set of groups based on datumIsEqual comparisons and then
arrange to later merge some of the groups. But that's not something
we ever do in the executor today, so it might require quite a lot of
hacking. Also, it seems like it might really suck in some cases. For
instance, consider something like SELECT scale(one.a), sum(two.a) FROM
one, two WHERE one.a = two.a GROUP BY 1. Doing a Partial Aggregate on
two.a using datumIsEqual semantics, joining to a, and then doing a
Finalize Aggregate looks legal, but the Partial Aggregate may produce
a tremendous number of groups compared to the Finalize Aggregate. In
other words, this technique wouldn't merge any groups that shouldn't
be merged, but it might fail to merge groups that really need to be
merged to get good performance.
> One of my ideas is to check whether the source and destination types are
> binary coercible (i.e. pg_cast(castfunc) = 0) but this might be a misuse of
> the binary coercibility.
Yeah, binary coercibility has nothing to do with this; that tells you
whether the two types are the same on disk, not whether they have the
same equality semantics. For instance, I think text and citext are
binary coercible, but their equality semantics are not the same.
> Another idea is to allow only such changes that the
> destination type is in the same operator class as the source, and explicitly
> enumerate the "safe opclasses". But that would mean that user cannot define
> new opclasses within which the translation is possible --- not sure this is a
> serious issue.
Enumerating specific opclasses in the source code is a non-starter --
Tom Lane would roll over in his grave if he weren't still alive. What
we could perhaps consider doing is adding some mechanism for an
opclass or opfamily to say whether its equality semantics happen to be
exactly datumIsEqual() semantics. That's a little grotty because it
leaves data types for which that isn't true out in the cold, but on
the other hand it seems like it would be useful as a way of optimizing
a bunch of things other than this. Maybe it could also include a way
to specify that the comparator happens to have the semantics as C's
built-in < and > operators, which seems like a case that might also
lend itself to some optimizations.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-01-29 08:32 Antonin Houska <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 2 replies; 59+ messages in thread
From: Antonin Houska @ 2018-01-29 08:32 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
Robert Haas <[email protected]> wrote:
> On Fri, Jan 26, 2018 at 8:04 AM, Antonin Houska <[email protected]> wrote:
> > So one problem is that the grouping expression can be inappropriate for
> > partial aggregation even if there's no type change during the
> > translation. What I consider typical for this case is that the equality
> > operator used to identify groups (SortGroupClause.eqop) can return true even
> > if binary (stored) values of the inputs differ. The partial aggregation could
> > only take place if we had a special equality operator which distinguishes the
> > *binary* values (I don't know yet how to store this operator the catalog ---
> > in pg_opclass recors for the hash opclasses?)..
>
> We don't have an operator that tests for binary equality, but it's
> certainly testable from C; see datumIsEqual. I'm not sure how much
> this really helps, though. I think it would be safe to build an
> initial set of groups based on datumIsEqual comparisons and then
> arrange to later merge some of the groups. But that's not something
> we ever do in the executor today, so it might require quite a lot of
> hacking. Also, it seems like it might really suck in some cases. For
> instance, consider something like SELECT scale(one.a), sum(two.a) FROM
> one, two WHERE one.a = two.a GROUP BY 1. Doing a Partial Aggregate on
> two.a using datumIsEqual semantics, joining to a, and then doing a
> Finalize Aggregate looks legal, but the Partial Aggregate may produce
> a tremendous number of groups compared to the Finalize Aggregate. In
> other words, this technique wouldn't merge any groups that shouldn't
> be merged, but it might fail to merge groups that really need to be
> merged to get good performance.
I don't insist on doing Partial Aggregate in any case. If we wanted to group
by the binary value, we'd probably have to enhance statistics accordingly. The
important thing is to recognize the special case like this. Rejection of the
Partial Aggregate would be the default response.
> > Another idea is to allow only such changes that the
> > destination type is in the same operator class as the source, and explicitly
> > enumerate the "safe opclasses". But that would mean that user cannot define
> > new opclasses within which the translation is possible --- not sure this is a
> > serious issue.
>
> Enumerating specific opclasses in the source code is a non-starter --
> Tom Lane would roll over in his grave if he weren't still alive. What
> we could perhaps consider doing is adding some mechanism for an
> opclass or opfamily to say whether its equality semantics happen to be
> exactly datumIsEqual() semantics. That's a little grotty because it
> leaves data types for which that isn't true out in the cold, but on
> the other hand it seems like it would be useful as a way of optimizing
> a bunch of things other than this. Maybe it could also include a way
> to specify that the comparator happens to have the semantics as C's
> built-in < and > operators, which seems like a case that might also
> lend itself to some optimizations.
I think of a variant of this: implement an universal function that tests the
binary values for equality (besides the actual arguments, caller would have to
pass info like typlen, typalign, typbyval for each argument, and cache these
for repeated calls) and set pg_proc(oprcode) to 0 wherever this function is
sufficient. Thus the problematic cases like numeric, citext, etc. would be
detected by their non-zero oprcode.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-01-29 14:01 Chapman Flack <[email protected]>
parent: Antonin Houska <[email protected]>
1 sibling, 0 replies; 59+ messages in thread
From: Chapman Flack @ 2018-01-29 14:01 UTC (permalink / raw)
To: [email protected]
On 01/29/18 03:32, Antonin Houska wrote:
> Robert Haas <[email protected]> wrote:
>>> only take place if we had a special equality operator which distinguishes the
>>> *binary* values (I don't know yet how to store this operator the catalog ---
...
>> We don't have an operator that tests for binary equality, but it's
>> certainly testable from C; see datumIsEqual.
Disclaimer: I haven't been following the whole thread closely. But could
there be some way to exploit the composite-type *= operator?
https://www.postgresql.org/docs/current/static/functions-comparisons.html#COMPOSITE-TYPE-COMPARISON
-Chap
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-01-29 14:43 Robert Haas <[email protected]>
parent: Antonin Houska <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: Robert Haas @ 2018-01-29 14:43 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Mon, Jan 29, 2018 at 3:32 AM, Antonin Houska <[email protected]> wrote:
> I think of a variant of this: implement an universal function that tests the
> binary values for equality (besides the actual arguments, caller would have to
> pass info like typlen, typalign, typbyval for each argument, and cache these
> for repeated calls) and set pg_proc(oprcode) to 0 wherever this function is
> sufficient. Thus the problematic cases like numeric, citext, etc. would be
> detected by their non-zero oprcode.
I don't think that's a good option, because we don't want int4eq to
have to go through a code path that has branches to support varlenas
and cstrings. Andres is busy trying to speed up expression evaluation
by removing unnecessary code branches; adding new ones would be
undoing that valuable work.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-02-23 16:08 Antonin Houska <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2018-02-23 16:08 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
Robert Haas <[email protected]> wrote:
> On Mon, Jan 29, 2018 at 3:32 AM, Antonin Houska <[email protected]> wrote:
> > I think of a variant of this: implement an universal function that tests the
> > binary values for equality (besides the actual arguments, caller would have to
> > pass info like typlen, typalign, typbyval for each argument, and cache these
> > for repeated calls) and set pg_proc(oprcode) to 0 wherever this function is
> > sufficient. Thus the problematic cases like numeric, citext, etc. would be
> > detected by their non-zero oprcode.
>
> I don't think that's a good option, because we don't want int4eq to
> have to go through a code path that has branches to support varlenas
> and cstrings. Andres is busy trying to speed up expression evaluation
> by removing unnecessary code branches; adding new ones would be
> undoing that valuable work.
I spent some more time thinking about this. What about adding a new strategy
number for hash index operator classes, e.g. HTBinaryEqualStrategyNumber? For
most types both HTEqualStrategyNumber and HTBinaryEqualStrategyNumber strategy
would point to the same operator, but types like numeric would naturally have
them different.
Thus the pushed-down partial aggregation can only use the
HTBinaryEqualStrategyNumber's operator to compare grouping expressions. In the
initial version (until we have useful statistics for the binary values) we can
avoid the aggregation push-down if the grouping expression output type has the
two strategies implemented using different functions because, as you noted
upthread, grouping based on binary equality can result in excessive number of
groups.
One open question is whether the binary equality operator needs a separate
operator class or not. If an opclass cares only about the binary equality, its
hash function(s) can be a lot simpler.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-02-24 14:08 Robert Haas <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Robert Haas @ 2018-02-24 14:08 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On Fri, Feb 23, 2018 at 11:08 AM, Antonin Houska <[email protected]> wrote:
> I spent some more time thinking about this. What about adding a new strategy
> number for hash index operator classes, e.g. HTBinaryEqualStrategyNumber? For
> most types both HTEqualStrategyNumber and HTBinaryEqualStrategyNumber strategy
> would point to the same operator, but types like numeric would naturally have
> them different.
>
> Thus the pushed-down partial aggregation can only use the
> HTBinaryEqualStrategyNumber's operator to compare grouping expressions. In the
> initial version (until we have useful statistics for the binary values) we can
> avoid the aggregation push-down if the grouping expression output type has the
> two strategies implemented using different functions because, as you noted
> upthread, grouping based on binary equality can result in excessive number of
> groups.
>
> One open question is whether the binary equality operator needs a separate
> operator class or not. If an opclass cares only about the binary equality, its
> hash function(s) can be a lot simpler.
Hmm. How about instead adding another regproc field to pg_type which
stores the OID of a function that tests binary equality for that
datatype? If that happens to be equal to the OID you got from the
opclass, then you're all set.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-02-28 16:02 Antonin Houska <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2018-02-28 16:02 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
Robert Haas <[email protected]> wrote:
> On Fri, Dec 22, 2017 at 10:43 AM, Antonin Houska <[email protected]> wrote:
> > Michael Paquier <[email protected]> wrote:
> >> On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <[email protected]> wrote:
> >> > I'm not about to add any other features now. Implementation of the missing
> >> > parts (see the TODO comments in the code) is the next step. But what I'd
> >> > appreciate most is a feedback on the design. Thanks.
> >>
> >> I am getting a conflict after applying patch 5 but this did not get
> >> any reviews so moved to next CF with waiting on author as status.
> >
> > Attached is the next version.
> translate_expression_to_rels() looks unsafe. Equivalence members are
> known to be equal under the semantics of the relevant operator class,
> but that doesn't mean that one can be freely substituted for another.
> For example:
>
> rhaas=# create table one (a numeric);
> CREATE TABLE
> rhaas=# create table two (a numeric);
> CREATE TABLE
> rhaas=# insert into one values ('0');
> INSERT 0 1
> rhaas=# insert into two values ('0.00');
> INSERT 0 1
> rhaas=# select one.a, count(*) from one, two where one.a = two.a group by 1;
> a | count
> ---+-------
> 0 | 1
> (1 row)
>
> rhaas=# select two.a, count(*) from one, two where one.a = two.a group by 1;
> a | count
> ------+-------
> 0.00 | 1
> (1 row)
>
> There are, admittedly, a large number of data types for which such a
> substitution would work just fine. numeric will not, citext will not,
> but many others are fine. Regrettably, we have no framework in the
> system for identifying equality operators which actually test identity
> versus some looser notion of equality. Cross-type operators are a
> problem, too; if we have foo.x = bar.y in the query, one might be int4
> and the other int8, for example, but they can still belong to the same
> equivalence class.
>
> Concretely, in your test query "SELECT p.i, avg(c1.v) FROM
> agg_pushdown_parent AS p JOIN agg_pushdown_child1 AS c1 ON c1.parent =
> p.i GROUP BY p.i" you assume that it's OK to do a Partial
> HashAggregate over c1.parent rather than p.i. This will be false if,
> say, c1.parent is of type citext and p.i is of type text; this will
> get grouped together that shouldn't. It will also be false if the
> grouping expression is something like GROUP BY length(p.i::text),
> because one value could be '0'::numeric and the other '0.00'::numeric.
> I can't think of a reason why it would be false if the grouping
> expressions are both simple Vars of the same underlying data type, but
> I'm a little nervous that I might be wrong even about that case.
> Maybe you've handled all of this somehow, but it's not obvious to me
> that it has been considered.
These problems are still being investigated, see [1]
> Another thing I noticed is that the GroupedPathInfo looks a bit like a
> stripped-down RelOptInfo, in that for example it has both a pathlist
> and a partial_pathlist. I'm inclined to think that we should build new
> RelOptInfos instead. As you have it, there are an awful lot of things
> that seem to know about the grouped/ungrouped distinction, many of
> which are quite low-level functions like add_path(),
> add_paths_to_append_rel(), try_nestloop_path(), etc. I think that
> some of this would go away if you had separate RelOptInfos instead of
> GroupedPathInfo.
I've reworked the patch so that separate RelOptInfo is used for grouped
relation. The attached patch is only the core part. Neither postgres_fdw
changes nor the part that tries to avoid the final aggregation is included
here. I'll add these when the patch does not seem to need another major rework.
[1] https://www.postgresql.org/message-id/[email protected]...
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] agg_pushdown_v6.diff (293.2K, ../../31585.1519833763@localhost/2-agg_pushdown_v6.diff)
download | inline diff:
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index db5fcaf..813249a
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, ExprState *s
*** 794,799 ****
--- 794,840 ----
break;
}
+ case T_GroupedVar:
+
+ /*
+ * If GroupedVar appears in targetlist of Agg node, it can
+ * represent either Aggref or grouping expression.
+ */
+ if (state->parent && (IsA(state->parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * The only reason to execute GroupedVar is to generate either
+ * aggregate transient state or grouping expression value. So
+ * any contained Aggref must be partial.
+ *
+ * (The purpose of propagating GroupedVars to upper plan nodes
+ * is just to transfer the value, no execution takes place
+ * there.)
+ */
+ if (IsA(gvar->gvexpr, Aggref))
+
+ ExecInitExprRec((Expr *) gvar->agg_partial, state,
+ resv, resnull);
+ else
+ ExecInitExprRec((Expr *) gvar->gvexpr, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ *
+ * XXX Should we error out here? There's at least one legal
+ * case here which we'd have to check: a Result plan with no
+ * outer plan which represents an empty Append plan.
+ */
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 266a3ef..dca0653
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyAggref(const Aggref *from)
*** 1367,1372 ****
--- 1367,1373 ----
COPY_SCALAR_FIELD(aggcollid);
COPY_SCALAR_FIELD(inputcollid);
COPY_SCALAR_FIELD(aggtranstype);
+ COPY_SCALAR_FIELD(aggcombinefn);
COPY_NODE_FIELD(aggargtypes);
COPY_NODE_FIELD(aggdirectargs);
COPY_NODE_FIELD(args);
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2219,2224 ****
--- 2220,2241 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** _copyPlaceHolderInfo(const PlaceHolderIn
*** 2292,2297 ****
--- 2309,2329 ----
return newnode;
}
+ static GroupedVarInfo *
+ _copyGroupedVarInfo(const GroupedVarInfo *from)
+ {
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_SCALAR_FIELD(gvid);
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(gv_width);
+
+ return newnode;
+ }
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
*************** copyObjectImpl(const void *from)
*** 5034,5039 ****
--- 5066,5074 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
*************** copyObjectImpl(const void *from)
*** 5046,5051 ****
--- 5081,5089 ----
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index bbffc87..fb26311
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 873,878 ****
--- 873,886 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3179,3184 ****
--- 3187,3195 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index 6c76c41..d34b26b
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 259,264 ****
--- 259,270 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ type = exprType((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprTypmod(const Node *expr)
*** 492,497 ****
--- 498,508 ----
return ((const SetToDefault *) expr)->typeMod;
case T_PlaceHolderVar:
return exprTypmod((Node *) ((const PlaceHolderVar *) expr)->phexpr);
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->gvexpr);
default:
break;
}
*************** exprCollation(const Node *expr)
*** 903,908 ****
--- 914,925 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2176,2181 ****
--- 2193,2200 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2968,2973 ****
--- 2987,3002 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 011d2a3..37d2fb0
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outAggref(StringInfo str, const Aggref
*** 1142,1147 ****
--- 1142,1148 ----
WRITE_OID_FIELD(aggcollid);
WRITE_OID_FIELD(inputcollid);
WRITE_OID_FIELD(aggtranstype);
+ WRITE_OID_FIELD(aggcombinefn);
WRITE_NODE_FIELD(aggargtypes);
WRITE_NODE_FIELD(aggdirectargs);
WRITE_NODE_FIELD(args);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2218,2223 ****
--- 2219,2225 ----
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(join_grouped_rel_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2232,2237 ****
--- 2234,2240 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2239,2244 ****
--- 2242,2248 ----
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
*************** _outRelOptInfo(StringInfo str, const Rel
*** 2278,2283 ****
--- 2282,2288 ----
WRITE_NODE_FIELD(cheapest_parameterized_paths);
WRITE_BITMAPSET_FIELD(direct_lateral_relids);
WRITE_BITMAPSET_FIELD(lateral_relids);
+ WRITE_NODE_FIELD(agg_info);
WRITE_UINT_FIELD(relid);
WRITE_OID_FIELD(reltablespace);
WRITE_ENUM_FIELD(rtekind, RTEKind);
*************** _outParamPathInfo(StringInfo str, const
*** 2454,2459 ****
--- 2459,2476 ----
}
static void
+ _outRelAggInfo(StringInfo str, const RelAggInfo *node)
+ {
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(input);
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2497,2502 ****
--- 2514,2530 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** _outPlaceHolderInfo(StringInfo str, cons
*** 2551,2556 ****
--- 2579,2597 ----
}
static void
+ _outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_UINT_FIELD(gvid);
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_INT_FIELD(gv_width);
+ }
+
+ static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
*************** outNode(StringInfo str, const void *obj)
*** 4060,4071 ****
--- 4101,4118 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
*************** outNode(StringInfo str, const void *obj)
*** 4078,4083 ****
--- 4125,4133 ----
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index 068db35..f5dceb0
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 534,539 ****
--- 534,555 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(sortgroupref);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** _readAggref(void)
*** 589,594 ****
--- 605,611 ----
READ_OID_FIELD(aggcollid);
READ_OID_FIELD(inputcollid);
READ_OID_FIELD(aggtranstype);
+ READ_OID_FIELD(aggcombinefn);
READ_NODE_FIELD(aggargtypes);
READ_NODE_FIELD(aggdirectargs);
READ_NODE_FIELD(args);
*************** parseNodeString(void)
*** 2483,2488 ****
--- 2500,2507 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index 0be2a73..ecff708
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 249,254 ****
--- 249,255 ----
if (force ||
desirable_join(root, old_clump->joinrel, new_clump->joinrel))
{
+ JoinSearchResult *joinrels;
RelOptInfo *joinrel;
/*
*************** merge_clump(PlannerInfo *root, List *clu
*** 257,265 ****
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
*/
! joinrel = make_join_rel(root,
! old_clump->joinrel,
! new_clump->joinrel);
/* Keep searching if join order is not valid */
if (joinrel)
--- 258,267 ----
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
*/
! joinrels = make_join_rel(root,
! old_clump->joinrel,
! new_clump->joinrel);
! joinrel = joinrels->plain;
/* Keep searching if join order is not valid */
if (joinrel)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 1c792a0..be61d30
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** typedef struct pushdown_safety_info
*** 57,62 ****
--- 57,63 ----
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+ bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
*************** set_rel_pathlist_hook_type set_rel_pathl
*** 68,76 ****
join_search_hook_type join_search_hook = NULL;
! static void set_base_rel_consider_startup(PlannerInfo *root);
! static void set_base_rel_sizes(PlannerInfo *root);
! static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
--- 69,77 ----
join_search_hook_type join_search_hook = NULL;
! static void set_base_rel_consider_startup(PlannerInfo *root, bool grouped);
! static void set_base_rel_sizes(PlannerInfo *root, bool grouped);
! static void set_base_rel_pathlists(PlannerInfo *root, bool grouped);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
*************** static void set_namedtuplestore_pathlist
*** 117,123 ****
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
! static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
--- 118,125 ----
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
! static JoinSearchResult *make_rel_from_joinlist(PlannerInfo *root,
! List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
*************** static void add_paths_to_append_rel(Plan
*** 141,152 ****
/*
* make_one_rel
* Finds all possible access paths for executing a query, returning a
! * single rel that represents the join of all base rels in the query.
*/
! RelOptInfo *
make_one_rel(PlannerInfo *root, List *joinlist)
{
! RelOptInfo *rel;
Index rti;
/*
--- 143,155 ----
/*
* make_one_rel
* Finds all possible access paths for executing a query, returning a
! * single rel that represents the join of all base rels in the query. If
! * possible, also return a join that contains partial aggregate(s).
*/
! JoinSearchResult *
make_one_rel(PlannerInfo *root, List *joinlist)
{
! JoinSearchResult *rels;
Index rti;
/*
*************** make_one_rel(PlannerInfo *root, List *jo
*** 170,196 ****
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
! /* Mark base rels as to whether we care about fast-start plans */
! set_base_rel_consider_startup(root);
/*
! * Compute size estimates and consider_parallel flags for each base rel,
! * then generate access paths.
*/
! set_base_rel_sizes(root);
! set_base_rel_pathlists(root);
/*
* Generate access paths for the entire join tree.
*/
! rel = make_rel_from_joinlist(root, joinlist);
/*
* The result should join all and only the query's base rels.
*/
! Assert(bms_equal(rel->relids, root->all_baserels));
! return rel;
}
/*
--- 173,215 ----
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
! /*
! * Mark base rels as to whether we care about fast-start plans. XXX We
! * deliberately do not mark grouped rels --- see the comment on
! * consider_startup in build_simple_rel().
! */
! set_base_rel_consider_startup(root, false);
/*
! * As for grouped relations, paths differ substantially by the
! * AggStrategy. Paths that use AGG_HASHED should not be parameterized
! * (because creation of hashtable would have to be repeated for different
! * parameters) but paths using AGG_SORTED can be. The latter seem to
! * justify calling the function for grouped relations too.
*/
! set_base_rel_consider_startup(root, true);
!
! /*
! * Compute size estimates and consider_parallel flags for each plain and
! * each grouped base rel, then generate access paths.
! */
! set_base_rel_sizes(root, false);
! set_base_rel_pathlists(root, false);
!
! set_base_rel_sizes(root, true);
! set_base_rel_pathlists(root, true);
/*
* Generate access paths for the entire join tree.
*/
! rels = make_rel_from_joinlist(root, joinlist);
/*
* The result should join all and only the query's base rels.
*/
! Assert(rels->plain && bms_equal(rels->plain->relids, root->all_baserels));
! return rels;
}
/*
*************** make_one_rel(PlannerInfo *root, List *jo
*** 204,210 ****
* be better to move it here.
*/
static void
! set_base_rel_consider_startup(PlannerInfo *root)
{
/*
* Since parameterized paths can only be used on the inside of a nestloop
--- 223,229 ----
* be better to move it here.
*/
static void
! set_base_rel_consider_startup(PlannerInfo *root, bool grouped)
{
/*
* Since parameterized paths can only be used on the inside of a nestloop
*************** set_base_rel_consider_startup(PlannerInf
*** 229,237 ****
if ((sjinfo->jointype == JOIN_SEMI || sjinfo->jointype == JOIN_ANTI) &&
bms_get_singleton_member(sjinfo->syn_righthand, &varno))
{
! RelOptInfo *rel = find_base_rel(root, varno);
! rel->consider_param_startup = true;
}
}
}
--- 248,262 ----
if ((sjinfo->jointype == JOIN_SEMI || sjinfo->jointype == JOIN_ANTI) &&
bms_get_singleton_member(sjinfo->syn_righthand, &varno))
{
! RelOptInfo *rel = !grouped ? find_base_rel(root, varno) :
! find_grouped_base_rel(root, varno);
! if (rel != NULL)
! rel->consider_param_startup = true;
! #ifdef USE_ASSERT_CHECKING
! else
! Assert(grouped);
! #endif /* USE_ASSERT_CHECKING */
}
}
}
*************** set_base_rel_consider_startup(PlannerInf
*** 247,262 ****
* generate paths.
*/
static void
! set_base_rel_sizes(PlannerInfo *root)
{
Index rti;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = root->simple_rel_array[rti];
RangeTblEntry *rte;
! /* there may be empty slots corresponding to non-baserel RTEs */
if (rel == NULL)
continue;
--- 272,294 ----
* generate paths.
*/
static void
! set_base_rel_sizes(PlannerInfo *root, bool grouped)
{
Index rti;
+ RelOptInfo **rel_array;
+
+ rel_array = !grouped ? root->simple_rel_array :
+ root->simple_grouped_rel_array;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = rel_array[rti];
RangeTblEntry *rte;
! /*
! * There may be empty slots corresponding to non-baserel RTEs, or to
! * baserel which cannot be grouped.
! */
if (rel == NULL)
continue;
*************** set_base_rel_sizes(PlannerInfo *root)
*** 290,304 ****
* Each useful path is attached to its relation's 'pathlist' field.
*/
static void
! set_base_rel_pathlists(PlannerInfo *root)
{
Index rti;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = root->simple_rel_array[rti];
! /* there may be empty slots corresponding to non-baserel RTEs */
if (rel == NULL)
continue;
--- 322,343 ----
* Each useful path is attached to its relation's 'pathlist' field.
*/
static void
! set_base_rel_pathlists(PlannerInfo *root, bool grouped)
{
Index rti;
+ RelOptInfo **rel_array;
+
+ rel_array = !grouped ? root->simple_rel_array :
+ root->simple_grouped_rel_array;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = rel_array[rti];
! /*
! * There may be empty slots corresponding to non-baserel RTEs, or to
! * baserel which cannot be grouped.
! */
if (rel == NULL)
continue;
*************** static void
*** 320,325 ****
--- 359,372 ----
set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte)
{
+ bool grouped = rel->agg_info != NULL;
+
+ /*
+ * build_simple_rel() should not have created rels that do not match this
+ * condition.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (rel->reloptkind == RELOPT_BASEREL &&
relation_excluded_by_constraints(root, rel, rte))
{
*************** set_rel_size(PlannerInfo *root, RelOptIn
*** 349,354 ****
--- 396,403 ----
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
/* Foreign table */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_foreign_size(root, rel, rte);
}
else if (rte->relkind == RELKIND_PARTITIONED_TABLE)
*************** set_rel_size(PlannerInfo *root, RelOptIn
*** 362,367 ****
--- 411,418 ----
else if (rte->tablesample != NULL)
{
/* Sampled relation */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_tablesample_rel_size(root, rel, rte);
}
else
*************** static void
*** 423,428 ****
--- 474,487 ----
set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte)
{
+ bool grouped = rel->agg_info != NULL;
+
+ /*
+ * build_simple_rel() should not have created rels that do not match this
+ * condition.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (IS_DUMMY_REL(rel))
{
/* We already proved the relation empty, so nothing more to do */
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 440,450 ****
--- 499,513 ----
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
/* Foreign table */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_foreign_pathlist(root, rel, rte);
}
else if (rte->tablesample != NULL)
{
/* Sampled relation */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_tablesample_rel_pathlist(root, rel, rte);
}
else
*************** static void
*** 689,694 ****
--- 752,759 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
+ bool grouped = rel->agg_info != NULL;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 697,714 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
! /* Consider index scans */
create_index_paths(root, rel);
/* Consider TID scans */
! create_tidscan_paths(root, rel);
}
/*
--- 762,796 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! /*
! * It's probably not good idea to repeat hashed aggregation with different
! * parameters, so check if there are no parameters.
! */
! if (!grouped)
! add_path(rel, seq_path);
! else if (required_outer == NULL)
! {
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! }
!
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
! /*
! * Consider index scans.
! */
create_index_paths(root, rel);
/* Consider TID scans */
! create_tidscan_paths(root, rel, grouped);
}
/*
*************** static void
*** 719,734 ****
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
! parallel_workers = compute_parallel_worker(rel, rel->pages, -1,
! max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 801,907 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
+ bool grouped = rel->agg_info != NULL;
! parallel_workers = compute_parallel_worker(rel, rel->pages, -1, max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
!
! if (!grouped)
! add_partial_path(rel, path);
! else
! {
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it. Only AGG_HASHED is suitable here as it does not
! * expect the input set to be sorted.
! */
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck() / add_partial_path_precheck().
! *
! * If "partial" is true, the aggregation path is considered partial in terms
! * of parallel execution.
! *
! * The return value tells whether the path was added to the pathlist.
! */
! bool
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! Path *agg_path;
! RelAggInfo *agg_info = rel->agg_info;
!
! Assert(agg_info != NULL);
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Repeated creation of hash table does not sound like a good idea. Caller
! * should avoid asking us to do so.
! */
! Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL))
! return false;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost,
! pathkeys))
! return false;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path);
! else
! add_partial_path(rel, (Path *) agg_path);
!
! return true;
! }
!
! return false;
}
/*
*************** set_append_rel_size(PlannerInfo *root, R
*** 869,874 ****
--- 1042,1048 ----
double *parent_attrsizes;
int nattrs;
ListCell *l;
+ bool grouped = rel->agg_info != NULL;
/* Guard against stack overflow due to overly deep inheritance tree. */
check_stack_depth();
*************** set_append_rel_size(PlannerInfo *root, R
*** 919,925 ****
* The child rel's RelOptInfo was already created during
* add_base_rels_to_query.
*/
! childrel = find_base_rel(root, childRTindex);
Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL);
if (rel->part_scheme)
--- 1093,1100 ----
* The child rel's RelOptInfo was already created during
* add_base_rels_to_query.
*/
! childrel = !grouped ? find_base_rel(root, childRTindex) :
! find_grouped_base_rel(root, childRTindex);
Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL);
if (rel->part_scheme)
*************** set_append_rel_size(PlannerInfo *root, R
*** 929,938 ****
/*
* We need attr_needed data for building targetlist of a join
* relation representing join between matching partitions for
! * partitionwise join. A given attribute of a child will be
! * needed in the same highest joinrel where the corresponding
! * attribute of parent is needed. Hence it suffices to use the
! * same Relids set for parent and child.
*/
for (attno = rel->min_attr; attno <= rel->max_attr; attno++)
{
--- 1104,1113 ----
/*
* We need attr_needed data for building targetlist of a join
* relation representing join between matching partitions for
! * partitionwise join. A given attribute of a child will be needed
! * in the same highest joinrel where the corresponding attribute
! * of parent is needed. Hence it suffices to use the same Relids
! * set for parent and child.
*/
for (attno = rel->min_attr; attno <= rel->max_attr; attno++)
{
*************** set_append_rel_size(PlannerInfo *root, R
*** 982,991 ****
* PlaceHolderVars.) XXX we do not bother to update the cost or width
* fields of childrel->reltarget; not clear if that would be useful.
*/
! childrel->reltarget->exprs = (List *)
! adjust_appendrel_attrs(root,
! (Node *) rel->reltarget->exprs,
! 1, &appinfo);
/*
* We have to make child entries in the EquivalenceClass data
--- 1157,1190 ----
* PlaceHolderVars.) XXX we do not bother to update the cost or width
* fields of childrel->reltarget; not clear if that would be useful.
*/
! if (grouped)
! {
! /*
! * Special attention is needed in the grouped case.
! *
! * build_simple_rel() didn't create empty target because it's
! * better to start with copying one from the parent rel.
! */
! Assert(childrel->reltarget == NULL && childrel->agg_info == NULL);
! Assert(rel->reltarget != NULL && rel->agg_info != NULL);
!
! /*
! * Translate the targets and grouping expressions so they match
! * this child.
! */
! childrel->agg_info = translate_rel_agg_info(root, rel->agg_info,
! &appinfo, 1);
!
! /*
! * The relation paths will generate input for partial aggregation.
! */
! childrel->reltarget = childrel->agg_info->input;
! }
! else
! childrel->reltarget->exprs = (List *)
! adjust_appendrel_attrs(root,
! (Node *) rel->reltarget->exprs,
! 1, &appinfo);
/*
* We have to make child entries in the EquivalenceClass data
*************** set_append_rel_size(PlannerInfo *root, R
*** 1140,1145 ****
--- 1339,1364 ----
1, &appinfo);
/*
+ * We have to make child entries in the EquivalenceClass data
+ * structures as well. This is needed either if the parent
+ * participates in some eclass joins (because we will want to consider
+ * inner-indexscan joins on the individual children) or if the parent
+ * has useful pathkeys (because we should try to build MergeAppend
+ * paths that produce those sort orderings).
+ */
+ if (rel->has_eclass_joins || has_useful_pathkeys(root, rel))
+ add_child_rel_equivalences(root, appinfo, rel, childrel);
+ childrel->has_eclass_joins = rel->has_eclass_joins;
+
+ /*
+ * Note: we could compute appropriate attr_needed data for the child's
+ * variables, by transforming the parent's attr_needed through the
+ * translated_vars mapping. However, currently there's no need
+ * because attr_needed is only examined for base relations not
+ * otherrels. So we just leave the child's attr_needed empty.
+ */
+
+ /*
* If parallelism is allowable for this query in general, see whether
* it's allowable for this childrel in particular. But if we've
* already decided the appendrel is not parallel-safe as a whole,
*************** set_append_rel_pathlist(PlannerInfo *roo
*** 1263,1268 ****
--- 1482,1488 ----
int parentRTindex = rti;
List *live_childrels = NIL;
ListCell *l;
+ bool grouped = rel->agg_info != NULL;
/*
* Generate access paths for each member relation, and remember the
*************** set_append_rel_pathlist(PlannerInfo *roo
*** 1282,1288 ****
/* Re-locate the child RTE and RelOptInfo */
childRTindex = appinfo->child_relid;
childRTE = root->simple_rte_array[childRTindex];
! childrel = root->simple_rel_array[childRTindex];
/*
* If set_append_rel_size() decided the parent appendrel was
--- 1502,1510 ----
/* Re-locate the child RTE and RelOptInfo */
childRTindex = appinfo->child_relid;
childRTE = root->simple_rte_array[childRTindex];
! childrel = !grouped ?
! find_base_rel(root, childRTindex) :
! find_grouped_base_rel(root, childRTindex);
/*
* If set_append_rel_size() decided the parent appendrel was
*************** generate_mergeappend_paths(PlannerInfo *
*** 1732,1737 ****
--- 1954,1960 ----
List *partitioned_rels)
{
ListCell *lcp;
+ PathTarget *target = NULL;
foreach(lcp, all_child_pathkeys)
{
*************** generate_mergeappend_paths(PlannerInfo *
*** 1740,1762 ****
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
Path *cheapest_startup,
*cheapest_total;
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
TOTAL_COST,
--- 1963,1987 ----
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
+ Path *path;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
+ List *pathlist = childrel->pathlist;
Path *cheapest_startup,
*cheapest_total;
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
TOTAL_COST,
*************** generate_mergeappend_paths(PlannerInfo *
*** 1789,1807 ****
}
/* ... and build the MergeAppend paths */
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels));
if (startup_neq_total)
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels));
}
}
--- 2014,2041 ----
}
/* ... and build the MergeAppend paths */
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
!
! add_path(rel, path);
!
if (startup_neq_total)
! {
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
! add_path(rel, path);
! }
!
}
}
*************** generate_gather_paths(PlannerInfo *root,
*** 2508,2518 ****
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
! static RelOptInfo *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
! List *initial_rels;
ListCell *jl;
/*
--- 2742,2753 ----
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
! static JoinSearchResult *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
! List *initial_rels,
! *initial_rels_grouped;
ListCell *jl;
/*
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2531,2568 ****
* sub-joinlists.
*/
initial_rels = NIL;
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
! RelOptInfo *thisrel;
if (IsA(jlnode, RangeTblRef))
{
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
}
else if (IsA(jlnode, List))
{
/* Recurse to handle subproblem */
! thisrel = make_rel_from_joinlist(root, (List *) jlnode);
}
else
{
elog(ERROR, "unrecognized joinlist node type: %d",
(int) nodeTag(jlnode));
thisrel = NULL; /* keep compiler quiet */
}
initial_rels = lappend(initial_rels, thisrel);
}
if (levels_needed == 1)
{
/*
* Single joinlist node, so we're done.
*/
! return (RelOptInfo *) linitial(initial_rels);
}
else
{
--- 2766,2817 ----
* sub-joinlists.
*/
initial_rels = NIL;
+ initial_rels_grouped = NIL;
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
! RelOptInfo *thisrel,
! *thisrel_grouped;
if (IsA(jlnode, RangeTblRef))
{
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+ thisrel_grouped = find_grouped_base_rel(root, varno);
}
else if (IsA(jlnode, List))
{
+ JoinSearchResult *rels;
+
/* Recurse to handle subproblem */
! rels = make_rel_from_joinlist(root, (List *) jlnode);
! thisrel = rels->plain;
! thisrel_grouped = rels->grouped;
}
else
{
elog(ERROR, "unrecognized joinlist node type: %d",
(int) nodeTag(jlnode));
thisrel = NULL; /* keep compiler quiet */
+ thisrel_grouped = NULL;
}
initial_rels = lappend(initial_rels, thisrel);
+ initial_rels_grouped = lappend(initial_rels_grouped, thisrel_grouped);
}
if (levels_needed == 1)
{
+ JoinSearchResult *result;
+
/*
* Single joinlist node, so we're done.
*/
! result = (JoinSearchResult *) palloc(sizeof(JoinSearchResult));
! result->plain = (RelOptInfo *) linitial(initial_rels);
! result->grouped = (RelOptInfo *) linitial(initial_rels_grouped);
! return result;
}
else
{
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2576,2586 ****
root->initial_rels = initial_rels;
if (join_search_hook)
! return (*join_search_hook) (root, levels_needed, initial_rels);
else if (enable_geqo && levels_needed >= geqo_threshold)
! return geqo(root, levels_needed, initial_rels);
else
! return standard_join_search(root, levels_needed, initial_rels);
}
}
--- 2825,2849 ----
root->initial_rels = initial_rels;
if (join_search_hook)
! return (*join_search_hook) (root, levels_needed,
! initial_rels,
! initial_rels_grouped);
else if (enable_geqo && levels_needed >= geqo_threshold)
! {
! JoinSearchResult *result;
!
! /*
! * TODO Teach GEQO about grouped relations. Don't forget that
! * pathlist can be NIL before set_cheapest() gets called.
! */
! result = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
! result->plain = geqo(root, levels_needed, initial_rels);
! return result;
! }
else
! return standard_join_search(root, levels_needed,
! initial_rels,
! initial_rels_grouped);
}
}
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2596,2601 ****
--- 2859,2868 ----
* jointree item. These are the components to be joined together.
* Note that levels_needed == list_length(initial_rels).
*
+ * 'initial_rels_grouped' is a list where i-th position is either RelOptInfo
+ * representing i-th item of 'initial_rels' grouped or NULL if there's no
+ * such grouped relation.
+ *
* Returns the final level of join relations, i.e., the relation that is
* the result of joining all the original relations together.
* At least one implementation path must be provided for this relation and
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2613,2629 ****
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
! RelOptInfo *
! standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
{
int lev;
! RelOptInfo *rel;
/*
* This function cannot be invoked recursively within any one planning
! * problem, so join_rel_level[] can't be in use already.
*/
Assert(root->join_rel_level == NULL);
/*
* We employ a simple "dynamic programming" algorithm: we first find all
--- 2880,2901 ----
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
! JoinSearchResult *
! standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels,
! List *initial_rels_grouped)
{
int lev;
! Size levels_size;
! List *top_list;
! JoinSearchResult *result;
/*
* This function cannot be invoked recursively within any one planning
! * problem, so join_rel_level[] / join_grouped_rel_level[] can't be in use
! * already.
*/
Assert(root->join_rel_level == NULL);
+ Assert(root->join_grouped_rel_level == NULL);
/*
* We employ a simple "dynamic programming" algorithm: we first find all
*************** standard_join_search(PlannerInfo *root,
*** 2636,2671 ****
* set root->join_rel_level[1] to represent all the single-jointree-item
* relations.
*/
! root->join_rel_level = (List **) palloc0((levels_needed + 1) * sizeof(List *));
!
root->join_rel_level[1] = initial_rels;
for (lev = 2; lev <= levels_needed; lev++)
{
ListCell *lc;
/*
* Determine all possible pairs of relations to be joined at this
* level, and build paths for making each one from every available
* pair of lower-level relations.
*/
join_search_one_level(root, lev);
/*
! * Run generate_partitionwise_join_paths() and
! * generate_gather_paths() for each just-processed joinrel. We could
! * not do this earlier because both regular and partial paths can get
! * added to a particular joinrel at multiple times within
! * join_search_one_level.
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
*/
! foreach(lc, root->join_rel_level[lev])
{
! rel = (RelOptInfo *) lfirst(lc);
! /* Create paths for partitionwise joins. */
generate_partitionwise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
--- 2908,2952 ----
* set root->join_rel_level[1] to represent all the single-jointree-item
* relations.
*/
! levels_size = (levels_needed + 1) * sizeof(List *);
! root->join_rel_level = (List **) palloc0(levels_size);
root->join_rel_level[1] = initial_rels;
+ root->join_grouped_rel_level = (List **) palloc0(levels_size);
+ root->join_grouped_rel_level[1] = initial_rels_grouped;
for (lev = 2; lev <= levels_needed; lev++)
{
+ List *levels;
ListCell *lc;
/*
* Determine all possible pairs of relations to be joined at this
* level, and build paths for making each one from every available
* pair of lower-level relations.
+ *
+ * This step includes creation of grouped relations.
*/
join_search_one_level(root, lev);
/*
! * Run generate_partitionwise_join_paths() and generate_gather_paths()
! * for each just-processed joinrel. We could not do this earlier
! * because both regular and partial paths can get added to a
! * particular joinrel at multiple times within join_search_one_level.
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
+ *
+ * This processing makes no difference betweend plain and grouped
+ * rels, so process them in the same loop.
*/
! levels = list_concat(list_copy(root->join_rel_level[lev]),
! root->join_grouped_rel_level[lev]);
! foreach(lc, levels)
{
! RelOptInfo *rel = lfirst_node(RelOptInfo, lc);
! /* Create paths for partition-wise joins. */
generate_partitionwise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
*************** standard_join_search(PlannerInfo *root,
*** 2681,2697 ****
}
/*
! * We should have a single rel at the final level.
*/
! if (root->join_rel_level[levels_needed] == NIL)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
! Assert(list_length(root->join_rel_level[levels_needed]) == 1);
! rel = (RelOptInfo *) linitial(root->join_rel_level[levels_needed]);
root->join_rel_level = NULL;
! return rel;
}
/*****************************************************************************
--- 2962,2989 ----
}
/*
! * We should have a single plain rel at the final level.
*/
! if ((top_list = root->join_rel_level[levels_needed]) == NIL)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
! Assert(list_length(top_list) == 1);
! result = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
! result->plain = linitial_node(RelOptInfo, top_list);
!
! /*
! * Grouped relation might have been created too.
! */
! if ((top_list = root->join_grouped_rel_level[levels_needed]) != NIL)
! {
! Assert(list_length(top_list) == 1);
! result->grouped = linitial_node(RelOptInfo, top_list);
! }
root->join_rel_level = NULL;
+ root->join_grouped_rel_level = NULL;
! return result;
}
/*****************************************************************************
*************** create_partial_bitmap_paths(PlannerInfo
*** 3311,3316 ****
--- 3603,3609 ----
{
int parallel_workers;
double pages_fetched;
+ Path *bmhpath;
/* Compute heap pages for bitmap heap scan */
pages_fetched = compute_bitmap_pages(root, rel, bitmapqual, 1.0,
*************** create_partial_bitmap_paths(PlannerInfo
*** 3322,3329 ****
if (parallel_workers <= 0)
return;
! add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3615,3635 ----
if (parallel_workers <= 0)
return;
! bmhpath = (Path *) create_bitmap_heap_path(root, rel, bitmapqual,
! rel->lateral_relids, 1.0,
! parallel_workers);
!
! if (rel->agg_info == NULL)
! add_partial_path(rel, bmhpath);
! else
! {
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, rel, (Path *) bmhpath, false, true,
! AGG_HASHED);
! }
}
/*
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index d8db0b2..14146bd
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
***************
*** 91,96 ****
--- 91,97 ----
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
*************** cost_bitmap_tree_node(Path *path, Cost *
*** 1065,1070 ****
--- 1066,1082 ----
*cost = path->total_cost;
*selec = ((BitmapOrPath *) path)->bitmapselectivity;
}
+ else if (IsA(path, AggPath))
+ {
+ /*
+ * If partial aggregation was already applied, use only the input
+ * path.
+ *
+ * TODO Take the aggregation into account, both cost and its effect on
+ * selectivity (i.e. how it reduces the number of rows).
+ */
+ cost_bitmap_tree_node(((AggPath *) path)->subpath, cost, selec);
+ }
else
{
elog(ERROR, "unrecognized node type: %d", nodeTag(path));
*************** cost_group(Path *path, PlannerInfo *root
*** 2287,2292 ****
--- 2299,2330 ----
path->total_cost = total_cost;
}
+ static void
+ estimate_join_rows(PlannerInfo *root, Path *path, RelAggInfo *agg_info)
+ {
+ bool grouped = agg_info != NULL;
+
+ if (path->param_info)
+ {
+ double nrows;
+
+ path->rows = path->param_info->ppi_rows;
+ if (grouped)
+ {
+ nrows = estimate_num_groups(root, agg_info->group_exprs,
+ path->rows, NULL);
+ path->rows = clamp_row_est(nrows);
+ }
+ }
+ else
+ {
+ if (!grouped)
+ path->rows = path->parent->rows;
+ else
+ path->rows = agg_info->rows;
+ }
+ }
+
/*
* initial_cost_nestloop
* Preliminary estimate of the cost of a nestloop join path.
*************** final_cost_nestloop(PlannerInfo *root, N
*** 2408,2417 ****
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! if (path->path.param_info)
! path->path.rows = path->path.param_info->ppi_rows;
! else
! path->path.rows = path->path.parent->rows;
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
--- 2446,2452 ----
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! estimate_join_rows(root, (Path *) path, path->path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
*************** final_cost_mergejoin(PlannerInfo *root,
*** 2854,2863 ****
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! if (path->jpath.path.param_info)
! path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
! else
! path->jpath.path.rows = path->jpath.path.parent->rows;
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
--- 2889,2896 ----
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! estimate_join_rows(root, (Path *) path,
! path->jpath.path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
*************** final_cost_hashjoin(PlannerInfo *root, H
*** 3277,3286 ****
ListCell *hcl;
/* Mark the path with the correct row estimate */
! if (path->jpath.path.param_info)
! path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
! else
! path->jpath.path.rows = path->jpath.path.parent->rows;
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
--- 3310,3317 ----
ListCell *hcl;
/* Mark the path with the correct row estimate */
! estimate_join_rows(root, (Path *) path,
! path->jpath.path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
*************** cost_qual_eval_walker(Node *node, cost_q
*** 3803,3810 ****
* estimated execution cost given by pg_proc.procost (remember to multiply
* this by cpu_operator_cost).
*
! * Vars and Consts are charged zero, and so are boolean operators (AND,
! * OR, NOT). Simplistic, but a lot better than no model at all.
*
* Should we try to account for the possibility of short-circuit
* evaluation of AND/OR? Probably *not*, because that would make the
--- 3834,3842 ----
* estimated execution cost given by pg_proc.procost (remember to multiply
* this by cpu_operator_cost).
*
! * Vars, GroupedVars and Consts are charged zero, and so are boolean
! * operators (AND, OR, NOT). Simplistic, but a lot better than no model at
! * all.
*
* Should we try to account for the possibility of short-circuit
* evaluation of AND/OR? Probably *not*, because that would make the
*************** approx_tuple_count(PlannerInfo *root, Jo
*** 4283,4293 ****
--- 4315,4327 ----
* restriction clauses).
* width: the estimated average output tuple width in bytes.
* baserestrictcost: estimated cost of evaluating baserestrictinfo clauses.
+ * grouped: will partial aggregation be applied to each path?
*/
void
set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
{
double nrows;
+ bool grouped = rel->agg_info != NULL;
/* Should only be applied to base relations */
Assert(rel->relid > 0);
*************** set_baserel_size_estimates(PlannerInfo *
*** 4298,4309 ****
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel);
}
/*
--- 4332,4362 ----
0,
JOIN_INNER,
NULL);
rel->rows = clamp_row_est(nrows);
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ if (grouped)
+ {
+ nrows = estimate_num_groups(root,
+ rel->agg_info->group_exprs, nrows,
+ NULL);
+ rel->agg_info->rows = clamp_row_est(nrows);
+ }
+
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! /*
! * The grouped target should have the cost and width set immediately on
! * creation, see create_rel_agg_info().
! */
! if (!grouped)
! set_rel_width(root, rel);
! #ifdef USE_ASSERT_CHECKING
! else
! Assert(rel->reltarget->width > 0);
! #endif
}
/*
*************** set_pathtarget_cost_width(PlannerInfo *r
*** 5285,5290 ****
--- 5338,5358 ----
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+ GroupedVarInfo *gvinfo;
+
+ gvinfo = find_grouped_var_info(root, gvar);
+ tuple_width += gvinfo->gv_width;
+
+ /*
+ * Only AggPath can evaluate GroupedVar, whether it's an aggregate
+ * or generic grouping expression. In the other cases the
+ * GroupedVar we see here only bubbled up from a lower AggPath, so
+ * it does not add any cost to the path that owns this target.
+ */
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
new file mode 100644
index 70a925c..6c0a33f
*** a/src/backend/optimizer/path/equivclass.c
--- b/src/backend/optimizer/path/equivclass.c
*************** static bool reconsider_outer_join_clause
*** 65,70 ****
--- 65,83 ----
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
+ typedef struct translate_expr_context
+ {
+ Var **keys; /* Dictionary keys. */
+ Var **values; /* Dictionary values */
+ int nitems; /* Number of dictionary items. */
+ Relids *gv_eval_at_p; /* See GroupedVarInfo. */
+ Index relid; /* Translate into this relid. */
+ } translate_expr_context;
+
+ static Node *translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context *context);
+ static int var_dictionary_comparator(const void *a, const void *b);
+
/*
* process_equivalence
*************** is_redundant_derived_clause(RestrictInfo
*** 2510,2512 ****
--- 2523,2844 ----
return false;
}
+
+ /*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid.
+ */
+ GroupedVarInfo *
+ translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+ {
+ List *vars;
+ ListCell *l1;
+ int i,
+ j;
+ int nkeys,
+ nkeys_resolved;
+ Var **keys,
+ **values,
+ **keys_tmp;
+ Var *key,
+ *key_prev;
+ translate_expr_context context;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ /*
+ * Before actually trying to modify the expression tree, find out if all
+ * vars can be translated.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+
+ /* No vars to translate? */
+ if (vars == NIL)
+ return NULL;
+
+ /*
+ * Search for individual replacement vars as well as the actual expression
+ * translation will be more efficient if we use a dictionary with the keys
+ * (i.e. the "source vars") unique and sorted.
+ */
+ nkeys = list_length(vars);
+ keys = (Var **) palloc(nkeys * sizeof(Var *));
+ i = 0;
+ foreach(l1, vars)
+ {
+ key = lfirst_node(Var, l1);
+ keys[i++] = key;
+ }
+
+ /*
+ * Sort the keys by varno. varattno decides where varnos are equal.
+ */
+ if (nkeys > 1)
+ pg_qsort(keys, nkeys, sizeof(Var *), var_dictionary_comparator);
+
+ /*
+ * Pick unique values and get rid of the vars that need no translation.
+ */
+ keys_tmp = (Var **) palloc(nkeys * sizeof(Var *));
+ key_prev = NULL;
+ j = 0;
+ for (i = 0; i < nkeys; i++)
+ {
+ key = keys[i];
+
+ if ((key_prev == NULL || (key->varno != key_prev->varno &&
+ key->varattno != key_prev->varattno)) &&
+ key->varno != relid)
+ keys_tmp[j++] = key;
+
+ key_prev = key;
+ }
+ pfree(keys);
+ keys = keys_tmp;
+ nkeys = j;
+
+ /*
+ * Is there actually nothing to be translated?
+ */
+ if (nkeys == 0)
+ {
+ pfree(keys);
+ return NULL;
+ }
+
+ nkeys_resolved = 0;
+
+ /*
+ * Find the replacement vars.
+ */
+ values = (Var **) palloc0(nkeys * sizeof(Var *));
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ Relids ec_var_relids;
+ Var **ec_vars;
+ int ec_nvars;
+ ListCell *l2;
+
+ /* TODO Re-check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ ec_vars = (Var **) palloc(list_length(ec->ec_members) * sizeof(Var *));
+ ec_nvars = 0;
+ ec_var_relids = NULL;
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ ec_vars[ec_nvars++] = ec_var;
+ ec_var_relids = bms_add_member(ec_var_relids, ec_var->varno);
+ }
+
+ /*
+ * At least two vars are needed so that the EC is usable for
+ * translation.
+ */
+ if (ec_nvars <= 1)
+ {
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+ continue;
+ }
+
+ /*
+ * Now check where this EC can help.
+ */
+ for (i = 0; i < nkeys; i++)
+ {
+ Relids ec_rest;
+ bool relid_ok,
+ key_found;
+ Var *key = keys[i];
+ Var *value = values[i];
+
+ /* Skip this item if it's already resolved. */
+ if (value != NULL)
+ continue;
+
+ /*
+ * Can't translate if the EC does not mention key->varno.
+ */
+ if (!bms_is_member(key->varno, ec_var_relids))
+ continue;
+
+ /*
+ * Besides key, at least one EC member must belong to the relation
+ * we're translating our expression to.
+ */
+ ec_rest = bms_copy(ec_var_relids);
+ ec_rest = bms_del_member(ec_rest, key->varno);
+ relid_ok = bms_is_member(relid, ec_rest);
+ bms_free(ec_rest);
+ if (!relid_ok)
+ continue;
+
+ /*
+ * The preliminary checks passed, so try to find the exact vars.
+ */
+ key_found = false;
+ for (j = 0; j < ec_nvars; j++)
+ {
+ Var *ec_var = ec_vars[j];
+
+ if (!key_found && key->varno == ec_var->varno &&
+ key->varattno == ec_var->varattno)
+ key_found = true;
+
+ /*
+ *
+ * Is this Var useful for our dictionary?
+ *
+ * XXX Shouldn't ec_var be copied?
+ */
+ if (value == NULL && ec_var->varno == relid)
+ value = ec_var;
+
+ if (key_found && value != NULL)
+ break;
+ }
+
+ if (key_found && value != NULL)
+ {
+ values[i] = value;
+ nkeys_resolved++;
+
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+ }
+
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+
+ /* Don't need to check the remaining ECs? */
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+
+ /* Couldn't compose usable dictionary? */
+ if (nkeys_resolved < nkeys)
+ {
+ pfree(keys);
+ pfree(values);
+ return NULL;
+ }
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_copy(result->gv_eval_at);
+
+ /* The dictionary is ready, so perform the translation. */
+ context.keys = keys;
+ context.values = values;
+ context.nitems = nkeys;
+ context.gv_eval_at_p = &result->gv_eval_at;
+ context.relid = relid;
+ result->gvexpr = (Expr *)
+ translate_expression_to_rels_mutator((Node *) gvi->gvexpr, &context);
+
+ pfree(keys);
+ pfree(values);
+ return result;
+ }
+
+ static Node *
+ translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context *context)
+ {
+ if (node == NULL)
+ return NULL;
+
+ if (IsA(node, Var))
+ {
+ Var *var = castNode(Var, node);
+ Var **key_p;
+ Var *value;
+ int index;
+
+ /*
+ * Simply return the existing variable if already belongs to the
+ * relation we're adjusting the expression to.
+ */
+ if (var->varno == context->relid)
+ return (Node *) var;
+
+ key_p = bsearch(&var, context->keys, context->nitems, sizeof(Var *),
+ var_dictionary_comparator);
+
+ /* We shouldn't have omitted any var from the dictionary. */
+ Assert(key_p != NULL);
+
+ index = key_p - context->keys;
+ Assert(index >= 0 && index < context->nitems);
+ value = context->values[index];
+
+ /* All values should be present in the dictionary. */
+ Assert(value != NULL);
+
+ /* Update gv_eval_at accordingly. */
+ bms_del_member(*context->gv_eval_at_p, var->varno);
+ *context->gv_eval_at_p = bms_add_member(*context->gv_eval_at_p,
+ value->varno);
+
+ return (Node *) value;
+ }
+
+ return expression_tree_mutator(node, translate_expression_to_rels_mutator,
+ (void *) context);
+ }
+
+ static int
+ var_dictionary_comparator(const void *a, const void *b)
+ {
+ Var **var1_p,
+ **var2_p;
+ Var *var1,
+ *var2;
+
+ var1_p = (Var **) a;
+ var1 = castNode(Var, *var1_p);
+ var2_p = (Var **) b;
+ var2 = castNode(Var, *var2_p);
+
+ if (var1->varno < var2->varno)
+ return -1;
+ else if (var1->varno > var2->varno)
+ return 1;
+
+ if (var1->varattno < var2->varattno)
+ return -1;
+ else if (var1->varattno > var2->varattno)
+ return 1;
+
+ return 0;
+ }
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 7fc7080..1e05faf
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** typedef struct
*** 78,84 ****
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
-
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
--- 79,84 ----
*************** static Const *string_to_const(const char
*** 227,232 ****
--- 227,234 ----
* index quals ... but for now, it doesn't seem worth troubling over.
* In particular, comments below about "unparameterized" paths should be read
* as meaning "unparameterized so far as the indexquals are concerned".
+ *
+ * If agg_info is passed, grouped paths are generated too.
*/
void
create_index_paths(PlannerInfo *root, RelOptInfo *rel)
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,282 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
*************** create_index_paths(PlannerInfo *root, Re
*** 307,318 ****
&bitjoinpaths);
}
/*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
! indexpaths = generate_bitmap_or_paths(root, rel,
! rel->baserestrictinfo, NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
--- 308,328 ----
&bitjoinpaths);
}
+
+ /*
+ * Bitmap paths are currently not aggregated: AggPath does not accept the
+ * TID bitmap as input, and even if it did, it'd seem weird to aggregate
+ * the individual paths and then AND them together.
+ */
+ if (rel->agg_info != NULL)
+ return;
+
/*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
! indexpaths = generate_bitmap_or_paths(root, rel, rel->baserestrictinfo,
! NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
*************** bms_equal_any(Relids relids, List *relid
*** 717,723 ****
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
--- 727,732 ----
*************** get_index_paths(PlannerInfo *root, RelOp
*** 748,753 ****
--- 757,766 ----
* clauses only if the index AM supports them natively, and skip any such
* clauses for index columns after the first (so that we produce ordered
* paths if possible).
+ *
+ * These paths are good candidates for AGG_SORTED, so pass the output
+ * lists for this strategy. AGG_HASHED should be applied to paths with no
+ * pathkeys.
*/
indexpaths = build_index_paths(root, rel,
index, clauses,
*************** get_index_paths(PlannerInfo *root, RelOp
*** 760,765 ****
--- 773,781 ----
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
* that supports them, then try again including those clauses. This will
* produce paths with more selectivity but no ordering.
+ *
+ * As for the grouping paths, only AGG_HASHED is considered due to the
+ * missing ordering.
*/
if (skip_lower_saop)
{
*************** get_index_paths(PlannerInfo *root, RelOp
*** 801,806 ****
--- 817,825 ----
* If there were ScalarArrayOpExpr clauses that the index can't handle
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
+ *
+ * As for grouping, only AGG_HASHED is possible here. Again, because
+ * there's no ordering.
*/
if (skip_nonnative_saop)
{
*************** get_index_paths(PlannerInfo *root, RelOp
*** 847,859 ****
* NULL, we do not ignore non-first ScalarArrayOpExpr clauses, but they will
* result in considering the scan's output to be unordered.
*
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'clauses' is the collection of indexable clauses (RestrictInfo nodes)
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
! * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
--- 866,883 ----
* NULL, we do not ignore non-first ScalarArrayOpExpr clauses, but they will
* result in considering the scan's output to be unordered.
*
+ * If 'agg_info' is passed, 'agg_sorted' and / or 'agg_hashed' must be passed
+ * too. In that case AGG_SORTED and / or AGG_HASHED aggregation is applied to
+ * the index path (as long as the index path is appropriate) and the resulting
+ * grouped path is stored here.
+ *
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'clauses' is the collection of indexable clauses (RestrictInfo nodes)
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
! * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 902,913 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool grouped;
+ bool can_agg_sorted,
+ can_agg_hashed;
+ AggPath *agg_path;
+
+ grouped = rel->agg_info != NULL;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 1031,1037 ****
--- 1061,1072 ----
* in the current clauses, OR the index ordering is potentially useful for
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
+ *
+ * This is where grouped path start to be considered.
*/
+ can_agg_sorted = true;
+ can_agg_hashed = true;
+
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1083,1145 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /*
! * Try to create the grouped paths if caller is interested in
! * them.
! */
! if (useful_pathkeys != NIL)
! {
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! ipath->path.rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! {
! /*
! * If ipath could not be used as a source for AGG_SORTED
! * partial aggregation, it probably does not have the
! * appropriate pathkeys. Avoid trying to apply AGG_SORTED
! * to the next index paths because those will have the
! * same pathkeys.
! */
! can_agg_sorted = false;
! }
! }
! else
! can_agg_sorted = false;
!
! /*
! * Hashed aggregation should not be parameterized: the cost of
! * repeated creation of the hashtable (for different parameter
! * values) is probably not worth.
! */
! if (outer_relids != NULL)
! {
! agg_path = create_partial_agg_hashed_path(root,
! (Path *) ipath,
! ipath->path.rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! {
! /*
! * If ipath could not be used as a source for AGG_HASHED,
! * we should not expect any other path of the same index
! * to succeed. Avoid wasting the effort next time.
! */
! can_agg_hashed = false;
! }
! }
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1168,1213 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath);
! else
! {
! if (useful_pathkeys != NIL && can_agg_sorted)
! {
! /*
! * No need to check the pathkeys again.
! */
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! ipath->path.rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path);
! }
!
! if (can_agg_hashed && outer_relids == NULL)
! {
! agg_path = create_partial_agg_hashed_path(root,
! (Path *) ipath,
! ipath->path.rows);
!
! /*
! * If create_agg_hashed_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path);
! }
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1235,1266 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /*
! * As the input set ordering does not matter to AGG_HASHED,
! * only AGG_SORTED makes sense here. (The AGG_HASHED path we'd
! * create here should already exist.)
! *
! * The existing value of can_agg_sorted is not up-to-date for
! * the new pathkeys.
! */
! can_agg_sorted = true;
!
! /* pathkeys are new, so check them. */
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! ipath->path.rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1284,1309 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath);
! else
! {
! if (can_agg_sorted)
! {
! /*
! * The non-partial path above should have been
! * created, so no need to check pathkeys.
! */
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! ipath->path.rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path);
! }
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1164,1169 ****
--- 1338,1344 ----
* 'rel' is the relation for which we want to generate index paths
* 'clauses' is the current list of clauses (RestrictInfo nodes)
* 'other_clauses' is the list of additional upper-level clauses
+ * 'agg_info' indicates that grouped paths should be added to 'agg_hashed'.
*/
static List *
build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1237,1243 ****
match_clauses_to_index(index, other_clauses, &clauseset);
/*
! * Construct paths if possible.
*/
indexpaths = build_index_paths(root, rel,
index, &clauseset,
--- 1412,1419 ----
match_clauses_to_index(index, other_clauses, &clauseset);
/*
! * Construct paths if possible. Forbid partial aggregation even if the
! * relation is grouped --- it'll be applied to the bitmap heap path.
*/
indexpaths = build_index_paths(root, rel,
index, &clauseset,
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 688f440..0cc6268
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** static void try_partial_mergejoin_path(P
*** 48,76 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
--- 48,82 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool do_aggregate);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool do_aggregate);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool do_aggregate);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
*************** static void generate_mergejoin_paths(Pla
*** 87,93 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 93,100 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool do_aggregate);
/*
*************** static void generate_mergejoin_paths(Pla
*** 112,117 ****
--- 119,127 ----
* however. Path cost estimation code may need to recognize that it's
* dealing with such a case --- the combination of nominal jointype INNER
* with sjinfo->jointype == JOIN_SEMI indicates that.
+ *
+ * agg_info is passed iff partial aggregation should be applied to the join
+ * path.
*/
void
add_paths_to_joinrel(PlannerInfo *root,
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 120,126 ****
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
! List *restrictlist)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
--- 130,137 ----
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
! List *restrictlist,
! bool do_aggregate)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 265,271 ****
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 2. Consider paths where the outer relation need not be explicitly
--- 276,282 ----
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, do_aggregate);
/*
* 2. Consider paths where the outer relation need not be explicitly
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 276,282 ****
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
#ifdef NOT_USED
--- 287,293 ----
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, do_aggregate);
#ifdef NOT_USED
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 303,309 ****
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 314,320 ----
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, do_aggregate);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 364,370 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
--- 375,382 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
*************** try_nestloop_path(PlannerInfo *root,
*** 374,379 ****
--- 386,392 ----
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ bool success = false;
/*
* Paths are parameterized by top-level parents, so run parameterization
*************** try_nestloop_path(PlannerInfo *root,
*** 420,429 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
--- 433,463 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if ((!do_aggregate &&
! add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer)) ||
! do_aggregate)
{
+ Path *path;
+ PathTarget *target;
+
+ /*
+ * If the join output is subject to partial aggregation, the path must
+ * have the appropriate target.
+ */
+ if (!do_aggregate)
+ target = joinrel->reltarget;
+ else
+ {
+ Assert(joinrel->agg_info != NULL);
+ target = joinrel->agg_info->input;
+ }
+
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
*************** try_nestloop_path(PlannerInfo *root,
*** 445,465 ****
}
}
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer));
}
! else
{
! /* Waste no memory when we reject a path here */
bms_free(required_outer);
}
}
--- 479,534 ----
}
}
! path = (Path *) create_nestloop_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer);
! if (!do_aggregate)
! {
! add_path(joinrel, path);
! success = true;
! }
! else
! {
! /*
! * Try both AGG_HASHED and AGG_SORTED partial aggregation.
! *
! * AGG_HASHED should not be parameterized because we don't want to
! * create the hashtable again for each set of parameters.
! */
! if (required_outer == NULL)
! success = create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_HASHED);
!
! /*
! * Don't try AGG_SORTED if create_grouped_path() would reject it
! * anyway.
! */
! if (pathkeys != NIL)
! success = success ||
! create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_SORTED);
! }
}
!
! if (!success)
{
! /* Waste no memory when we reject path(s) here */
bms_free(required_outer);
}
}
*************** try_partial_nestloop_path(PlannerInfo *r
*** 476,484 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 545,556 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *path;
+ PathTarget *target;
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 513,519 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/*
--- 585,597 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
!
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if (!do_aggregate &&
! !add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/*
*************** try_partial_nestloop_path(PlannerInfo *r
*** 532,549 ****
return;
}
/* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL));
}
/*
--- 610,663 ----
return;
}
+ /*
+ * If the join output is subject to partial aggregation, the path must
+ * have the appropriate target.
+ */
+ if (!do_aggregate)
+ target = joinrel->reltarget;
+ else
+ {
+ Assert(joinrel->agg_info != NULL);
+ target = joinrel->agg_info->input;
+ }
+
/* Might be good enough to be worth trying, so let's try it. */
! path = (Path *) create_nestloop_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL);
!
! if (!do_aggregate)
! add_partial_path(joinrel, path);
! else
! {
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_HASHED);
!
! /*
! * Don't try AGG_SORTED if create_grouped_path() would reject it
! * anyway.
! */
! if (pathkeys != NIL)
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_SORTED);
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 562,571 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
if (is_partial)
{
--- 676,687 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool success = false;
if (is_partial)
{
*************** try_mergejoin_path(PlannerInfo *root,
*** 578,584 ****
outersortkeys,
innersortkeys,
jointype,
! extra);
return;
}
--- 694,701 ----
outersortkeys,
innersortkeys,
jointype,
! extra,
! do_aggregate);
return;
}
*************** try_mergejoin_path(PlannerInfo *root,
*** 615,640 ****
outersortkeys, innersortkeys,
extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
! else
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
--- 732,799 ----
outersortkeys, innersortkeys,
extra);
! if ((!do_aggregate &&
! add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer)) ||
! do_aggregate)
{
! Path *path;
! PathTarget *target;
!
! /*
! * If the join output is subject to partial aggregation, the path must
! * have the appropriate target.
! */
! if (!do_aggregate)
! target = joinrel->reltarget;
! else
! {
! Assert(joinrel->agg_info != NULL);
! target = joinrel->agg_info->input;
! }
!
! path = (Path *) create_mergejoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys);
! if (!do_aggregate)
! {
! add_path(joinrel, path);
! success = true;
! }
! else
! {
! if (required_outer == NULL)
! success = create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_HASHED);
!
! if (pathkeys != NIL)
! success = success ||
! create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_SORTED);
! }
}
!
! if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
*************** try_partial_mergejoin_path(PlannerInfo *
*** 656,664 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 815,826 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *path;
+ PathTarget *target;
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 691,714 ****
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
/*
--- 853,909 ----
outersortkeys, innersortkeys,
extra);
! if (!do_aggregate &&
! !add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
+ /*
+ * If the join output is subject to partial aggregation, the path must
+ * have the appropriate target.
+ */
+ if (!do_aggregate)
+ target = joinrel->reltarget;
+ else
+ {
+ Assert(joinrel->agg_info != NULL);
+ target = joinrel->agg_info->input;
+ }
+
/* Might be good enough to be worth trying, so let's try it. */
! path = (Path *) create_mergejoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys);
!
! if (!do_aggregate)
! add_partial_path(joinrel, path);
! else
! {
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_HASHED);
!
! if (pathkeys != NIL)
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_SORTED);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 723,732 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
--- 918,930 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *path = NULL;
+ bool success = false;
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_hashjoin_path(PlannerInfo *root,
*** 743,772 ****
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! false, /* parallel_hash */
! extra->restrictlist,
! required_outer,
! hashclauses));
}
! else
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
--- 941,1020 ----
}
/*
+ * Parameterized execution of grouped path would mean repeated hashing of
+ * the output of the hashjoin output, so forget about AGG_HASHED if there
+ * are any parameters. And AGG_SORTED makes no sense because the hash join
+ * output is not sorted.
+ */
+ if (required_outer && joinrel->agg_info)
+ return;
+
+ /*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if ((!do_aggregate &&
! add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer)) ||
! do_aggregate)
{
! PathTarget *target;
!
! /*
! * If the join output is subject to partial aggregation, the path must
! * have the appropriate target.
! */
! if (!do_aggregate)
! target = joinrel->reltarget;
! else
! {
! Assert(joinrel->agg_info != NULL);
! target = joinrel->agg_info->input;
! }
!
! path = (Path *) create_hashjoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! false, /* parallel_hash */
! extra->restrictlist,
! required_outer,
! hashclauses);
!
! if (!do_aggregate)
! {
! add_path(joinrel, path);
! success = true;
! }
! else
! {
!
! /*
! * As the hashjoin path is not sorted, only try AGG_HASHED.
! */
! if (create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_HASHED))
! success = true;
! }
}
!
! if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 790,798 ****
List *hashclauses,
JoinType jointype,
JoinPathExtraData *extra,
! bool parallel_hash)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1038,1049 ----
List *hashclauses,
JoinType jointype,
JoinPathExtraData *extra,
! bool parallel_hash,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *path;
+ PathTarget *target;
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 815,836 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, true);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! parallel_hash,
! extra->restrictlist,
! NULL,
! hashclauses));
}
/*
--- 1066,1118 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, true);
!
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if (!do_aggregate &&
! !add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /*
! * If the join output is subject to partial aggregation, the path must
! * have the appropriate target.
! */
! if (!do_aggregate)
! target = joinrel->reltarget;
! else
! {
! Assert(joinrel->agg_info != NULL);
! target = joinrel->agg_info->input;
! }
!
! path = (Path *) create_hashjoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! parallel_hash,
! extra->restrictlist,
! NULL,
! hashclauses);
! if (!do_aggregate)
! add_partial_path(joinrel, path);
! else
! {
! /*
! * Only AGG_HASHED is useful, see comments in try_hashjoin_path().
! */
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_HASHED);
! }
}
/*
*************** clause_sides_match_join(RestrictInfo *ri
*** 874,879 ****
--- 1156,1162 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'agg_info' tells if/how to apply partial aggregation to the output.
*/
static void
sort_inner_and_outer(PlannerInfo *root,
*************** sort_inner_and_outer(PlannerInfo *root,
*** 881,887 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
Path *outer_path;
--- 1164,1171 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
Path *outer_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1043,1049 ****
innerkeys,
jointype,
extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
--- 1327,1334 ----
innerkeys,
jointype,
extra,
! false,
! do_aggregate);
/*
* If we have partial outer and parallel safe inner path then try
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1059,1065 ****
outerkeys,
innerkeys,
jointype,
! extra);
}
}
--- 1344,1351 ----
outerkeys,
innerkeys,
jointype,
! extra,
! do_aggregate);
}
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1087,1093 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1373,1380 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1148,1154 ****
innersortkeys,
jointype,
extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1435,1442 ----
innersortkeys,
jointype,
extra,
! is_partial,
! do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1245,1251 ****
NIL,
jointype,
extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
--- 1533,1540 ----
NIL,
jointype,
extra,
! is_partial,
! do_aggregate);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1289,1295 ****
NIL,
jointype,
extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1578,1585 ----
NIL,
jointype,
extra,
! is_partial,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** match_unsorted_outer(PlannerInfo *root,
*** 1331,1337 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
--- 1621,1628 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1454,1460 ****
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra);
}
else if (nestjoinOK)
{
--- 1745,1752 ----
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra,
! do_aggregate);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1476,1482 ****
innerpath,
merge_pathkeys,
jointype,
! extra);
}
/* Also consider materialized form of the cheapest inner path */
--- 1768,1775 ----
innerpath,
merge_pathkeys,
jointype,
! extra,
! do_aggregate);
}
/* Also consider materialized form of the cheapest inner path */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1487,1493 ****
matpath,
merge_pathkeys,
jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 1780,1787 ----
matpath,
merge_pathkeys,
jointype,
! extra,
! do_aggregate);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1502,1508 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 1796,1802 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, do_aggregate);
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1523,1529 ****
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 1817,1823 ----
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, do_aggregate);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1543,1549 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
--- 1837,1844 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total,
! do_aggregate);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1566,1572 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
--- 1861,1868 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool do_aggregate)
{
ListCell *lc1;
*************** consider_parallel_mergejoin(PlannerInfo
*** 1584,1590 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 1880,1886 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
extra, false, inner_cheapest_total,
! merge_pathkeys, true, do_aggregate);
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1605,1611 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
--- 1901,1908 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
ListCell *lc1;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1655,1661 ****
}
try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
--- 1952,1959 ----
}
try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! do_aggregate);
}
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1670,1675 ****
--- 1968,1974 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'agg_info' tells if/how to apply partial aggregation to the output.
*/
static void
hash_inner_and_outer(PlannerInfo *root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1677,1683 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
--- 1976,1983 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1752,1758 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
--- 2052,2059 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1768,1774 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
--- 2069,2076 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1777,1783 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
}
else
{
--- 2079,2086 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
}
else
{
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1798,1804 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
foreach(lc1, outerrel->cheapest_parameterized_paths)
{
--- 2101,2108 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
foreach(lc1, outerrel->cheapest_parameterized_paths)
{
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1832,1838 ****
innerpath,
hashclauses,
jointype,
! extra);
}
}
}
--- 2136,2143 ----
innerpath,
hashclauses,
jointype,
! extra,
! do_aggregate);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1875,1881 ****
cheapest_partial_outer,
cheapest_partial_inner,
hashclauses, jointype, extra,
! true /* parallel_hash */ );
}
/*
--- 2180,2187 ----
cheapest_partial_outer,
cheapest_partial_inner,
hashclauses, jointype, extra,
! true /* parallel_hash */ ,
! do_aggregate);
}
/*
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1896,1902 ****
cheapest_partial_outer,
cheapest_safe_inner,
hashclauses, jointype, extra,
! false /* parallel_hash */ );
}
}
}
--- 2202,2209 ----
cheapest_partial_outer,
cheapest_safe_inner,
hashclauses, jointype, extra,
! false /* parallel_hash */ ,
! do_aggregate);
}
}
}
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 3f1c1b3..59273c4
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
***************
*** 17,28 ****
--- 17,31 ----
#include "miscadmin.h"
#include "catalog/partition.h"
#include "optimizer/clauses.h"
+ #include "optimizer/cost.h"
#include "optimizer/joininfo.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/prep.h"
+ #include "optimizer/tlist.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+ #include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
*************** static void make_rels_by_clause_joins(Pl
*** 31,36 ****
--- 34,43 ----
static void make_rels_by_clauseless_joins(PlannerInfo *root,
RelOptInfo *old_rel,
ListCell *other_rels);
+ static void set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool is_dummy_rel(RelOptInfo *rel);
*************** static bool restriction_is_constant_fals
*** 38,48 ****
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist);
! static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
--- 45,57 ----
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist,
! bool do_aggregate);
! static void try_partition_wise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist,
! bool do_aggregate);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
*************** join_search_one_level(PlannerInfo *root,
*** 69,74 ****
--- 78,92 ----
Assert(joinrels[level] == NIL);
+ /*
+ * Subroutines will eventually call make_join_rel() with both input rels
+ * from the joinrels list, i.e. both non-grouped. In addition to joining
+ * these, make_join_rel() will try to combine each of these with grouped
+ * rel and also apply partial aggregation. All the grouped joins will be
+ * added to root->join_grouped_rel_level[level].
+ */
+ Assert(root->join_grouped_rel_level[level] == NIL);
+
/* Set join_cur_level so that new joinrels are added to proper list */
root->join_cur_level = level;
*************** make_rels_by_clauseless_joins(PlannerInf
*** 321,326 ****
--- 339,392 ----
}
}
+ /*
+ * Set joinrel's reltarget according to agg_info and estimate the number of
+ * rows.
+ */
+ static void
+ set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info)
+ {
+ Assert(agg_info != NULL);
+
+ /*
+ * build_join_rel() / build_child_join_rel() does not create the target
+ * for grouped relation.
+ */
+ Assert(joinrel->reltarget == NULL);
+ Assert(joinrel->agg_info == NULL);
+
+ /*
+ * The output will actually be grouped, i.e. partially aggregated. No
+ * additional processing needed.
+ */
+ joinrel->reltarget = copy_pathtarget(agg_info->target);
+
+ /*
+ * The rest of agg_info will be needed at aggregation time.
+ */
+ joinrel->agg_info = agg_info;
+
+ /*
+ * Now that we have the target, compute the estimates.
+ */
+ set_joinrel_size_estimates(root, joinrel, rel1, rel2, sjinfo,
+ restrictlist);
+
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is partially aggregated, or the partial aggregation has been
+ * already applied to one of the input rels. Is this worth extra effort,
+ * e.g. maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, joinrel->agg_info->group_exprs,
+ joinrel->rows, NULL);
+ }
/*
* join_is_legal
*************** join_is_legal(PlannerInfo *root, RelOptI
*** 659,670 ****
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
! * NB: will return NULL if attempted join is not valid. This can happen
! * when working with outer joins, or with IN or EXISTS clauses that have been
! * turned into joins.
*/
! RelOptInfo *
! make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
--- 725,745 ----
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
! * 'agg_info' contains the reltarget of grouped relation and everything we
! * need to perform partial aggregation. If NULL, then the join relation
! * should not be grouped.
! *
! * 'do_aggregate' tells that two non-grouped rels should be grouped and
! * partial aggregation should be applied to all their paths.
! *
! * NB: will return NULL if attempted join is not valid. This can happen when
! * working with outer joins, or with IN or EXISTS clauses that have been
! * turned into joins. NULL is also returned if caller is interested in a
! * grouped relation but there's no useful grouped input relation.
*/
! static RelOptInfo *
! make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
! RelAggInfo *agg_info, bool do_aggregate)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 672,681 ****
--- 747,760 ----
SpecialJoinInfo sjinfo_data;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
+ /* do_aggregate implies the output to be grouped. */
+ Assert(!do_aggregate || grouped);
+
/* Construct Relids set that identifies the joinrel. */
joinrelids = bms_union(rel1->relids, rel2->relids);
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 725,731 ****
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
! &restrictlist);
/*
* If we've already proven this join is empty, we needn't consider any
--- 804,829 ----
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
! &restrictlist, grouped);
!
! /*
! * Make sure the joinrel has reltarget initialized. Caller should supply
! * the target for group relation, so build_join_rel() should have omitted
! * its creation.
! *
! * The target can already be there if we were already called with
! * grouped=true.
! */
! if (grouped && joinrel->reltarget == NULL)
! {
! set_grouped_joinrel_target(root, joinrel, rel1, rel2, sjinfo,
! restrictlist, agg_info);
!
! if (rel1->consider_parallel && rel2->consider_parallel &&
! is_parallel_safe(root, (Node *) restrictlist) &&
! is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
! joinrel->consider_parallel = true;
! }
/*
* If we've already proven this join is empty, we needn't consider any
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 739,745 ****
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
! restrictlist);
bms_free(joinrelids);
--- 837,843 ----
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
! restrictlist, do_aggregate);
bms_free(joinrelids);
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 747,752 ****
--- 845,988 ----
}
/*
+ * Front-end to make_join_rel_common(). Generates plain (non-grouped) join and
+ * then uses all the possible strategies to generate the grouped one.
+ */
+ JoinSearchResult *
+ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+ {
+ Relids joinrelids;
+ RelAggInfo *agg_info;
+ RelOptInfo *rel1_grouped,
+ *rel2_grouped,
+ *joinrel;
+ double nrows_plain;
+ JoinSearchResult *result;
+ bool rel1_grouped_useful,
+ rel2_grouped_useful;
+
+ result = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
+
+ /* 1) form the plain join. */
+ result->plain = make_join_rel_common(root, rel1, rel2, NULL, false);
+
+ if (result->plain == NULL)
+ return result;
+
+ nrows_plain = result->plain->rows;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return result;
+
+ /*
+ * If the same joinrel was already formed, just with the base rels divided
+ * between rel1 and rel2 in a different way, we might already have the
+ * matching agg_info.
+ */
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_join_rel(root, joinrelids, true);
+ if (joinrel != NULL && joinrel->agg_info != NULL)
+ agg_info = joinrel->agg_info;
+ else
+ {
+ double nrows;
+
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, result->plain);
+
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ if (agg_info != NULL)
+ {
+ nrows = estimate_num_groups(root,
+ agg_info->group_exprs,
+ nrows_plain,
+ NULL);
+ agg_info->rows = clamp_row_est(nrows);
+ }
+ }
+
+ /*
+ * Cannot we build grouped join?
+ */
+ if (agg_info == NULL)
+ return result;
+
+ /*
+ * 2) join two plain rels and aggregate the join paths.
+ */
+ result->grouped = make_join_rel_common(root, rel1, rel2, agg_info, true);
+
+ /*
+ * Retrieve the grouped relations.
+ *
+ * Dummy rel indicates join relation able to generate grouped paths as
+ * such (i.e. it has valid agg_info), but for which the path actually
+ * could not be created (e.g. only AGG_HASHED strategy was possible but
+ * work_mem was not sufficient for hash table).
+ */
+ rel1_grouped = IS_JOIN_REL(rel1) ?
+ find_join_rel(root, rel1->relids, true) :
+ find_grouped_base_rel(root, rel1->relid);
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+
+ rel2_grouped = IS_JOIN_REL(rel2) ?
+ find_join_rel(root, rel2->relids, true) :
+ find_grouped_base_rel(root, rel2->relid);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /*
+ * Nothing else to do?
+ */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return result;
+
+ /*
+ * 3) combine plain and grouped relation.
+ *
+ * At maximum one input rel can be grouped. If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped_useful && rel2_grouped_useful)
+ return result;
+
+ /*
+ * 4) join grouped relation to plain one. The same target we used for
+ * aggregation above should be applicable to either case here.
+ */
+ if (rel1_grouped_useful)
+ joinrel = make_join_rel_common(root, rel1_grouped, rel2, agg_info,
+ false);
+ else if (rel2_grouped_useful)
+ joinrel = make_join_rel_common(root, rel1, rel2_grouped, agg_info,
+ false);
+
+ /*
+ * We expect make_join_rel_common() to return the same joinrel it did in
+ * the 2) case.
+ */
+ Assert(joinrel && result->grouped);
+
+ return result;
+ }
+
+ /*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 756,762 ****
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist)
{
/*
* Consider paths using each rel as both outer and inner. Depending on
--- 992,999 ----
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist,
! bool do_aggregate)
{
/*
* Consider paths using each rel as both outer and inner. Depending on
*************** populate_joinrel_with_paths(PlannerInfo
*** 787,796 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
! restrictlist);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
--- 1024,1033 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
! restrictlist, do_aggregate);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
*************** populate_joinrel_with_paths(PlannerInfo
*** 804,813 ****
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
! restrictlist);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
--- 1041,1050 ----
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
! restrictlist, do_aggregate);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
*************** populate_joinrel_with_paths(PlannerInfo
*** 818,827 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
! restrictlist);
/*
* If there are join quals that aren't mergeable or hashable, we
--- 1055,1064 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
! restrictlist, do_aggregate);
/*
* If there are join quals that aren't mergeable or hashable, we
*************** populate_joinrel_with_paths(PlannerInfo
*** 854,860 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
! restrictlist);
}
/*
--- 1091,1097 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
! restrictlist, do_aggregate);
}
/*
*************** populate_joinrel_with_paths(PlannerInfo
*** 877,886 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
! restrictlist);
}
break;
case JOIN_ANTI:
--- 1114,1123 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
! restrictlist, do_aggregate);
}
break;
case JOIN_ANTI:
*************** populate_joinrel_with_paths(PlannerInfo
*** 895,901 ****
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
! restrictlist);
break;
default:
/* other values not expected here */
--- 1132,1138 ----
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
! restrictlist, do_aggregate);
break;
default:
/* other values not expected here */
*************** populate_joinrel_with_paths(PlannerInfo
*** 903,910 ****
break;
}
! /* Apply partitionwise join technique, if possible. */
! try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
}
--- 1140,1148 ----
break;
}
! /* Apply partition-wise join technique, if possible. */
! try_partition_wise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist,
! do_aggregate);
}
*************** restriction_is_constant_false(List *rest
*** 1304,1315 ****
* obtained by translating the respective parent join structures.
*/
static void
! try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
RelOptInfo *joinrel, SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist)
{
int nparts;
int cnt_parts;
/* Guard against stack overflow due to overly deep partition hierarchy. */
check_stack_depth();
--- 1542,1554 ----
* obtained by translating the respective parent join structures.
*/
static void
! try_partition_wise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
RelOptInfo *joinrel, SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist, bool do_aggregate)
{
int nparts;
int cnt_parts;
+ bool grouped = joinrel->agg_info != NULL;
/* Guard against stack overflow due to overly deep partition hierarchy. */
check_stack_depth();
*************** try_partitionwise_join(PlannerInfo *root
*** 1334,1341 ****
joinrel->part_scheme == rel2->part_scheme);
/*
! * Since we allow partitionwise join only when the partition bounds of
! * the joining relations exactly match, the partition bounds of the join
* should match those of the joining relations.
*/
Assert(partition_bounds_equal(joinrel->part_scheme->partnatts,
--- 1573,1580 ----
joinrel->part_scheme == rel2->part_scheme);
/*
! * Since we allow partitionwise join only when the partition bounds of the
! * joining relations exactly match, the partition bounds of the join
* should match those of the joining relations.
*/
Assert(partition_bounds_equal(joinrel->part_scheme->partnatts,
*************** try_partitionwise_join(PlannerInfo *root
*** 1386,1392 ****
(List *) adjust_appendrel_attrs(root,
(Node *) parent_restrictlist,
nappinfos, appinfos);
- pfree(appinfos);
child_joinrel = joinrel->part_rels[cnt_parts];
if (!child_joinrel)
--- 1625,1630 ----
*************** try_partitionwise_join(PlannerInfo *root
*** 1394,1408 ****
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
child_sjinfo,
! child_sjinfo->jointype);
joinrel->part_rels[cnt_parts] = child_joinrel;
}
Assert(bms_equal(child_joinrel->relids, child_joinrelids));
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
! child_restrictlist);
}
}
--- 1632,1678 ----
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
child_sjinfo,
! child_sjinfo->jointype,
! grouped);
!
! if (grouped)
! {
! RelAggInfo *child_agg_info;
!
! /*
! * Make sure the child_joinrel has reltarget initialized.
! *
! * Although build_child_join_rel() creates reltarget for each
! * child join from scratch as opposed to translating the
! * parent reltarget (XXX set_append_rel_size() uses the
! * translation --- is this inconsistency justified?), we just
! * translate the parent reltarget here. Per-child call of
! * create_rel_agg_info() would introduce too much duplicate
! * work because it needs the *parent* target as a source and
! * that one is identical for all the child joins
! */
! child_agg_info = translate_rel_agg_info(root,
! joinrel->agg_info,
! appinfos, nappinfos);
!
! /*
! * Make sure the child joinrel has reltarget initialized.
! */
! set_grouped_joinrel_target(root, child_joinrel, rel1, rel2,
! child_sjinfo, child_restrictlist,
! child_agg_info);
! }
!
joinrel->part_rels[cnt_parts] = child_joinrel;
}
+ pfree(appinfos);
Assert(bms_equal(child_joinrel->relids, child_joinrelids));
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
! child_restrictlist,
! do_aggregate);
}
}
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index 3bb5b8d..bb0f814
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** TidQualFromBaseRestrictinfo(RelOptInfo *
*** 250,259 ****
* Candidate paths are added to the rel's pathlist (using add_path).
*/
void
! create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
{
Relids required_outer;
List *tidquals;
/*
* We don't support pushing join clauses into the quals of a tidscan, but
--- 250,260 ----
* Candidate paths are added to the rel's pathlist (using add_path).
*/
void
! create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Relids required_outer;
List *tidquals;
+ Path *tidpath;
/*
* We don't support pushing join clauses into the quals of a tidscan, but
*************** create_tidscan_paths(PlannerInfo *root,
*** 263,270 ****
required_outer = rel->lateral_relids;
tidquals = TidQualFromBaseRestrictinfo(rel);
! if (tidquals)
! add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 264,283 ----
required_outer = rel->lateral_relids;
tidquals = TidQualFromBaseRestrictinfo(rel);
+ if (!tidquals)
+ return;
! tidpath = (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer);
!
! if (!grouped)
! add_path(rel, tidpath);
! else if (required_outer == NULL)
! {
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, rel, tidpath, false, false, AGG_HASHED);
! }
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index 9ae1bf3..bc405bf
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** use_physical_tlist(PlannerInfo *root, Pa
*** 815,820 ****
--- 815,826 ----
return false;
/*
+ * Grouped relation's target list contains GroupedVars.
+ */
+ if (rel->agg_info != NULL)
+ return false;
+
+ /*
* If a bitmap scan's tlist is empty, keep it as-is. This may allow the
* executor to skip heap page fetches, and in any case, the benefit of
* using a physical tlist instead would be minimal.
*************** create_projection_plan(PlannerInfo *root
*** 1593,1600 ****
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
--- 1599,1607 ----
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (!best_path->force_result &&
! (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist)))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
*************** find_ec_member_for_tle(EquivalenceClass
*** 5827,5832 ****
--- 5834,5854 ----
while (tlexpr && IsA(tlexpr, RelabelType))
tlexpr = ((RelabelType *) tlexpr)->arg;
+ /*
+ * GroupedVar can contain either non-Var grouping expression or aggregate.
+ * The grouping expression might be useful for sorting, however aggregates
+ * shouldn't currently appear among pathkeys.
+ */
+ if (IsA(tlexpr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, tlexpr);
+
+ if (!IsA(gvar->gvexpr, Aggref))
+ tlexpr = gvar->gvexpr;
+ else
+ return NULL;
+ }
+
foreach(lc, ec->ec_members)
{
EquivalenceMember *em = (EquivalenceMember *) lfirst(lc);
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index a436b53..5d66785
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,19 ****
--- 14,20 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "catalog/pg_class.h"
#include "nodes/nodeFuncs.h"
***************
*** 27,32 ****
--- 28,34 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 46,51 ****
--- 48,56 ----
} PostponedQual;
+ static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+ static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
+ static RelOptInfo *copy_simple_rel(PlannerInfo *root, RelOptInfo *rel);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** static void check_hashjoinable(RestrictI
*** 96,105 ****
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
! * every non-join RTE that is used in the query. Therefore, this routine
! * is the only place that should call build_simple_rel with reloptkind
! * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
! * "other rel" RelOptInfos for the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
--- 101,109 ----
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
! * every non-grouped non-join RTE that is used in the query. (Note:
! * build_simple_rel recurses internally to build "other rel" RelOptInfos for
! * the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
*************** add_vars_to_targetlist(PlannerInfo *root
*** 241,246 ****
--- 245,701 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression and setup RelOptInfo for each base or
+ * 'other' relation that can product grouped paths.
+ *
+ * Note that targets of the 'other' relations are not set here ---
+ * set_append_rel_size() will create them by translating the targets of the
+ * base rel.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouped_base_rels_to_query(PlannerInfo *root)
+ {
+ int i;
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /*
+ * TODO Consider if this is a real limitation.
+ */
+ if (root->parse->hasWindowFuncs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /*
+ * Are all the aggregates AGGSPLIT_SIMPLE?
+ */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+ RangeTblEntry *rte;
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* NULL should mean a join relation. */
+ if (rel == NULL)
+ continue;
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return;
+
+ /*
+ * Grouped "other member rels" should not be created until we know
+ * whether the parent can be grouped.
+ */
+ if (rel->reloptkind != RELOPT_BASEREL)
+ continue;
+
+ /*
+ * Retrieve the information we need for aggregation of the rel
+ * contents.
+ */
+ agg_info = create_rel_agg_info(root, rel);
+ if (agg_info == NULL)
+ continue;
+
+ /*
+ * Create the grouped counterpart of "rel".
+ */
+ rel_grouped = copy_simple_rel(root, rel);
+
+ /*
+ * Assign it the aggregation-specific info.
+ *
+ * The aggregation paths will get their input target from agg_info, so
+ * store it too.
+ */
+ rel_grouped->reltarget = agg_info->target;
+ rel_grouped->agg_info = agg_info;
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+ static void
+ create_aggregate_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /*
+ * Aggregation push-down does not work w/o aggcombinefn. This field is
+ * not mandatory, so check if this particular aggregate can handle
+ * partial aggregation.
+ */
+ if (!OidIsValid(aggref->aggcombinefn))
+ {
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ /*
+ * The transient state is what appears in the target.
+ */
+ gvi->gv_width =
+ get_typavgwidth(exprType((Node *) gvi->agg_partial),
+ exprTypmod((Node *) gvi->agg_partial));
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+ static void
+ create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+ {
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(expr);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = pull_varnos((Node *) expr);
+
+ /*
+ * As a special case, Var can be present in the non-grouped target, so
+ * set_rel_widths() could take care of the width computation. However
+ * the Var might not be propagated high enough in the join tree, so
+ * set_rel_width() might miss it. Handle it manually.
+ */
+ if (IsA(expr, Var))
+ {
+ Var *var = castNode(Var, expr);
+ RelOptInfo *rel = root->simple_rel_array[var->varno];
+ Oid reloid = rel->relid;
+
+ if (reloid != InvalidOid && var->varattno > 0)
+ {
+ int32 width = get_attavgwidth(reloid, var->varattno);
+
+ if (width > 0)
+ {
+ gvi->gv_width += width;
+ continue;
+ }
+ }
+ }
+
+ /*
+ * In general we should not expect any statistics to exist for an
+ * expression just because it's a grouping expression. So use the type
+ * information to get the width estimate.
+ */
+ gvi->gv_width = get_typavgwidth(exprType((Node *) gvi->gvexpr),
+ exprTypmod((Node *) gvi->gvexpr));
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ /*
+ * Take a flat copy of already initialized RelOptInfo and process child rels
+ * recursively.
+ *
+ * Flat copy ensures that we do not miss any information that the non-grouped
+ * rel already contains. XXX Do we need to copy any Node field?
+ */
+ static RelOptInfo *
+ copy_simple_rel(PlannerInfo *root, RelOptInfo *rel)
+ {
+ Index relid = rel->relid;
+ RangeTblEntry *rte;
+ ListCell *l;
+ List *indexlist = NIL;
+ RelOptInfo *result;
+
+ result = makeNode(RelOptInfo);
+ memcpy(result, rel, sizeof(RelOptInfo));
+ root->simple_grouped_rel_array[relid] = result;
+
+ /*
+ * The target for grouped paths will be initialized later.
+ */
+ result->reltarget = NULL;
+
+ /*
+ * Make sure that index paths have access to the parent rel's agg_info,
+ * which is used to indicate that the rel should produce grouped paths.
+ */
+ foreach(l, result->indexlist)
+ {
+ IndexOptInfo *src,
+ *dst;
+
+ src = lfirst_node(IndexOptInfo, l);
+ dst = makeNode(IndexOptInfo);
+ memcpy(dst, src, sizeof(IndexOptInfo));
+
+ dst->rel = result;
+ indexlist = lappend(indexlist, dst);
+ }
+ result->indexlist = indexlist;
+
+ /*
+ * This is very similar to child rel processing in build_simple_rel().
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte->inh)
+ {
+ int nparts = rel->nparts;
+ int cnt_parts = 0;
+
+ if (nparts > 0)
+ result->part_rels = (RelOptInfo **)
+ palloc(sizeof(RelOptInfo *) * nparts);
+
+ foreach(l, root->append_rel_list)
+ {
+ AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(l);
+ RelOptInfo *childrel;
+
+ /* append_rel_list contains all append rels; ignore others */
+ if (appinfo->parent_relid != relid)
+ continue;
+
+ /*
+ * The non-grouped child rel must already exist.
+ */
+ childrel = root->simple_rel_array[appinfo->child_relid];
+ Assert(childrel != NULL);
+
+ /*
+ * Create the copy.
+ */
+ childrel = copy_simple_rel(root, childrel);
+
+ /* Nothing more to do for an unpartitioned table. */
+ if (!rel->part_scheme)
+ continue;
+
+ Assert(cnt_parts < nparts);
+ result->part_rels[cnt_parts] = childrel;
+ cnt_parts++;
+ }
+
+ /* We should have seen all the child partitions. */
+ Assert(cnt_parts == nparts);
+ }
+
+ return result;
+ }
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 95cbffb..7a25f22
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** build_minmax_path(PlannerInfo *root, Min
*** 441,447 ****
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
! final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
--- 441,447 ----
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
! final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 7a34abc..ba16454
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
***************
*** 43,48 ****
--- 43,50 ----
* (this is NOT necessarily root->parse->targetList!)
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
+ * *partially_grouped receives relation that contains partial aggregate
+ * anywhere in the join tree.
*
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
***************
*** 52,62 ****
*/
RelOptInfo *
query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra)
{
Query *parse = root->parse;
List *joinlist;
! RelOptInfo *final_rel;
Index rti;
double total_pages;
--- 54,66 ----
*/
RelOptInfo *
query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra,
! RelOptInfo **partially_grouped)
{
Query *parse = root->parse;
List *joinlist;
! JoinSearchResult *final_rels;
! RelOptInfo *plain_rel;
Index rti;
double total_pages;
*************** query_planner(PlannerInfo *root, List *t
*** 66,73 ****
*/
if (parse->jointree->fromlist == NIL)
{
/* We need a dummy joinrel to describe the empty set of baserels */
! final_rel = build_empty_join_rel(root);
/*
* If query allows parallelism in general, check whether the quals are
--- 70,82 ----
*/
if (parse->jointree->fromlist == NIL)
{
+ JoinSearchResult *final_rels;
+ RelOptInfo *final_rel;
+
+ final_rels = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
+
/* We need a dummy joinrel to describe the empty set of baserels */
! final_rels->plain = final_rel = build_empty_join_rel(root);
/*
* If query allows parallelism in general, check whether the quals are
*************** query_planner(PlannerInfo *root, List *t
*** 106,111 ****
--- 115,122 ----
*/
root->join_rel_list = NIL;
root->join_rel_hash = NULL;
+ root->join_grouped_rel_list = NIL;
+ root->join_grouped_rel_hash = NULL;
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 125,131 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 226,231 ****
--- 238,253 ----
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup root->grouped_var_list
+ * and create RelOptInfo for base relations capable to do the grouping.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ add_grouped_base_rels_to_query(root);
+
+ /*
* We should now have size estimates for every actual table involved in
* the query, and we also know which if any have been deleted from the
* query by join removal; so we can compute total_table_pages.
*************** query_planner(PlannerInfo *root, List *t
*** 256,267 ****
/*
* Ready to do the primary planning.
*/
! final_rel = make_one_rel(root, joinlist);
/* Check that we got at least one usable path */
! if (!final_rel || !final_rel->cheapest_total_path ||
! final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
! return final_rel;
}
--- 278,292 ----
/*
* Ready to do the primary planning.
*/
! final_rels = make_one_rel(root, joinlist);
! plain_rel = final_rels->plain;
! if (partially_grouped != NULL)
! *partially_grouped = final_rels->grouped;
/* Check that we got at least one usable path */
! if (!plain_rel || !plain_rel->cheapest_total_path ||
! plain_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
! return plain_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index de1257d..f1a21c0
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 131,141 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd);
--- 131,139 ----
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *partially_grouped_input_rel,
PathTarget *target,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd);
*************** static void add_paths_to_partial_groupin
*** 200,205 ****
--- 198,205 ----
grouping_sets_data *gd,
bool can_sort,
bool can_hash);
+ static void gather_partial_grouping_rel_paths(PlannerInfo *root,
+ RelOptInfo *partially_grouped_rel);
static bool can_parallel_agg(PlannerInfo *root, RelOptInfo *input_rel,
RelOptInfo *grouped_rel, const AggClauseCosts *agg_costs);
*************** grouping_planner(PlannerInfo *root, bool
*** 1688,1693 ****
--- 1688,1694 ----
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *partially_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
*************** grouping_planner(PlannerInfo *root, bool
*** 1795,1801 ****
* of the query's sort clause, distinct clause, etc.
*/
current_rel = query_planner(root, tlist,
! standard_qp_callback, &qp_extra);
/*
* Convert the query's result tlist into PathTarget format.
--- 1796,1803 ----
* of the query's sort clause, distinct clause, etc.
*/
current_rel = query_planner(root, tlist,
! standard_qp_callback, &qp_extra,
! &partially_grouped);
/*
* Convert the query's result tlist into PathTarget format.
*************** grouping_planner(PlannerInfo *root, bool
*** 1983,1988 ****
--- 1985,1991 ----
{
current_rel = create_grouping_paths(root,
current_rel,
+ partially_grouped,
grouping_target,
&agg_costs,
gset_data);
*************** get_number_of_groups(PlannerInfo *root,
*** 3561,3600 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3564,3569 ----
*************** estimate_hashagg_tablesize(Path *path, c
*** 3605,3610 ****
--- 3574,3580 ----
* is, they need a Gather and then a FinalizeAggregate.
*
* input_rel: contains the source-data Paths
+ * partially_grouped_input_rel: contains Paths with aggregation pushed down.
* target: the pathtarget for the result Paths to compute
* agg_costs: cost info about all aggregates in query (in AGGSPLIT_SIMPLE mode)
* rollup_lists: list of grouping sets, or NIL if not doing grouping sets
*************** estimate_hashagg_tablesize(Path *path, c
*** 3622,3627 ****
--- 3592,3598 ----
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *partially_grouped_input_rel,
PathTarget *target,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd)
*************** create_grouping_paths(PlannerInfo *root,
*** 3824,3837 ****
get_agg_clause_costs(root, (Node *) partial_grouping_target->exprs,
AGGSPLIT_INITIAL_SERIAL,
&agg_partial_costs);
-
- /* final phase */
- get_agg_clause_costs(root, (Node *) target->exprs,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
- get_agg_clause_costs(root, parse->havingQual,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
}
add_paths_to_partial_grouping_rel(root, input_rel,
--- 3795,3800 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3840,3845 ****
--- 3803,3862 ----
gd, can_sort, can_hash);
}
+ /*
+ * Paths generated due to aggregation push-down are passed in a separate
+ * relation. Unlike "partially grouped_rel", reltarget of which contains
+ * Aggrefs, this relation's reltarget contains GroupedVars.
+ */
+ if (partially_grouped_input_rel)
+ {
+ ListCell *lc;
+
+ Assert(enable_agg_pushdown);
+
+ /*
+ * Aggregation push-down could have produced partial paths as well.
+ * These are already aggregated, so only apply Gather / GatherMerge to
+ * them.
+ */
+ gather_partial_grouping_rel_paths(root, partially_grouped_input_rel);
+
+ /*
+ * If non-partial paths were generated above, and / or the aggregate
+ * push-down resulted in non-partial paths, just add them all to
+ * partially_grouped_rel for common processing.
+ *
+ * The only difference is that the paths we add here have GroupedVars
+ * in their pathtarget, while ones already contained in the pathlist
+ * of partially_grouped_rel (i.e. the paths resulting from parallel
+ * processing) have Aggrefs. This difference will be handled later by
+ * set_upper_references().
+ */
+ foreach(lc, partially_grouped_input_rel->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
+
+ /*
+ * Prepare for the final aggregation if it's expected to take place.
+ */
+ if (partially_grouped_rel->pathlist)
+ {
+ /* Choose the best path(s) */
+ set_cheapest(partially_grouped_rel);
+
+ /* final phase */
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ }
+
/* Build final grouping paths */
add_paths_to_grouping_rel(root, input_rel, grouped_rel, target,
partially_grouped_rel, agg_costs,
*************** add_paths_to_partial_grouping_rel(Planne
*** 6334,6339 ****
--- 6351,6375 ----
* Try adding Gather or Gather Merge to partial paths to produce
* non-partial paths.
*/
+ gather_partial_grouping_rel_paths(root, partially_grouped_rel);
+ }
+
+ /*
+ * Apply Gather or GatherMerge to partial paths of partially_grouped_rel. The
+ * input paths should already be partially aggregated.
+ */
+ static void
+ gather_partial_grouping_rel_paths(PlannerInfo *root,
+ RelOptInfo *partially_grouped_rel)
+ {
+ Path *cheapest_partial_path;
+
+ /* If there are no partial paths, there's nothing to do here. */
+ if (partially_grouped_rel->partial_pathlist == NIL)
+ return;
+
+ cheapest_partial_path = linitial(partially_grouped_rel->partial_pathlist);
+
generate_gather_paths(root, partially_grouped_rel, true);
/* Get cheapest partial path from partially_grouped_rel */
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index 4617d12..f385792
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
bool has_conv_whole_rows; /* are there ConvertRowtypeExpr
* entries encapsulating a whole-row
*************** set_upper_references(PlannerInfo *root,
*** 1739,1747 ****
--- 1740,1802 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the current
+ * subplan->targetlist.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ replace_grouped_vars_with_aggrefs(root, subplan->targetlist);
+ }
+ else if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /*
+ * Partial aggregation node can have GroupedVar's on the input
+ * if those represent generic (non-Var) grouping expressions.
+ * Unlike above, the restored expressions should stay there.
+ */
+ subplan->targetlist =
+ replace_grouped_vars_with_aggrefs(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1996,2001 ****
--- 2051,2057 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
itlist->has_conv_whole_rows = false;
*************** build_tlist_index(List *tlist)
*** 2016,2021 ****
--- 2072,2079 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else if (is_converted_whole_row_reference((Node *) tle->expr))
itlist->has_conv_whole_rows = true;
else
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2299,2304 ****
--- 2357,2387 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2461,2467 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
--- 2544,2551 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
new file mode 100644
index 45d82da..d52f229
*** a/src/backend/optimizer/prep/prepjointree.c
--- b/src/backend/optimizer/prep/prepjointree.c
*************** pull_up_simple_subquery(PlannerInfo *roo
*** 911,916 ****
--- 911,917 ----
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ subroot->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index fe3b458..835623b
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 27,32 ****
--- 27,33 ----
#include "optimizer/planmain.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if create_grouped_path ends up in another module. */
#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
*************** static List *reparameterize_pathlist_by_
*** 57,63 ****
List *pathlist,
RelOptInfo *child_rel);
-
/*****************************************************************************
* MISC. PATH UTILITIES
*****************************************************************************/
--- 58,63 ----
*************** compare_path_costs_fuzzily(Path *path1,
*** 243,248 ****
--- 243,249 ----
void
set_cheapest(RelOptInfo *parent_rel)
{
+ bool grouped = parent_rel->agg_info != NULL;
Path *cheapest_startup_path;
Path *cheapest_total_path;
Path *best_param_path;
*************** set_cheapest(RelOptInfo *parent_rel)
*** 252,258 ****
Assert(IsA(parent_rel, RelOptInfo));
if (parent_rel->pathlist == NIL)
! elog(ERROR, "could not devise a query plan for the given query");
cheapest_startup_path = cheapest_total_path = best_param_path = NULL;
parameterized_paths = NIL;
--- 253,273 ----
Assert(IsA(parent_rel, RelOptInfo));
if (parent_rel->pathlist == NIL)
! {
! if (!grouped)
! elog(ERROR, "could not devise a query plan for the given query");
! else
! {
! /*
! * Creation of grouped paths is not guaranteed.
! */
! if (IS_SIMPLE_REL(parent_rel) || IS_JOIN_REL(parent_rel))
! mark_dummy_rel(parent_rel);
! else
! Assert(false);
! return;
! }
! }
cheapest_startup_path = cheapest_total_path = best_param_path = NULL;
parameterized_paths = NIL;
*************** create_seqscan_path(PlannerInfo *root, R
*** 949,958 ****
Relids required_outer, int parallel_workers)
{
Path *pathnode = makeNode(Path);
pathnode->pathtype = T_SeqScan;
pathnode->parent = rel;
! pathnode->pathtarget = rel->reltarget;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
--- 964,978 ----
Relids required_outer, int parallel_workers)
{
Path *pathnode = makeNode(Path);
+ bool grouped = rel->agg_info != NULL;
pathnode->pathtype = T_SeqScan;
pathnode->parent = rel;
! /* For grouped relation only generate the aggregation input. */
! if (!grouped)
! pathnode->pathtarget = rel->reltarget;
! else
! pathnode->pathtarget = rel->agg_info->input;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
*************** create_index_path(PlannerInfo *root,
*** 1032,1041 ****
RelOptInfo *rel = index->rel;
List *indexquals,
*indexqualcols;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1052,1066 ----
RelOptInfo *rel = index->rel;
List *indexquals,
*indexqualcols;
+ bool grouped = rel->agg_info != NULL;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
pathnode->path.parent = rel;
! /* For grouped relation only generate the aggregation input. */
! if (!grouped)
! pathnode->path.pathtarget = rel->reltarget;
! else
! pathnode->path.pathtarget = rel->agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** create_tidscan_path(PlannerInfo *root, R
*** 1183,1192 ****
Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
pathnode->path.pathtype = T_TidScan;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1208,1222 ----
Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
+ bool grouped = rel->agg_info != NULL;
pathnode->path.pathtype = T_TidScan;
pathnode->path.parent = rel;
! /* For grouped relation only generate the aggregation input. */
! if (!grouped)
! pathnode->path.pathtarget = rel->reltarget;
! else
! pathnode->path.pathtarget = rel->agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** create_append_path(RelOptInfo *rel,
*** 1218,1229 ****
{
AppendPath *pathnode = makeNode(AppendPath);
ListCell *l;
Assert(!parallel_aware || parallel_workers > 0);
pathnode->path.pathtype = T_Append;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = parallel_aware;
--- 1248,1261 ----
{
AppendPath *pathnode = makeNode(AppendPath);
ListCell *l;
+ bool grouped = rel->agg_info != NULL;
Assert(!parallel_aware || parallel_workers > 0);
pathnode->path.pathtype = T_Append;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = !grouped ? rel->reltarget :
! rel->agg_info->target;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = parallel_aware;
*************** append_startup_cost_compare(const void *
*** 1317,1327 ****
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
List *subpaths,
List *pathkeys,
Relids required_outer,
--- 1349,1361 ----
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode. target can be supplied by caller. If NULL is passed, the field
! * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** create_merge_append_path(PlannerInfo *ro
*** 1334,1340 ****
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1368,1374 ----
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = target ? target : rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** create_unique_path(PlannerInfo *root, Re
*** 1504,1510 ****
MemoryContext oldcontext;
int numCols;
! /* Caller made a mistake if subpath isn't cheapest_total ... */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
--- 1538,1546 ----
MemoryContext oldcontext;
int numCols;
! /*
! * Caller made a mistake if subpath isn't cheapest_total.
! */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
*************** calc_non_nestloop_required_outer(Path *o
*** 2125,2130 ****
--- 2161,2167 ----
* relations.
*
* 'joinrel' is the join relation.
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_nestloop
* 'extra' contains various information about the join
*************** calc_non_nestloop_required_outer(Path *o
*** 2139,2144 ****
--- 2176,2182 ----
NestPath *
create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** create_nestloop_path(PlannerInfo *root,
*** 2179,2185 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2217,2223 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_nestloop_path(PlannerInfo *root,
*** 2211,2216 ****
--- 2249,2255 ----
* two relations
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_mergejoin
* 'extra' contains various information about the join
*************** create_nestloop_path(PlannerInfo *root,
*** 2227,2232 ****
--- 2266,2272 ----
MergePath *
create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2243,2249 ****
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2283,2289 ----
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2279,2284 ****
--- 2319,2325 ----
* Creates a pathnode corresponding to a hash join between two relations.
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_hashjoin
* 'extra' contains various information about the join
*************** create_mergejoin_path(PlannerInfo *root,
*** 2293,2298 ****
--- 2334,2340 ----
HashPath *
create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2307,2313 ****
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2349,2355 ----
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_projection_path(PlannerInfo *root
*** 2389,2396 ****
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs))
{
/* No separate Result node needed */
pathnode->dummypp = true;
--- 2431,2438 ----
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if ((is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
*************** create_agg_path(PlannerInfo *root,
*** 2775,2782 ****
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
! /* For now, assume we are above any joins, so no parameterization */
! pathnode->path.param_info = NULL;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
--- 2817,2823 ----
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
! pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
*************** create_agg_path(PlannerInfo *root,
*** 2809,2814 ****
--- 2850,3004 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * check_pathkeys can be passed FALSE if the function was already called for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * agg_info contains both aggregate and grouping expressions.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool check_pathkeys, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+ RelAggInfo *agg_info;
+
+ rel = subpath->parent;
+ agg_info = rel->agg_info;
+ Assert(agg_info != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ if (check_pathkeys)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(agg_info->agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) agg_info->agg_exprs,
+ AGGSPLIT_INITIAL_SERIAL, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ Assert(agg_info->group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, agg_info->target,
+ AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
+ agg_info->group_clauses, NIL, &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Apply partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+ RelAggInfo *agg_info;
+
+ rel = subpath->parent;
+ agg_info = rel->agg_info;
+ Assert(agg_info != NULL);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(agg_info->agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) agg_info->agg_exprs,
+ AGGSPLIT_INITIAL_SERIAL, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(agg_info->group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ agg_info->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ agg_info->group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index da8f0f9..6eaba9f
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 17,22 ****
--- 17,23 ----
#include <limits.h>
#include "miscadmin.h"
+ #include "catalog/pg_constraint_fn.h"
#include "catalog/partition.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
***************
*** 27,32 ****
--- 28,35 ----
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
+ #include "parser/parse_oper.h"
#include "utils/hsearch.h"
*************** static List *subbuild_joinrel_joinlist(R
*** 53,62 ****
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
! static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
/*
--- 56,69 ----
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
! static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel,
! bool grouped);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
+ static void init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
*************** setup_simple_rel_arrays(PlannerInfo *roo
*** 72,80 ****
/* Arrays are accessed using RT indexes (1..N) */
root->simple_rel_array_size = list_length(root->parse->rtable) + 1;
! /* simple_rel_array is initialized to all NULLs */
root->simple_rel_array = (RelOptInfo **)
palloc0(root->simple_rel_array_size * sizeof(RelOptInfo *));
/* simple_rte_array is an array equivalent of the rtable list */
root->simple_rte_array = (RangeTblEntry **)
--- 79,92 ----
/* Arrays are accessed using RT indexes (1..N) */
root->simple_rel_array_size = list_length(root->parse->rtable) + 1;
! /*
! * simple_rel_array / simple_grouped_rel_array are both initialized to all
! * NULLs
! */
root->simple_rel_array = (RelOptInfo **)
palloc0(root->simple_rel_array_size * sizeof(RelOptInfo *));
+ root->simple_grouped_rel_array = (RelOptInfo **)
+ palloc0(root->simple_rel_array_size * sizeof(RelOptInfo *));
/* simple_rte_array is an array equivalent of the rtable list */
root->simple_rte_array = (RangeTblEntry **)
*************** build_simple_rel(PlannerInfo *root, int
*** 111,117 ****
rel->reloptkind = parent ? RELOPT_OTHER_MEMBER_REL : RELOPT_BASEREL;
rel->relids = bms_make_singleton(relid);
rel->rows = 0;
! /* cheap startup cost is interesting iff not all tuples to be retrieved */
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
--- 123,136 ----
rel->reloptkind = parent ? RELOPT_OTHER_MEMBER_REL : RELOPT_BASEREL;
rel->relids = bms_make_singleton(relid);
rel->rows = 0;
!
! /*
! * Cheap startup cost is interesting iff not all tuples to be retrieved.
! * XXX As for grouped relation, the startup cost might be interesting for
! * AGG_SORTED (if it can produce the ordering that matches
! * root->query_pathkeys) but not in general (other kinds of aggregation
! * need the whole relation). Yet it seems worth trying.
! */
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
*************** build_simple_rel(PlannerInfo *root, int
*** 125,130 ****
--- 144,150 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->agg_info = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** find_base_rel(PlannerInfo *root, int rel
*** 293,306 ****
}
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
*/
static void
! build_join_rel_hash(PlannerInfo *root)
{
HTAB *hashtab;
HASHCTL hash_ctl;
ListCell *l;
/* Create the hash table */
--- 313,349 ----
}
/*
+ * find_grouped_base_rel
+ * Find a grouped base or other relation entry, which does not have to
+ * exist.
+ */
+ RelOptInfo *
+ find_grouped_base_rel(PlannerInfo *root, int relid)
+ {
+ RelOptInfo *rel;
+
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ rel = root->simple_grouped_rel_array[relid];
+ if (rel)
+ return rel;
+ }
+
+ return NULL;
+ }
+
+ /*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
*/
static void
! build_join_rel_hash(PlannerInfo *root, bool grouped)
{
HTAB *hashtab;
HASHCTL hash_ctl;
+ List *join_rel_list;
ListCell *l;
/* Create the hash table */
*************** build_join_rel_hash(PlannerInfo *root)
*** 316,322 ****
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
! foreach(l, root->join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
JoinHashEntry *hentry;
--- 359,367 ----
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
! join_rel_list = !grouped ? root->join_rel_list :
! root->join_grouped_rel_list;
! foreach(l, join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
JoinHashEntry *hentry;
*************** build_join_rel_hash(PlannerInfo *root)
*** 330,336 ****
hentry->join_rel = rel;
}
! root->join_rel_hash = hashtab;
}
/*
--- 375,384 ----
hentry->join_rel = rel;
}
! if (!grouped)
! root->join_rel_hash = hashtab;
! else
! root->join_grouped_rel_hash = hashtab;
}
/*
*************** build_join_rel_hash(PlannerInfo *root)
*** 339,352 ****
* or NULL if none exists. This is for join relations.
*/
RelOptInfo *
! find_join_rel(PlannerInfo *root, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
! if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
! build_join_rel_hash(root);
/*
* Use either hashtable lookup or linear search, as appropriate.
--- 387,419 ----
* or NULL if none exists. This is for join relations.
*/
RelOptInfo *
! find_join_rel(PlannerInfo *root, Relids relids, bool grouped)
{
+ HTAB *join_rel_hash;
+ List *join_rel_list;
+
+ if (!grouped)
+ {
+ join_rel_hash = root->join_rel_hash;
+ join_rel_list = root->join_rel_list;
+ }
+ else
+ {
+ join_rel_hash = root->join_grouped_rel_hash;
+ join_rel_list = root->join_grouped_rel_list;
+ }
+
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
! if (!join_rel_hash && list_length(join_rel_list) > 32)
! {
! build_join_rel_hash(root, grouped);
!
! join_rel_hash = !grouped ? root->join_rel_hash :
! root->join_grouped_rel_hash;
! }
/*
* Use either hashtable lookup or linear search, as appropriate.
*************** find_join_rel(PlannerInfo *root, Relids
*** 356,367 ****
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
! if (root->join_rel_hash)
{
Relids hashkey = relids;
JoinHashEntry *hentry;
! hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
&hashkey,
HASH_FIND,
NULL);
--- 423,434 ----
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
! if (join_rel_hash)
{
Relids hashkey = relids;
JoinHashEntry *hentry;
! hentry = (JoinHashEntry *) hash_search(join_rel_hash,
&hashkey,
HASH_FIND,
NULL);
*************** find_join_rel(PlannerInfo *root, Relids
*** 372,378 ****
{
ListCell *l;
! foreach(l, root->join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
--- 439,445 ----
{
ListCell *l;
! foreach(l, join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
*************** set_foreign_rel_properties(RelOptInfo *j
*** 440,457 ****
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
! add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
{
! /* GEQO requires us to append the new joinrel to the end of the list! */
! root->join_rel_list = lappend(root->join_rel_list, joinrel);
/* store it into the auxiliary hashtable if there is one. */
! if (root->join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
! hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
&(joinrel->relids),
HASH_ENTER,
&found);
--- 507,537 ----
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
! add_join_rel(PlannerInfo *root, RelOptInfo *joinrel, bool grouped)
{
! HTAB *join_rel_hash;
!
! if (!grouped)
! {
! /*
! * GEQO requires us to append the new joinrel to the end of the list!
! */
! root->join_rel_list = lappend(root->join_rel_list, joinrel);
! }
! else
! root->join_grouped_rel_list = lappend(root->join_grouped_rel_list,
! joinrel);
!
! join_rel_hash = !grouped ? root->join_rel_hash :
! root->join_grouped_rel_hash;
/* store it into the auxiliary hashtable if there is one. */
! if (join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
! hentry = (JoinHashEntry *) hash_search(join_rel_hash,
&(joinrel->relids),
HASH_ENTER,
&found);
*************** add_join_rel(PlannerInfo *root, RelOptIn
*** 472,477 ****
--- 552,559 ----
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'grouped': does the join contain partial aggregate? (If it does, then
+ * caller is responsible for setup of reltarget.)
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
*************** build_join_rel(PlannerInfo *root,
*** 482,491 ****
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr)
{
RelOptInfo *joinrel;
List *restrictlist;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
--- 564,575 ----
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr,
! bool grouped)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool create_target = !grouped;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
*************** build_join_rel(PlannerInfo *root,
*** 493,499 ****
/*
* See if we already have a joinrel for this set of base rels.
*/
! joinrel = find_join_rel(root, joinrelids);
if (joinrel)
{
--- 577,583 ----
/*
* See if we already have a joinrel for this set of base rels.
*/
! joinrel = find_join_rel(root, joinrelids, grouped);
if (joinrel)
{
*************** build_join_rel(PlannerInfo *root,
*** 516,526 ****
joinrel->reloptkind = RELOPT_JOINREL;
joinrel->relids = bms_copy(joinrelids);
joinrel->rows = 0;
! /* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
--- 600,610 ----
joinrel->reloptkind = RELOPT_JOINREL;
joinrel->relids = bms_copy(joinrelids);
joinrel->rows = 0;
! /* See the comment in build_simple_rel(). */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
*************** build_join_rel(PlannerInfo *root,
*** 534,539 ****
--- 618,624 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->agg_info = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 582,590 ****
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
--- 667,679 ----
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! if (create_target)
! {
! joinrel->reltarget = create_empty_pathtarget();
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
! }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
*************** build_join_rel(PlannerInfo *root,
*** 621,651 ****
/*
* Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
*
! * Note that if there are more than two rels in this relation, they could
! * be divided between inner_rel and outer_rel in any arbitrary way. We
! * assume this doesn't matter, because we should hit all the same baserels
! * and joinclauses while building up to this joinrel no matter which we
! * take; therefore, we should make the same decision here however we get
! * here.
*/
! if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
! is_parallel_safe(root, (Node *) restrictlist) &&
! is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
! joinrel->consider_parallel = true;
/* Add the joinrel to the PlannerInfo. */
! add_join_rel(root, joinrel);
/*
* Also, if dynamic-programming join search is active, add the new joinrel
--- 710,747 ----
/*
* Set estimates of the joinrel's size.
*
! * XXX The function claims to need reltarget but it does not seem to
! * actually use it. Should we call it unconditionally so that callers of
! * build_join_rel() do not have to care?
*/
! if (create_target)
! {
! set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
! sjinfo, restrictlist);
!
! /*
! * Set the consider_parallel flag if this joinrel could potentially be
! * scanned within a parallel worker. If this flag is false for either
! * inner_rel or outer_rel, then it must be false for the joinrel also.
! * Even if both are true, there might be parallel-restricted
! * expressions in the targetlist or quals.
! *
! * Note that if there are more than two rels in this relation, they
! * could be divided between inner_rel and outer_rel in any arbitrary
! * way. We assume this doesn't matter, because we should hit all the
! * same baserels and joinclauses while building up to this joinrel no
! * matter which we take; therefore, we should make the same decision
! * here however we get here.
! */
! if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
! is_parallel_safe(root, (Node *) restrictlist) &&
! is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
! joinrel->consider_parallel = true;
! }
/* Add the joinrel to the PlannerInfo. */
! add_join_rel(root, joinrel, grouped);
/*
* Also, if dynamic-programming join search is active, add the new joinrel
*************** build_join_rel(PlannerInfo *root,
*** 653,664 ****
* of members should be for equality, but some of the level 1 rels might
* have been joinrels already, so we can only assert <=.
*/
! if (root->join_rel_level)
{
Assert(root->join_cur_level > 0);
Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
! root->join_rel_level[root->join_cur_level] =
! lappend(root->join_rel_level[root->join_cur_level], joinrel);
}
return joinrel;
--- 749,766 ----
* of members should be for equality, but some of the level 1 rels might
* have been joinrels already, so we can only assert <=.
*/
! if ((!grouped && root->join_rel_level) ||
! (grouped && root->join_grouped_rel_level))
{
Assert(root->join_cur_level > 0);
Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
! if (!grouped)
! root->join_rel_level[root->join_cur_level] =
! lappend(root->join_rel_level[root->join_cur_level], joinrel);
! else
! root->join_grouped_rel_level[root->join_cur_level] =
! lappend(root->join_grouped_rel_level[root->join_cur_level],
! joinrel);
}
return joinrel;
*************** build_join_rel(PlannerInfo *root,
*** 677,692 ****
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
* 'jointype' is the join type (inner, left, full, etc)
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
List *restrictlist, SpecialJoinInfo *sjinfo,
! JoinType jointype)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
int nappinfos;
/* Only joins between "other" relations land here. */
Assert(IS_OTHER_REL(outer_rel) && IS_OTHER_REL(inner_rel));
--- 779,797 ----
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
* 'jointype' is the join type (inner, left, full, etc)
+ * 'grouped': does the join contain partial aggregate? (If it does, then
+ * caller is responsible for setup of reltarget.)
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
List *restrictlist, SpecialJoinInfo *sjinfo,
! JoinType jointype, bool grouped)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
int nappinfos;
+ bool create_target = !grouped;
/* Only joins between "other" relations land here. */
Assert(IS_OTHER_REL(outer_rel) && IS_OTHER_REL(inner_rel));
*************** build_child_join_rel(PlannerInfo *root,
*** 694,704 ****
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
joinrel->rows = 0;
! /* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
--- 799,809 ----
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
joinrel->rows = 0;
! /* See the comment in build_simple_rel(). */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
*************** build_child_join_rel(PlannerInfo *root,
*** 708,713 ****
--- 813,819 ----
joinrel->cheapest_parameterized_paths = NIL;
joinrel->direct_lateral_relids = NULL;
joinrel->lateral_relids = NULL;
+ joinrel->agg_info = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_child_join_rel(PlannerInfo *root,
*** 744,754 ****
/* Compute information relevant to foreign relations. */
set_foreign_rel_properties(joinrel, outer_rel, inner_rel);
! /* Build targetlist */
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! /* Add placeholder variables. */
! add_placeholders_to_child_joinrel(root, joinrel, parent_joinrel);
/* Construct joininfo list. */
appinfos = find_appinfos_by_relids(root, joinrel->relids, &nappinfos);
--- 850,864 ----
/* Compute information relevant to foreign relations. */
set_foreign_rel_properties(joinrel, outer_rel, inner_rel);
! if (create_target)
! {
! /* Build targetlist */
! joinrel->reltarget = create_empty_pathtarget();
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! /* Add placeholder variables. */
! add_placeholders_to_child_joinrel(root, joinrel, parent_joinrel);
! }
/* Construct joininfo list. */
appinfos = find_appinfos_by_relids(root, joinrel->relids, &nappinfos);
*************** build_child_join_rel(PlannerInfo *root,
*** 756,762 ****
(Node *) parent_joinrel->joininfo,
nappinfos,
appinfos);
- pfree(appinfos);
/*
* Lateral relids referred in child join will be same as that referred in
--- 866,871 ----
*************** build_child_join_rel(PlannerInfo *root,
*** 783,796 ****
/* Set estimates of the child-joinrel's size. */
! set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
! sjinfo, restrictlist);
/* We build the join only once. */
! Assert(!find_join_rel(root, joinrel->relids));
/* Add the relation to the PlannerInfo. */
! add_join_rel(root, joinrel);
return joinrel;
}
--- 892,909 ----
/* Set estimates of the child-joinrel's size. */
! /* XXX See the corresponding comment in build_join_rel(). */
! if (create_target)
! set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
! sjinfo, restrictlist);
/* We build the join only once. */
! Assert(!find_join_rel(root, joinrel->relids, grouped));
/* Add the relation to the PlannerInfo. */
! add_join_rel(root, joinrel, grouped);
!
! pfree(appinfos);
return joinrel;
}
*************** build_joinrel_partition_info(RelOptInfo
*** 1751,1753 ****
--- 1864,2471 ----
joinrel->nullable_partexprs[cnt] = nullable_partexpr;
}
}
+
+ /*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+ RelAggInfo *
+ create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *gvis;
+ List *aggregates = NIL;
+ List *grp_exprs = NIL;
+ bool found_higher_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *target,
+ *agg_input;
+ List *grp_exprs_extra = NIL;
+ int i;
+ List *sortgroupclauses = NIL;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ /*
+ * The source relation has nothing to do with grouping.
+ */
+ Assert(rel->agg_info == NULL);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some grouping expression or Aggref
+ * argument), we'd just let init_grouping_targets create GroupedVar for
+ * the corresponding expression (phexpr). On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we'd ignore it because
+ * the referencing GroupedVar would take care of propagation of the value
+ * to upper joins. (PHV whose ph_eval_at is above the current rel make the
+ * aggregation push-down impossible in any case because the partial
+ * aggregation would receive wrong input if we ignored the ph_eval_at.)
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_higher_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ *
+ * XXX A plain Var could actually be handled w/o GroupedVar, but
+ * thus init_grouping_targets would have to spend extra effort
+ * looking for the EC-related vars, instead of relying on
+ * create_grouping_expr_grouped_var_infos. (Processing of
+ * particular expression would look different, so we could hardly
+ * reuse the same piece of code.)
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Accept the aggregate / grouping expression.
+ *
+ * (GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.)
+ */
+ if (IsA(gvi->gvexpr, Aggref))
+ aggregates = lappend(aggregates, gvi);
+ else
+ grp_exprs = lappend(grp_exprs, gvi);
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids) &&
+ IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs more than this rel.
+ */
+ found_higher_agg = true;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need multiple relations including
+ * the current one. The problem is that grouping of the current relation
+ * could make some input variables unavailable for the "higher aggregate",
+ * and it'd also decrease the number of input rows the "higher aggregate"
+ * receives.
+ *
+ * In contrast, grp_exprs is only supposed to contain generic grouping
+ * expression, so it can be NIL so far. If all the grouping keys are just
+ * plain Vars, init_grouping_targets will take care of them.
+ */
+ if (found_higher_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the partial aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+ init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra);
+ list_free(gvis);
+
+ /*
+ * Add (non-Var) grouping expressions (in the form of GroupedVar) to
+ * target_agg.
+ *
+ * Follow the convention that the grouping expressions should precede
+ * aggregates.
+ */
+ add_grouped_vars_to_target(root, target, grp_exprs);
+
+ /*
+ * Partial aggregation makes no sense w/o grouping expressions.
+ */
+ if (list_length(target->exprs) == 0)
+ return NULL;
+
+ /*
+ * If the aggregation target should have extra grouping expressions, add
+ * them now. This step includes assignment of tleSortGroupRef's which we
+ * can generate now (the "ordinary" grouping expression are present in the
+ * target by now).
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+ int i = 0;
+ ListCell *lc2;
+
+ /*
+ * TODO Verify that these fields are sufficient for this special
+ * SortGroupClause.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ NULL, &cl->eqop, NULL,
+ &cl->hashable);
+ sortgroupclauses = lappend(sortgroupclauses, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too. It can
+ * already be there, so avoid adding it again.
+ */
+ foreach(lc2, agg_input->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc2);
+
+ if (equal(expr, var))
+ {
+ /*
+ * The fact that the var is in agg_input does not imply
+ * that it has sortgroupref set. For example, the reason
+ * that it's there can be that a generic grouping
+ * expression references it, so grouping by the var alone
+ * hasn't been considered so far.
+ */
+ if (agg_input->sortgrouprefs == NULL)
+ {
+ agg_input->sortgrouprefs = (Index *)
+ palloc0(list_length(agg_input->exprs) *
+ sizeof(Index));
+ }
+ if (agg_input->sortgrouprefs[i] == 0)
+ agg_input->sortgrouprefs[i] = cl->tleSortGroupRef;
+
+ break;
+ }
+
+ i++;
+ }
+ if (lc2 != NULL)
+ continue;
+
+ /*
+ * Add the var if it's not in the target yet.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the grouping target.
+ */
+ add_grouped_vars_to_target(root, target, aggregates);
+
+ /*
+ * Make sure that the paths generating input data for partial aggregation
+ * include non-Var grouping expressions.
+ */
+ add_grouped_vars_to_target(root, agg_input, grp_exprs);
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result = makeNode(RelAggInfo);
+ result->target = target;
+ result->input = agg_input;
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, GroupedVar) &&
+ IsA(((GroupedVar *) texpr)->gvexpr, Aggref))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ cl = get_sortgroupref_clause_noerr(sortgroupref,
+ root->parse->groupClause);
+
+ /*
+ * If query does not have this clause, it must be target-specific.
+ */
+ if (cl == NULL)
+ cl = get_sortgroupref_clause(sortgroupref, sortgroupclauses);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(lc));
+
+ /*
+ * Aggregate GroupedVarInfo should always point to the partial
+ * aggregate.
+ */
+ Assert(gvar->agg_partial != NULL);
+ result->agg_exprs = lappend(result->agg_exprs, gvar->agg_partial);
+ lc = lnext(lc);
+ }
+
+ return result;
+ }
+
+ /*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for partial aggregation (agg_input).
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * The *group_exprs_extra_p list may receive additional grouping expressions
+ * that the query does not have. These can make the aggregation of base
+ * relation / join less efficient, but can allow for join of the grouped
+ * relation that wouldn't be possible otherwise.
+ */
+ static void
+ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+ {
+ ListCell *lc;
+ List *vars_unresolved = NIL;
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ GroupedVar *gvar;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var. (As for generic grouping expressions,
+ * add_grouped_vars_to_target will retrieve them from the query
+ * targetlist and add them to "target" outside this function.)
+ */
+ tvar = lfirst_node(Var, lc);
+
+ gvar = get_grouping_expression(gvis, (Expr *) tvar);
+ if (gvar != NULL)
+ {
+ /*
+ * It's o.k. to use the target expression for grouping.
+ *
+ * The actual Var is added to the target. If we used the
+ * containing GroupedVar, references from various clauses (e.g.
+ * join quals) wouldn't work.
+ */
+ add_column_to_pathtarget(target, gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the plain-var grouping expressions gathered. So far
+ * only store the var in a list.
+ */
+ vars_unresolved = lappend(vars_unresolved, tvar);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ foreach(lc, vars_unresolved)
+ {
+ Var *var;
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ var = lfirst_node(Var, lc);
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Dependent var is almost the same as one that has sortgroupref.
+ */
+ if (check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+
+ Index sortgroupref = 0;
+
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important. But once we have it ...
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) or aggregate arguments, but we already have it in
+ * the targets, so don't care.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon as GroupedVars are installed, the top level
+ * aggregates / grouping expressions no longer need direct reference
+ * to arg_var anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by upper join. This includes one that is
+ * referenced by a generic grouping expression but couldn't be
+ * recognized as grouping expression on its own at the top of the
+ * loop.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ *
+ * Since root->parse->groupClause is not supposed to contain this
+ * expression, we need to construct special SortGroupClause. Its
+ * tleSortGroupRef needs to be unique within "target", so postpone
+ * creation of the SortGroupRefs until we're done with the
+ * iteration of rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced either by aggregate argument
+ * or by generic grouping expression. The per-relation aggregation
+ * target should not contain it, as it only provides input for the
+ * final aggregation.
+ */
+ }
+
+ /*
+ * The var is not suitable for grouping, but agg_input ought to stay
+ * complete.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var, 0);
+ }
+ }
+
+
+ /*
+ * Translate RelAggInfo of parent relation so it matches given child relation.
+ */
+ RelAggInfo *
+ translate_rel_agg_info(PlannerInfo *root, RelAggInfo *parent,
+ AppendRelInfo **appinfos, int nappinfos)
+ {
+ RelAggInfo *result;
+
+ result = makeNode(RelAggInfo);
+
+ result->target = copy_pathtarget(parent->target);
+ result->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) result->target->exprs,
+ nappinfos, appinfos);
+
+ result->input = copy_pathtarget(parent->input);
+ result->input->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) result->input->exprs,
+ nappinfos, appinfos);
+
+ result->group_clauses = parent->group_clauses;
+
+ result->group_exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) parent->group_exprs,
+ nappinfos, appinfos);
+ result->agg_exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) parent->agg_exprs,
+ nappinfos, appinfos);
+ return result;
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 32160d5..6d74323
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,414 ****
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
--- 408,413 ----
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 782,900 ----
}
/*
+ * Replace each GroupedVar in the source targetlist with the original
+ * expression --- either Aggref or a non-Var grouping expression.
+ *
+ * Even if the query targetlist has the Aggref wrapped in a generic
+ * expression, any subplan should emit the corresponding GroupedVar
+ * alone. (Aggregate finalization is needed before the aggregate result can be
+ * used for any purposes and that happens at the top level of the query.)
+ * Therefore we do not have to recurse into the target expressions here.
+ */
+ List *
+ replace_grouped_vars_with_aggrefs(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te,
+ *te_new;
+ Expr *expr_new = NULL;
+
+ te = lfirst_node(TargetEntry, l);
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Partial aggregate should appear in the targetlist so that
+ * it looks as if convert_combining_aggrefs arranged it.
+ */
+ expr_new = (Expr *) gvar->agg_partial;
+ }
+ else
+ expr_new = gvar->gvexpr;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
+ * For each aggregate add GroupedVar to the grouped target.
+ *
+ * Caller passes the aggregates in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+ void
+ add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+ {
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+ GroupedVar *gvar;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_column_to_pathtarget(target, (Expr *) gvar, gvi->sortgroupref);
+ }
+ }
+
+ /*
+ * Return GroupedVar containing the passed-in expression if one exists, or
+ * NULL if the expression cannot be used as grouping key.
+ */
+ GroupedVar *
+ get_grouping_expression(List *gvis, Expr *expr)
+ {
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ GroupedVar *result = makeNode(GroupedVar);
+
+ Assert(gvi->sortgroupref > 0);
+ result->gvexpr = gvi->gvexpr;
+ result->gvid = gvi->gvid;
+ result->sortgroupref = gvi->sortgroupref;
+ return result;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return NULL;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
new file mode 100644
index b16b1e4..459dc30
*** a/src/backend/optimizer/util/var.c
--- b/src/backend/optimizer/util/var.c
*************** alias_relid_set(PlannerInfo *root, Relid
*** 840,842 ****
--- 840,864 ----
}
return result;
}
+
+ /*
+ * Return GroupedVarInfo for given GroupedVar.
+ *
+ * XXX Consider better location of this routine.
+ */
+ GroupedVarInfo *
+ find_grouped_var_info(PlannerInfo *root, GroupedVar *gvar)
+ {
+ ListCell *l;
+
+ foreach(l, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, l);
+
+ if (gvi->gvid == gvar->gvid)
+ return gvi;
+ }
+
+ elog(ERROR, "GroupedVarInfo not found");
+ return NULL; /* keep compiler quiet */
+ }
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
new file mode 100644
index 2a4ac09..267c0b9
*** a/src/backend/parser/parse_func.c
--- b/src/backend/parser/parse_func.c
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 98,103 ****
--- 98,104 ----
Oid vatype;
FuncDetailCode fdresult;
char aggkind = 0;
+ Oid aggcombinefn = InvalidOid;
ParseCallbackState pcbstate;
/*
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 350,355 ****
--- 351,357 ----
elog(ERROR, "cache lookup failed for aggregate %u", funcid);
classForm = (Form_pg_aggregate) GETSTRUCT(tup);
aggkind = classForm->aggkind;
+ aggcombinefn = classForm->aggcombinefn;
catDirectArgs = classForm->aggnumdirectargs;
ReleaseSysCache(tup);
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 695,700 ****
--- 697,703 ----
aggref->aggstar = agg_star;
aggref->aggvariadic = func_variadic;
aggref->aggkind = aggkind;
+ aggref->aggcombinefn = aggcombinefn;
/* agglevelsup will be set by transformAggregateCall */
aggref->aggsplit = AGGSPLIT_SIMPLE; /* planner might change this */
aggref->location = location;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index 3697466..4b49d32
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7686,7691 ****
--- 7686,7708 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+ Expr *expr = gvar->gvexpr;
+
+ if (IsA(expr, Aggref))
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ else if (IsA(expr, Var))
+ (void) get_variable((Var *) expr, 0, false, context);
+ else
+ {
+ Assert(IsA(gvar->gvexpr, OpExpr));
+ get_oper_expr((OpExpr *) expr, context);
+ }
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 9171,9180 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 9188,9205 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index fcc8323..72e14e5
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 114,119 ****
--- 114,120 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
*************** estimate_hash_bucket_stats(PlannerInfo *
*** 3863,3868 ****
--- 3864,3902 ----
ReleaseVariableStats(vardata);
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
*************** examine_variable(PlannerInfo *root, Node
*** 4793,4799 ****
if (varRelid == 0)
{
/* treat it as a variable of a join relation */
! vardata->rel = find_join_rel(root, varnos);
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
--- 4827,4833 ----
if (varRelid == 0)
{
/* treat it as a variable of a join relation */
! vardata->rel = find_join_rel(root, varnos, false);
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
*************** find_join_input_rel(PlannerInfo *root, R
*** 5651,5657 ****
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
! rel = find_join_rel(root, relids);
break;
}
--- 5685,5691 ----
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
! rel = find_join_rel(root, relids, false);
break;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index 1db7845..0caa05e
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_bool ConfigureNames
*** 923,928 ****
--- 923,937 ----
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index 74b094a..2914fc8
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 218,223 ****
--- 218,224 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 258,267 ****
--- 259,270 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index 1b4b0d7..6af31f2
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct Aggref
*** 296,301 ****
--- 296,302 ----
Oid aggcollid; /* OID of collation of result */
Oid inputcollid; /* OID of collation that function should use */
Oid aggtranstype; /* type Oid of aggregate's transition value */
+ Oid aggcombinefn; /* combine function (see pg_aggregate.h) */
List *aggargtypes; /* type Oids of direct and aggregated args */
List *aggdirectargs; /* direct arguments, if an ordered-set agg */
List *args; /* aggregated arguments and sort expressions */
*************** typedef struct Aggref
*** 306,311 ****
--- 307,313 ----
bool aggvariadic; /* true if variadic arguments have been
* combined into an array last argument */
char aggkind; /* aggregate kind (see pg_aggregate.h) */
+
Index agglevelsup; /* > 0 if agg belongs to outer query */
AggSplit aggsplit; /* expected agg-splitting mode of parent Agg */
int location; /* token location, or -1 if unknown */
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index db8de2d..1f10816
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 179,185 ****
* unreferenced view RTE; or if the RelOptInfo hasn't been made yet.
*/
struct RelOptInfo **simple_rel_array; /* All 1-rel RelOptInfos */
! int simple_rel_array_size; /* allocated size of array */
/*
* simple_rte_array is the same length as simple_rel_array and holds
--- 179,193 ----
* unreferenced view RTE; or if the RelOptInfo hasn't been made yet.
*/
struct RelOptInfo **simple_rel_array; /* All 1-rel RelOptInfos */
!
! /*
! * The same like simple_rel_array, but for grouped rels. In addition to
! * the meanings explained above, NULL can also mean that the relation
! * cannot be grouped alone, regardless its kind.
! */
! struct RelOptInfo **simple_grouped_rel_array; /* The same for grouped
! * rels. */
! int simple_rel_array_size; /* allocated size of the arrays above */
/*
* simple_rte_array is the same length as simple_rel_array and holds
*************** typedef struct PlannerInfo
*** 217,222 ****
--- 225,234 ----
List *join_rel_list; /* list of join-relation RelOptInfos */
struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ /* The same for grouped joins. */
+ List *join_grouped_rel_list;
+ struct HTAB *join_grouped_rel_hash;
+
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
*************** typedef struct PlannerInfo
*** 225,230 ****
--- 237,244 ----
* join_rel_level is NULL if not in use.
*/
List **join_rel_level; /* lists of join-relation RelOptInfos */
+ List **join_grouped_rel_level; /* lists of grouped join-relation
+ * RelOptInfos */
int join_cur_level; /* index of list being extended */
List *init_plans; /* init SubPlans for query */
*************** typedef struct PlannerInfo
*** 259,264 ****
--- 273,280 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 285,290 ****
--- 301,312 ----
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
*************** typedef struct PartitionSchemeData *Part
*** 440,445 ****
--- 462,469 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 611,616 ****
--- 635,643 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to apply partial aggregation to this rel's paths. */
+ struct RelAggInfo *agg_info;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 1009,1014 ****
--- 1036,1088 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * RelAggInfo
+ *
+ * RelOptInfo needs information contained here if its paths should be
+ * partially aggregated.
+ *
+ * "target" will be used as pathtarget of grouped paths produced by "explicit
+ * aggregation" of a relation, but also --- if the relation is a join --- by
+ * joining grouped path to a non-grouped one.
+ *
+ * The target contains plain-Var grouping expressions, generic grouping
+ * expressions wrapped in GroupedVar structure, or Aggrefs which are also
+ * wrapped in GroupedVar. Once GroupedVar is evaluated, its value is passed to
+ * the upper paths w/o being evaluated again. If final aggregation appears to
+ * be necessary above the final join, the contained Aggrefs are supposed to
+ * provide the final aggregation plan with input values, i.e. the aggregate
+ * transient state.
+ *
+ * Note: There's a convention that GroupedVars that contain Aggref expressions
+ * are supposed to follow the other expressions of the target. Iterations of
+ * target->exprs may rely on this arrangement.
+ *
+ * "input" contains Vars used either as grouping expressions or aggregate
+ * arguments, plus grouping expressions which are not plain vars. Paths
+ * providing the partial aggregation plan with input data should use this
+ * target.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively. "agg_exprs" is a list of
+ * Aggref nodes to be evaluated by the relation.
+ *
+ * "rows" is the estimated number of result tuples produced by grouped paths.
+ */
+ typedef struct RelAggInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* target of grouped paths. */
+ PathTarget *input; /* pathtarget of paths that generate input for
+ * the partial aggregation. */
+
+ List *group_clauses;
+ List *group_exprs;
+ List *agg_exprs;
+
+ double rows;
+ } RelAggInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct HashPath
*** 1486,1497 ****
--- 1560,1575 ----
* ProjectionPath node, which is marked dummy to indicate that we intend to
* assign the work to the input plan node. The estimated cost for the
* ProjectionPath node will account for whether a Result will be used or not.
+ *
+ * force_result field tells that the Result node must be used for some reason
+ * even though the subpath could normally handle the projection.
*/
typedef struct ProjectionPath
{
Path path;
Path *subpath; /* path representing input source */
bool dummypp; /* true if no separate Result is needed */
+ bool force_result; /* Is Result node required? */
} ProjectionPath;
/*
*************** typedef struct PlaceHolderVar
*** 1957,1962 ****
--- 2035,2065 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. Likewise, the variable is evaluated below the query targetlist (in
+ * particular, in the targetlist of AGGSPLIT_INITIAL_SERIAL aggregation node
+ * which has base relation or a join as the input) and bubbles up through the
+ * join tree until it reaches AGGSPLIT_FINAL_DESERIAL aggregation node.
+ *
+ * gvexpr is either Aggref or a generic (non-Var) grouping expression. (If a
+ * simple Var, we don't replace it with GroupedVar.)
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if
+ * gvexpr is Aggref.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index sortgroupref; /* SortGroupClause.tleSortGroupRef if gvexpr
+ * is grouping expression. */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PartitionedChildRelInfo
*** 2131,2138 ****
Index parent_relid;
List *child_rels;
! bool part_cols_updated; /* is the partition key of any of
! * the partitioned tables updated? */
} PartitionedChildRelInfo;
/*
--- 2234,2241 ----
Index parent_relid;
List *child_rels;
! bool part_cols_updated; /* is the partition key of any of the
! * partitioned tables updated? */
} PartitionedChildRelInfo;
/*
*************** typedef struct PlaceHolderInfo
*** 2174,2179 ****
--- 2277,2301 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
new file mode 100644
index ba4fa4b..6901f10
*** a/src/include/optimizer/clauses.h
--- b/src/include/optimizer/clauses.h
*************** extern Node *estimate_expression_value(P
*** 84,88 ****
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
!
#endif /* CLAUSES_H */
--- 84,89 ----
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
! extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
! GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
new file mode 100644
index 132e355..9d56761
*** a/src/include/optimizer/cost.h
--- b/src/include/optimizer/cost.h
*************** extern PGDLLIMPORT double parallel_tuple
*** 54,60 ****
extern PGDLLIMPORT double parallel_setup_cost;
extern PGDLLIMPORT int effective_cache_size;
extern PGDLLIMPORT Cost disable_cost;
! extern PGDLLIMPORT int max_parallel_workers_per_gather;
extern PGDLLIMPORT bool enable_seqscan;
extern PGDLLIMPORT bool enable_indexscan;
extern PGDLLIMPORT bool enable_indexonlyscan;
--- 54,60 ----
extern PGDLLIMPORT double parallel_setup_cost;
extern PGDLLIMPORT int effective_cache_size;
extern PGDLLIMPORT Cost disable_cost;
! extern PGDLLIMPORT int max_parallel_workers_per_gather;
extern PGDLLIMPORT bool enable_seqscan;
extern PGDLLIMPORT bool enable_indexscan;
extern PGDLLIMPORT bool enable_indexonlyscan;
*************** extern PGDLLIMPORT bool enable_gathermer
*** 70,76 ****
extern PGDLLIMPORT bool enable_partitionwise_join;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
! extern PGDLLIMPORT int constraint_exclusion;
extern double clamp_row_est(double nrows);
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
--- 70,77 ----
extern PGDLLIMPORT bool enable_partitionwise_join;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
! extern PGDLLIMPORT bool enable_agg_pushdown;
! extern PGDLLIMPORT int constraint_exclusion;
extern double clamp_row_est(double nrows);
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index ef7173f..8cafd54
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern AppendPath *create_append_path(Re
*** 71,76 ****
--- 71,77 ----
List *partitioned_rels, double rows);
extern MergeAppendPath *create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** extern Relids calc_non_nestloop_required
*** 123,128 ****
--- 124,130 ----
extern NestPath *create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** extern NestPath *create_nestloop_path(Pl
*** 134,139 ****
--- 136,142 ----
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** extern MergePath *create_mergejoin_path(
*** 148,153 ****
--- 151,157 ----
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** extern AggPath *create_agg_path(PlannerI
*** 197,202 ****
--- 201,213 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool check_pathkeys,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern void setup_simple_rel_arrays(Plan
*** 266,278 ****
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
! extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
--- 277,292 ----
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
! extern RelOptInfo *find_grouped_base_rel(PlannerInfo *root, int relid);
! extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids,
! bool grouped);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr,
! bool grouped);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
*************** extern ParamPathInfo *find_param_path_in
*** 300,305 ****
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
! SpecialJoinInfo *sjinfo, JoinType jointype);
!
#endif /* PATHNODE_H */
--- 314,324 ----
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
! SpecialJoinInfo *sjinfo, JoinType jointype,
! bool grouped);
! extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
! extern RelAggInfo *translate_rel_agg_info(PlannerInfo *root,
! RelAggInfo *agg_info,
! AppendRelInfo **appinfos,
! int nappinfos);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 94f9bb2..8352899
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
***************
*** 21,29 ****
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
! extern PGDLLIMPORT int geqo_threshold;
! extern PGDLLIMPORT int min_parallel_table_scan_size;
! extern PGDLLIMPORT int min_parallel_index_scan_size;
/* Hook for plugins to get control in set_rel_pathlist() */
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
--- 21,30 ----
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
! extern PGDLLIMPORT bool enable_agg_pushdown;
! extern PGDLLIMPORT int geqo_threshold;
! extern PGDLLIMPORT int min_parallel_table_scan_size;
! extern PGDLLIMPORT int min_parallel_index_scan_size;
/* Hook for plugins to get control in set_rel_pathlist() */
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
*************** typedef void (*set_join_pathlist_hook_ty
*** 41,66 ****
JoinPathExtraData *extra);
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
! typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
! int levels_needed,
! List *initial_rels);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
! extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
! extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
! List *initial_rels);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
Path *bitmapqual);
extern void generate_partitionwise_join_paths(PlannerInfo *root,
! RelOptInfo *rel);
#ifdef OPTIMIZER_DEBUG
extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
--- 42,89 ----
JoinPathExtraData *extra);
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
+ /*
+ * Result of standard_join_search() or join_search_hook().
+ *
+ * 'plain' is a join of two plain (non-grouped) relation.
+ *
+ * 'grouped' is either a join of one plain relation to one grouped, or a join
+ * of two plain relations whose (the join relation's) paths have all been
+ * subject to partial aggregation.
+ */
+ typedef struct JoinSearchResult
+ {
+ RelOptInfo *plain;
+ RelOptInfo *grouped;
+ } JoinSearchResult;
+
/* Hook for plugins to replace standard_join_search() */
! typedef JoinSearchResult *(*join_search_hook_type) (PlannerInfo *root,
! int levels_needed,
! List *initial_rels,
! List *initial_rels_grouped);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
! extern JoinSearchResult *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
! extern JoinSearchResult *standard_join_search(PlannerInfo *root,
! int levels_needed,
! List *initial_rels,
! List *initial_rels_grouped);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+
+ extern bool create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, bool precheck,
+ bool partial, AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
Path *bitmapqual);
extern void generate_partitionwise_join_paths(PlannerInfo *root,
! RelOptInfo *rel);
#ifdef OPTIMIZER_DEBUG
extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
*************** extern Expr *adjust_rowcompare_for_index
*** 92,98 ****
* tidpath.h
* routines to generate tid paths
*/
! extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
/*
* joinpath.c
--- 115,122 ----
* tidpath.h
* routines to generate tid paths
*/
! extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
/*
* joinpath.c
*************** extern void create_tidscan_paths(Planner
*** 101,114 ****
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
! List *restrictlist);
/*
* joinrels.c
* routines to determine which relations to join
*/
extern void join_search_one_level(PlannerInfo *root, int level);
! extern RelOptInfo *make_join_rel(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
--- 125,138 ----
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
! List *restrictlist, bool do_aggregate);
/*
* joinrels.c
* routines to determine which relations to join
*/
extern void join_search_one_level(PlannerInfo *root, int level);
! extern JoinSearchResult *make_join_rel(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index 7132c88..2bd1135
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** typedef void (*query_pathkeys_callback)
*** 38,44 ****
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra);
/*
* prototypes for plan/planagg.c
--- 38,45 ----
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra,
! RelOptInfo **partially_grouped);
/*
* prototypes for plan/planagg.c
*************** extern void add_base_rels_to_query(Plann
*** 76,81 ****
--- 77,84 ----
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+ extern void add_grouped_base_rels_to_query(PlannerInfo *root);
+ extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 9fa52e1..68c32e1
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
***************
*** 16,22 ****
#include "nodes/relation.h"
-
extern TargetEntry *tlist_member(Expr *node, List *targetlist);
extern TargetEntry *tlist_member_ignore_relabel(Expr *node, List *targetlist);
--- 16,21 ----
*************** extern Node *get_sortgroupclause_expr(So
*** 41,47 ****
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
-
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
extern SortGroupClause *get_sortgroupref_clause_noerr(Index sortref,
--- 40,45 ----
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 63,75 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *replace_grouped_vars_with_aggrefs(PlannerInfo *root, List *src);
+ extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+ extern GroupedVar *get_grouping_expression(List *gvis, Expr *expr);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/include/optimizer/var.h b/src/include/optimizer/var.h
new file mode 100644
index 43c53b5..5a795c3
*** a/src/include/optimizer/var.h
--- b/src/include/optimizer/var.h
*************** extern bool contain_vars_of_level(Node *
*** 36,40 ****
--- 36,42 ----
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
extern Node *flatten_join_alias_vars(PlannerInfo *root, Node *node);
+ extern GroupedVarInfo *find_grouped_var_info(PlannerInfo *root,
+ GroupedVar *gvar);
#endif /* VAR_H */
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 299c9f8..e033a90
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern void estimate_hash_bucket_stats(P
*** 210,215 ****
--- 210,219 ----
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
+
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index ...09b380d
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
***************
*** 0 ****
--- 1,316 ----
+ BEGIN;
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ CREATE INDEX ON agg_pushdown_child1(parent);
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ ANALYZE;
+ SET enable_agg_pushdown TO on;
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ (9 rows)
+
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (8 rows)
+
+ SET enable_seqscan TO on;
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (14 rows)
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+ (15 rows)
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (14 rows)
+
+ -- Generic grouping expression.
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Sort
+ Sort Key: (((c1.parent / 2)))
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (16 rows)
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+ SET enable_agg_pushdown TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Seq Scan on agg_pushdown_parent p
+ (11 rows)
+
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ SET enable_seqscan TO on;
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (16 rows)
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Parallel Seq Scan on agg_pushdown_parent p
+ -> Parallel Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Seq Scan on agg_pushdown_child2 c2
+ (17 rows)
+
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (16 rows)
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Sort
+ Sort Key: (((c1.parent / 2)))
+ -> Gather
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (18 rows)
+
+ ROLLBACK;
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
new file mode 100644
index 759f7d9..68d0407
*** a/src/test/regress/expected/sysviews.out
--- b/src/test/regress/expected/sysviews.out
*************** select count(*) >= 0 as ok from pg_prepa
*** 72,77 ****
--- 72,78 ----
select name, setting from pg_settings where name like 'enable%';
name | setting
---------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
*************** select name, setting from pg_settings wh
*** 87,93 ****
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (15 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
--- 88,94 ----
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (16 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
new file mode 100644
index ad9434f..611aeb4
*** a/src/test/regress/parallel_schedule
--- b/src/test/regress/parallel_schedule
*************** test: rules psql_crosstab amutils
*** 98,103 ****
--- 98,106 ----
test: select_parallel
test: write_parallel
+ # this one runs parallel workers too
+ test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
new file mode 100644
index 27cd498..fe8108e
*** a/src/test/regress/serial_schedule
--- b/src/test/regress/serial_schedule
*************** test: rules
*** 137,142 ****
--- 137,143 ----
test: psql_crosstab
test: select_parallel
test: write_parallel
+ test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index ...05e2f55
*** a/src/test/regress/sql/agg_pushdown.sql
--- b/src/test/regress/sql/agg_pushdown.sql
***************
*** 0 ****
--- 1,137 ----
+ BEGIN;
+
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ CREATE INDEX ON agg_pushdown_child1(parent);
+
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ ANALYZE;
+
+ SET enable_agg_pushdown TO on;
+
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_seqscan TO on;
+
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- Generic grouping expression.
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+
+ SET enable_agg_pushdown TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_seqscan TO on;
+
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ ROLLBACK;
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-07-06 11:11 Antonin Houska <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: Antonin Houska @ 2018-07-06 11:11 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: pgsql-hackers
Robert Haas <[email protected]> wrote:
> On Fri, Feb 23, 2018 at 11:08 AM, Antonin Houska <[email protected]> wrote:
> > I spent some more time thinking about this. What about adding a new strategy
> > number for hash index operator classes, e.g. HTBinaryEqualStrategyNumber? For
> > most types both HTEqualStrategyNumber and HTBinaryEqualStrategyNumber strategy
> > would point to the same operator, but types like numeric would naturally have
> > them different.
> >
> > Thus the pushed-down partial aggregation can only use the
> > HTBinaryEqualStrategyNumber's operator to compare grouping expressions. In the
> > initial version (until we have useful statistics for the binary values) we can
> > avoid the aggregation push-down if the grouping expression output type has the
> > two strategies implemented using different functions because, as you noted
> > upthread, grouping based on binary equality can result in excessive number of
> > groups.
> >
> > One open question is whether the binary equality operator needs a separate
> > operator class or not. If an opclass cares only about the binary equality, its
> > hash function(s) can be a lot simpler.
>
> Hmm. How about instead adding another regproc field to pg_type which
> stores the OID of a function that tests binary equality for that
> datatype? If that happens to be equal to the OID you got from the
> opclass, then you're all set.
I suppose you mean pg_operator, not pg_type. What I don't like about this is
that the new field would only be useful for very little fraction of
operators.
On the other hand, the drawback of an additional operator classes is that we'd
have to modify almost all the existing operator classes for the hash AM. (The
absence of the new strategy number in an operator class cannot mean that the
existing equality operator can be used to compare binary values too, because
thus we can't guarantee correct behavior of the already existing user-defined
operator classes.)
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-08-31 15:05 Antonin Houska <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2018-08-31 15:05 UTC (permalink / raw)
To: ; +Cc: pgsql-hackers
Antonin Houska <[email protected]> wrote:
> I've reworked the patch so that separate RelOptInfo is used for grouped
> relation. The attached patch is only the core part. Neither postgres_fdw
> changes nor the part that tries to avoid the final aggregation is included
> here. I'll add these when the patch does not seem to need another major rework.
This is the next version. After having posted a few notes to the [1] thread,
I'm returning to the original one so it can be referenced from my entry in the
CF application. As proposed in the other thread, it uses only "simple
aggregation". The 2-stage aggregation, which gives more power to the feature
and which also enables paralle processing for it, will be coded in a separate
patch.
I eventually decided abandon the Var substitution proposed by Heikki in
[2]. It's not necessary if we accept the restriction that the feature accepts
only simple column reference as grouping expression so far.
[1] https://www.postgresql.org/message-id/c165b72e-8dbb-2a24-291f-113aeb67b76a%40iki.fi
[2] https://www.postgresql.org/message-id/113e9594-7c08-0f1f-ad15-41773b56a86b%40iki.fi
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
[application/x-gzip] agg_pushdown_v8.tgz (45.1K, ../../18061.1535727958@localhost/2-agg_pushdown_v8.tgz)
download | 4 patch file(s) in archive:
agg_pushdown_v8/prepare_1.patch
commit b51f04769098d1aed9b9931f90926fc8b924cd2e
Author: Antonin Houska <[email protected]>
Date: Fri Aug 31 13:27:44 2018 +0200
Export estimate_hashagg_tablesize().
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 96bf0601a8..36de70a213 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -133,9 +133,6 @@ static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd,
List *target_list);
-static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
@@ -3646,40 +3643,6 @@ get_number_of_groups(PlannerInfo *root,
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
-static Size
-estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
-{
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
-}
-
-/*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index f1c78ffb65..bd4868cb56 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -113,6 +113,7 @@
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+#include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -3883,6 +3884,39 @@ estimate_hash_bucket_stats(PlannerInfo *root, Node *hashkey, double nbuckets,
ReleaseVariableStats(vardata);
}
+/*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+Size
+estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+{
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+}
/*-------------------------------------------------------------------------
*
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
index 95e44280c4..3a14fc6036 100644
--- a/src/include/utils/selfuncs.h
+++ b/src/include/utils/selfuncs.h
@@ -213,6 +213,9 @@ extern void estimate_hash_bucket_stats(PlannerInfo *root,
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
agg_pushdown_v8/prepare_2.patch
commit 4d952f9ec82b84adb008846705b90cc82042e3f4
Author: Antonin Houska <[email protected]>
Date: Fri Aug 31 13:34:27 2018 +0200
Make root->upper_targets available to query_planner().
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 36de70a213..a5fbf5f8af 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1887,23 +1887,6 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
else
root->limit_tuples = limit_tuples;
- /* Set up data needed by standard_qp_callback */
- qp_extra.tlist = tlist;
- qp_extra.activeWindows = activeWindows;
- qp_extra.groupClause = (gset_data
- ? (gset_data->rollups ? linitial_node(RollupData, gset_data->rollups)->groupClause : NIL)
- : parse->groupClause);
-
- /*
- * Generate the best unsorted and presorted paths for the scan/join
- * portion of this Query, ie the processing represented by the
- * FROM/WHERE clauses. (Note there may not be any presorted paths.)
- * We also generate (in standard_qp_callback) pathkey representations
- * of the query's sort clause, distinct clause, etc.
- */
- current_rel = query_planner(root, tlist,
- standard_qp_callback, &qp_extra);
-
/*
* Convert the query's result tlist into PathTarget format.
*
@@ -2015,6 +1998,38 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
scanjoin_targets_contain_srfs = NIL;
}
+ /*
+ * Save the various upper-rel PathTargets we just computed into
+ * root->upper_targets[]. It provides a convenient place for
+ * extensions to get at the info. Also the aggregate push-down feature
+ * uses the target of UPPERREL_GROUP_AGG for the final join instead of
+ * constructing the target on its own.
+ *
+ * For consistency, we save all the intermediate targets, even though
+ * some of the corresponding upperrels might not be needed for this
+ * query.
+ */
+ root->upper_targets[UPPERREL_FINAL] = final_target;
+ root->upper_targets[UPPERREL_WINDOW] = sort_input_target;
+ root->upper_targets[UPPERREL_GROUP_AGG] = grouping_target;
+
+ /* Set up data needed by standard_qp_callback */
+ qp_extra.tlist = tlist;
+ qp_extra.activeWindows = activeWindows;
+ qp_extra.groupClause = (gset_data
+ ? (gset_data->rollups ? linitial_node(RollupData, gset_data->rollups)->groupClause : NIL)
+ : parse->groupClause);
+
+ /*
+ * Generate the best unsorted and presorted paths for the scan/join
+ * portion of this Query, ie the processing represented by the
+ * FROM/WHERE clauses. (Note there may not be any presorted paths.)
+ * We also generate (in standard_qp_callback) pathkey representations
+ * of the query's sort clause, distinct clause, etc.
+ */
+ current_rel = query_planner(root, tlist,
+ standard_qp_callback, &qp_extra);
+
/* Apply scan/join target. */
scanjoin_target_same_exprs = list_length(scanjoin_targets) == 1
&& equal(scanjoin_target->exprs, current_rel->reltarget->exprs);
@@ -2024,17 +2039,6 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
scanjoin_target_same_exprs);
/*
- * Save the various upper-rel PathTargets we just computed into
- * root->upper_targets[]. The core code doesn't use this, but it
- * provides a convenient place for extensions to get at the info. For
- * consistency, we save all the intermediate targets, even though some
- * of the corresponding upperrels might not be needed for this query.
- */
- root->upper_targets[UPPERREL_FINAL] = final_target;
- root->upper_targets[UPPERREL_WINDOW] = sort_input_target;
- root->upper_targets[UPPERREL_GROUP_AGG] = grouping_target;
-
- /*
* If we have grouping and/or aggregation, consider ways to implement
* that. We build a new upperrel representing the output of this
* phase.
agg_pushdown_v8/prepare_3.patch
commit 67b9350e8bc9a6f13f15ca647e938da7cf0b73e9
Author: Antonin Houska <[email protected]>
Date: Fri Aug 31 16:10:56 2018 +0200
Introduced pg_operator(oprphys) column.
It references an operator that compares the physical (i.e. stored) values,
regardless their interpretation. If the value is zero, user should assume that
the corresponding operator is needed but not implemented yet. In contrast, if
zero value meant that that "physical operator" is not needed, the aggregation
push-down feature would not be guaranteed to work correctly for custom types.
This is a prototype, so only the operator we need for regression tests of the
aggregate push-down feature has the column set.
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index bba595ad1d..3158017eac 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -1338,6 +1338,30 @@ get_negator(Oid opno)
}
/*
+ * get_oprphys
+ *
+ * Returns the corresponding operator to compare physical values.
+ */
+Oid
+get_oprphys(Oid opno)
+{
+ HeapTuple tp;
+
+ tp = SearchSysCache1(OPEROID, ObjectIdGetDatum(opno));
+ if (HeapTupleIsValid(tp))
+ {
+ Form_pg_operator optup = (Form_pg_operator) GETSTRUCT(tp);
+ Oid result;
+
+ result = optup->oprphys;
+ ReleaseSysCache(tp);
+ return result;
+ }
+ else
+ return InvalidOid;
+}
+
+/*
* get_oprrest
*
* Returns procedure id for computing selectivity of an operator.
diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat
index d9b6bad614..e78f9adc95 100644
--- a/src/include/catalog/pg_operator.dat
+++ b/src/include/catalog/pg_operator.dat
@@ -91,8 +91,8 @@
{ oid => '96', oid_symbol => 'Int4EqualOperator', descr => 'equal',
oprname => '=', oprcanmerge => 't', oprcanhash => 't', oprleft => 'int4',
oprright => 'int4', oprresult => 'bool', oprcom => '=(int4,int4)',
- oprnegate => '<>(int4,int4)', oprcode => 'int4eq', oprrest => 'eqsel',
- oprjoin => 'eqjoinsel' },
+ oprnegate => '<>(int4,int4)', oprphys => '=(int4,int4)', oprcode => 'int4eq',
+ oprrest => 'eqsel', oprjoin => 'eqjoinsel' },
{ oid => '97', oid_symbol => 'Int4LessOperator', descr => 'less than',
oprname => '<', oprleft => 'int4', oprright => 'int4', oprresult => 'bool',
oprcom => '>(int4,int4)', oprnegate => '>=(int4,int4)', oprcode => 'int4lt',
diff --git a/src/include/catalog/pg_operator.h b/src/include/catalog/pg_operator.h
index 3212b21418..634163c027 100644
--- a/src/include/catalog/pg_operator.h
+++ b/src/include/catalog/pg_operator.h
@@ -64,6 +64,9 @@ CATALOG(pg_operator,2617,OperatorRelationId)
/* OID of negator oper, or 0 if none */
Oid oprnegate BKI_DEFAULT(0) BKI_LOOKUP(pg_operator);
+ /* OID of oper to act on physical args, or 0 if not implemented */
+ Oid oprphys BKI_DEFAULT(0) BKI_LOOKUP(pg_operator);
+
/* OID of underlying function */
regproc oprcode BKI_LOOKUP(pg_proc);
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index e55ea4035b..6fdd9cff1c 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -105,6 +105,7 @@ extern bool op_strict(Oid opno);
extern char op_volatile(Oid opno);
extern Oid get_commutator(Oid opno);
extern Oid get_negator(Oid opno);
+extern Oid get_oprphys(Oid opno);
extern RegProcedure get_oprrest(Oid opno);
extern RegProcedure get_oprjoin(Oid opno);
extern char *get_func_name(Oid funcid);
agg_pushdown_v8/agg_pushdown_simple.patch
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7c8220cf65..d5667979bf 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2201,8 +2201,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* relation.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2343,6 +2343,19 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5110,6 +5123,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b5af904c18..ea508b7696 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -1808,6 +1808,7 @@ _outPathInfo(StringInfo str, const Path *node)
WRITE_FLOAT_FIELD(startup_cost, "%.2f");
WRITE_FLOAT_FIELD(total_cost, "%.2f");
WRITE_NODE_FIELD(pathkeys);
+ WRITE_NODE_FIELD(uniquekeys);
}
/*
@@ -2297,6 +2298,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2344,6 +2346,8 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(cheapest_parameterized_paths);
WRITE_BITMAPSET_FIELD(direct_lateral_relids);
WRITE_BITMAPSET_FIELD(lateral_relids);
+ WRITE_NODE_FIELD(agg_info);
+ WRITE_NODE_FIELD(grouped);
WRITE_UINT_FIELD(relid);
WRITE_OID_FIELD(reltablespace);
WRITE_ENUM_FIELD(rtekind, RTEKind);
@@ -2521,6 +2525,22 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+/*
+ * Note we do NOT print plain_rel, else we'd be in infinite recursion.
+ */
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(input);
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+ WRITE_NODE_FIELD(uniquekeys);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2609,6 +2629,17 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4135,6 +4166,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4150,6 +4184,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 5db1688bf0..47d68a6607 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -58,6 +58,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -73,16 +74,17 @@ static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte, bool grouped);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte,
+ bool grouped);
static void set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, bool grouped);
static void create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel);
static void set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, bool grouped);
static void set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
@@ -118,7 +120,8 @@ static void set_namedtuplestore_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
-static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
+static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root,
+ List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
@@ -146,35 +149,17 @@ RelOptInfo *
make_one_rel(PlannerInfo *root, List *joinlist)
{
RelOptInfo *rel;
- Index rti;
/*
- * Construct the all_baserels Relids set.
+ * Mark base rels as to whether we care about fast-start plans. XXX We
+ * deliberately do not mark grouped rels --- see the comment on
+ * consider_startup in build_simple_rel().
*/
- root->all_baserels = NULL;
- for (rti = 1; rti < root->simple_rel_array_size; rti++)
- {
- RelOptInfo *brel = root->simple_rel_array[rti];
-
- /* there may be empty slots corresponding to non-baserel RTEs */
- if (brel == NULL)
- continue;
-
- Assert(brel->relid == rti); /* sanity check on array */
-
- /* ignore RTEs that are "other rels" */
- if (brel->reloptkind != RELOPT_BASEREL)
- continue;
-
- root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
- }
-
- /* Mark base rels as to whether we care about fast-start plans */
set_base_rel_consider_startup(root);
/*
- * Compute size estimates and consider_parallel flags for each base rel,
- * then generate access paths.
+ * Compute size estimates and consider_parallel flags for each plain and
+ * each grouped base rel, then generate access paths.
*/
set_base_rel_sizes(root);
set_base_rel_pathlists(root);
@@ -231,6 +216,19 @@ set_base_rel_consider_startup(PlannerInfo *root)
RelOptInfo *rel = find_base_rel(root, varno);
rel->consider_param_startup = true;
+
+ if (rel->grouped)
+ {
+ /*
+ * As for grouped relations, paths differ substantially by the
+ * AggStrategy. Paths that use AGG_HASHED should not be
+ * parameterized (because creation of hashtable would have to
+ * be repeated for different parameters) but paths using
+ * AGG_SORTED can be. The latter seems to justify considering
+ * the startup cost for grouped relation in general.
+ */
+ rel->grouped->consider_param_startup = true;
+ }
}
}
}
@@ -278,7 +276,9 @@ set_base_rel_sizes(PlannerInfo *root)
if (root->glob->parallelModeOK)
set_rel_consider_parallel(root, rel, rte);
- set_rel_size(root, rel, rti, rte);
+ set_rel_size(root, rel, rti, rte, false);
+ if (rel->grouped)
+ set_rel_size(root, rel, rti, rte, true);
}
}
@@ -297,7 +297,9 @@ set_base_rel_pathlists(PlannerInfo *root)
{
RelOptInfo *rel = root->simple_rel_array[rti];
- /* there may be empty slots corresponding to non-baserel RTEs */
+ /*
+ * there may be empty slots corresponding to non-baserel RTEs
+ */
if (rel == NULL)
continue;
@@ -307,7 +309,20 @@ set_base_rel_pathlists(PlannerInfo *root)
if (rel->reloptkind != RELOPT_BASEREL)
continue;
- set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti]);
+ set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti], false);
+
+ /*
+ * Create grouped paths for grouped relation if it exists.
+ */
+ if (rel->grouped)
+ {
+ Assert(rel->grouped->agg_info != NULL);
+ Assert(rel->grouped->grouped == NULL);
+
+ set_rel_pathlist(root, rel, rti,
+ root->simple_rte_array[rti],
+ true);
+ }
}
}
@@ -317,8 +332,14 @@ set_base_rel_pathlists(PlannerInfo *root)
*/
static void
set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, bool grouped)
{
+ /*
+ * build_simple_rel() should not have created rels that do not match this
+ * condition.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (rel->reloptkind == RELOPT_BASEREL &&
relation_excluded_by_constraints(root, rel, rte))
{
@@ -337,7 +358,13 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support grouped
+ * append relation.
+ */
+ Assert(!grouped);
set_append_rel_size(root, rel, rti, rte);
}
else
@@ -347,7 +374,13 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Grouped foreign table supported yet, see
+ * build_simple_rel().
+ */
+ Assert(!grouped);
set_foreign_size(root, rel, rte);
}
else if (rte->relkind == RELKIND_PARTITIONED_TABLE)
@@ -356,17 +389,22 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
* A partitioned table without any partitions is marked as
* a dummy rel.
*/
+ if (grouped)
+ rel = rel->grouped;
+
set_dummy_rel_pathlist(rel);
}
else if (rte->tablesample != NULL)
{
/* Sampled relation */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_tablesample_rel_size(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_size(root, rel, rte);
+ set_plain_rel_size(root, rel, rte, grouped);
}
break;
case RTE_SUBQUERY:
@@ -420,15 +458,29 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
*/
static void
set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, bool grouped)
{
+ RelOptInfo *rel_plain = rel; /* non-grouped relation */
+
+ /*
+ * add_grouped_base_rels_to_query() should not have created rels that do
+ * not match this condition.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (IS_DUMMY_REL(rel))
{
/* We already proved the relation empty, so nothing more to do */
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support grouped
+ * append relation.
+ */
+ Assert(!grouped);
set_append_rel_pathlist(root, rel, rti, rte);
}
else
@@ -439,17 +491,21 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
/* Foreign table */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_foreign_pathlist(root, rel, rte);
}
else if (rte->tablesample != NULL)
{
/* Sampled relation */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_tablesample_rel_pathlist(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_pathlist(root, rel, rte);
+ set_plain_rel_pathlist(root, rel, rte, grouped);
}
break;
case RTE_SUBQUERY:
@@ -479,6 +535,9 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
}
}
+ if (grouped)
+ rel = rel->grouped;
+
/*
* If this is a baserel, we should normally consider gathering any partial
* paths we may have created for it.
@@ -491,9 +550,13 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* Also, if this is the topmost scan/join rel (that is, the only baserel),
* we postpone this until the final scan/join targelist is available (see
* grouping_planner).
+ *
+ * Note on aggregation push-down: parallel paths are not supported until
+ * we implement the feature using 2-stage aggregation.
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON &&
+ !grouped)
generate_gather_paths(root, rel, false);
/*
@@ -504,6 +567,22 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
if (set_rel_pathlist_hook)
(*set_rel_pathlist_hook) (root, rel, rti, rte);
+ /*
+ * Get rid of the grouped relations which have no paths (and to which
+ * generate_gather_paths() won't add any).
+ */
+ if (grouped && rel->pathlist == NIL)
+ {
+ /*
+ * This grouped rel should not contain any partial paths.
+ */
+ Assert(rel->partial_pathlist == NIL);
+
+ pfree(rel_plain->grouped);
+ rel_plain->grouped = NULL;
+ return;
+ }
+
/* Now find the cheapest of the paths for this rel */
set_cheapest(rel);
@@ -517,8 +596,12 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* Set size estimates for a plain relation (no subquery, no inheritance)
*/
static void
-set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
+ bool grouped)
{
+ if (grouped)
+ rel = rel->grouped;
+
/*
* Test any partial indexes of rel for applicability. We must do this
* first since partial unique indexes can affect size estimates.
@@ -692,9 +775,24 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
* Build access paths for a plain relation (no subquery, no inheritance)
*/
static void
-set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
+ bool grouped)
{
Relids required_outer;
+ Path *seq_path;
+ RelOptInfo *rel_plain = rel;
+
+ if (grouped)
+ {
+ /*
+ * Do not apply AggPath if this the topmost join.
+ * create_grouping_paths() will do so anyway.
+ */
+ if (bms_equal(rel->relids, root->all_baserels))
+ return;
+
+ rel = rel->grouped;
+ }
/*
* We don't support pushing join clauses into the quals of a seqscan, but
@@ -703,18 +801,82 @@ set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
*/
required_outer = rel->lateral_relids;
- /* Consider sequential scan */
- add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
+ /* Consider sequential scan, both plain and grouped. */
+ seq_path = create_seqscan_path(root, rel_plain,
+ grouped ? rel->agg_info->input : NULL,
+ required_outer, 0);
+
+ /*
+ * Grouped path actually belongs to the grouped relation.
+ */
+ if (grouped)
+ seq_path->parent = rel;
+
+ /*
+ * It's probably not good idea to repeat hashed aggregation with different
+ * parameters, so check if there are no parameters.
+ */
+ if (!grouped)
+ add_path(rel, seq_path);
+
+ /*
+ * Do not waste cycles if there's no uniquekeys to be assigned to the
+ * AggPath.
+ */
+ else if (required_outer == NULL && rel->agg_info->uniquekeys != NIL)
+ {
+ /*
+ * Only AGG_HASHED is suitable here as it does not expect the input
+ * set to be sorted.
+ */
+ add_grouped_path(root, rel, seq_path, AGG_HASHED);
+ }
/* If appropriate, consider parallel sequential scan */
- if (rel->consider_parallel && required_outer == NULL)
- create_plain_partial_paths(root, rel);
+ if (rel->consider_parallel && required_outer == NULL && !grouped)
+ create_plain_partial_paths(root, rel_plain);
- /* Consider index scans */
+ /*
+ * Consider index scans, possibly including the grouped and grouped
+ * partial paths.
+ */
create_index_paths(root, rel);
/* Consider TID scans */
- create_tidscan_paths(root, rel);
+ /* TODO Regression test for these paths. */
+ create_tidscan_paths(root, rel, grouped);
+
+ /*
+ * uniquekeys should not be path-specific for plain relation, so create
+ * them once and assign to all the paths.
+ *
+ * create_grouped_path() sets uniquekeys for grouped paths because it
+ * seems the best place to initialize uniquekeys of AggPaths anyway
+ * (grouped path on the top of plain rel is essentially AggPath).
+ *
+ * uniquekeys of append relation are currently not supported, so only
+ * accept RELOPT_BASEREL.
+ *
+ * TODO indxpath.c spends extra effort to set uniquekeys of grouped path,
+ * i.e. it by-passes create_grouped_path(). Should uniquekeys of those be
+ * set here?
+ */
+ if (!grouped && rel->reloptkind == RELOPT_BASEREL)
+ {
+ List *uniquekeys = make_baserel_uniquekeys(root, rel);
+
+ if (uniquekeys && match_uniquekeys_to_groupkeys(root, uniquekeys))
+ {
+ ListCell *lc;
+
+ foreach(lc, rel->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ path->uniquekeys = uniquekeys;
+ }
+ }
+ }
}
/*
@@ -726,15 +888,62 @@ create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
- parallel_workers = compute_parallel_worker(rel, rel->pages, -1,
- max_parallel_workers_per_gather);
+ parallel_workers = compute_parallel_worker(rel, rel->pages, -1, max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
- add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
+ add_partial_path(rel, create_seqscan_path(root, rel, NULL, NULL,
+ parallel_workers));
+}
+
+/*
+ * Apply aggregation to a subpath and add the AggPath to the pathlist.
+ *
+ * The return value tells whether the path was added to the pathlist.
+ *
+ * XXX Pass the plain rel and fetch the grouped from rel->grouped? How about
+ * subroutines then?
+ */
+bool
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy)
+{
+ Path *agg_path;
+ RelAggInfo *agg_info;
+
+ Assert(IS_GROUPED_REL(rel));
+
+ agg_info = rel->agg_info;
+
+ /*
+ * Repeated creation of hash table does not sound like a good idea. Caller
+ * should avoid asking us to do so.
+ */
+ Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, subpath,
+ subpath->rows);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, subpath,
+ true,
+ subpath->rows);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ {
+ agg_path->uniquekeys = agg_info->uniquekeys;
+ add_path(rel, (Path *) agg_path);
+
+ return true;
+ }
+
+ return false;
}
/*
@@ -1172,7 +1381,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's size.
*/
- set_rel_size(root, childrel, childRTindex, childRTE);
+ set_rel_size(root, childrel, childRTindex, childRTE, false);
/*
* It is possible that constraint exclusion detected a contradiction
@@ -1316,7 +1525,7 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's access paths.
*/
- set_rel_pathlist(root, childrel, childRTindex, childRTE);
+ set_rel_pathlist(root, childrel, childRTindex, childRTE, false);
/*
* If child is dummy, ignore it.
@@ -2644,11 +2853,22 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
root->initial_rels = initial_rels;
if (join_search_hook)
- return (*join_search_hook) (root, levels_needed, initial_rels);
+ return (*join_search_hook) (root, levels_needed,
+ initial_rels);
else if (enable_geqo && levels_needed >= geqo_threshold)
+ {
+ /*
+ * TODO Teach GEQO about grouped relations. Don't forget that
+ * pathlist can be NIL before set_cheapest() gets called.
+ *
+ * This processing makes no difference betweend plain and grouped
+ * rels, so process them in the same loop.
+ */
return geqo(root, levels_needed, initial_rels);
+ }
else
- return standard_join_search(root, levels_needed, initial_rels);
+ return standard_join_search(root, levels_needed,
+ initial_rels);
}
}
@@ -2746,6 +2966,23 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ if (rel->grouped)
+ {
+ RelOptInfo *rel_grouped;
+
+ rel_grouped = rel->grouped;
+
+ Assert(rel_grouped->partial_pathlist == NIL);
+
+ if (rel_grouped->pathlist != NIL)
+ set_cheapest(rel_grouped);
+ else
+ {
+ pfree(rel_grouped);
+ rel->grouped = NULL;
+ }
+ }
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 7bf67a0529..075a6acbcd 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -88,6 +88,7 @@
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
@@ -2288,6 +2289,30 @@ cost_group(Path *path, PlannerInfo *root,
}
/*
+ * estimate_join_rows
+ * Set rows of a join path according to its parent relation or according
+ * to parameters. If agg_info is passed, the join path is grouped.
+ */
+static void
+estimate_join_rows(PlannerInfo *root, Path *path, RelAggInfo *agg_info)
+{
+ if (path->param_info)
+ {
+ double nrows;
+
+ path->rows = path->param_info->ppi_rows;
+ if (agg_info)
+ {
+ nrows = estimate_num_groups(root, agg_info->group_exprs,
+ path->rows, NULL);
+ path->rows = clamp_row_est(nrows);
+ }
+ }
+ else
+ path->rows = path->parent->rows;
+}
+
+/*
* initial_cost_nestloop
* Preliminary estimate of the cost of a nestloop join path.
*
@@ -2408,10 +2433,8 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->path.param_info)
- path->path.rows = path->path.param_info->ppi_rows;
- else
- path->path.rows = path->path.parent->rows;
+ estimate_join_rows(root, (Path *) path,
+ path->path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2854,10 +2877,8 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path,
+ path->jpath.path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3279,10 +3300,8 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path,
+ path->jpath.path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -4296,18 +4315,45 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
/* Should only be applied to base relations */
Assert(rel->relid > 0);
- nrows = rel->tuples *
- clauselist_selectivity(root,
- rel->baserestrictinfo,
- 0,
- JOIN_INNER,
- NULL);
+ if (!IS_GROUPED_REL(rel))
+ {
+ nrows = rel->tuples *
+ clauselist_selectivity(root,
+ rel->baserestrictinfo,
+ 0,
+ JOIN_INNER,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
- rel->rows = clamp_row_est(nrows);
+ /*
+ * Only set the estimate for grouped base rel if aggregation can take
+ * place. (Aggregation is the only way to build grouped base relation.)
+ */
+ else if (!bms_equal(rel->relids, root->all_baserels))
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ nrows = estimate_num_groups(root,
+ rel->agg_info->group_exprs,
+ rel->agg_info->plain_rel->rows,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
- set_rel_width(root, rel);
+ /*
+ * The grouped target should have the cost and width set immediately on
+ * creation, see create_rel_agg_info().
+ */
+ if (!IS_GROUPED_REL(rel))
+ set_rel_width(root, rel);
+#ifdef USE_ASSERT_CHECKING
+ else
+ Assert(rel->reltarget->width > 0);
+#endif
}
/*
@@ -5257,11 +5303,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5292,6 +5338,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index b22b36ec0e..25ad1a6202 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -65,7 +65,6 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
-
/*
* process_equivalence
* The given clause has a mergejoinable operator and can be applied without
@@ -2511,3 +2510,137 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Re-check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index f295558f76..2f1e36e528 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -32,6 +32,7 @@
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
@@ -76,7 +77,6 @@ typedef struct
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
-
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -272,8 +272,7 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
- get_index_paths(root, rel, index, &rclauseset,
- &bitindexpaths);
+ get_index_paths(root, rel, index, &rclauseset, &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
@@ -306,11 +305,18 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
}
/*
+ * It does not seem too efficient to aggregate the individual paths and
+ * then AND them together.
+ */
+ if (IS_GROUPED_REL(rel))
+ return;
+
+ /*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
- indexpaths = generate_bitmap_or_paths(root, rel,
- rel->baserestrictinfo, NIL);
+ indexpaths = generate_bitmap_or_paths(root, rel, rel->baserestrictinfo,
+ NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
@@ -715,7 +721,6 @@ bms_equal_any(Relids relids, List *relids_list)
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
@@ -746,6 +751,10 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* clauses only if the index AM supports them natively, and skip any such
* clauses for index columns after the first (so that we produce ordered
* paths if possible).
+ *
+ * These paths are good candidates for AGG_SORTED, so pass the output
+ * lists for this strategy. AGG_HASHED should be applied to paths with no
+ * pathkeys.
*/
indexpaths = build_index_paths(root, rel,
index, clauses,
@@ -758,6 +767,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
* that supports them, then try again including those clauses. This will
* produce paths with more selectivity but no ordering.
+ *
+ * As for the grouping paths, only AGG_HASHED is considered due to the
+ * missing ordering.
*/
if (skip_lower_saop)
{
@@ -799,6 +811,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* If there were ScalarArrayOpExpr clauses that the index can't handle
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
+ *
+ * As for grouping, only AGG_HASHED is possible here. Again, because
+ * there's no ordering.
*/
if (skip_nonnative_saop)
{
@@ -851,7 +866,7 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
- * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
+ * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -876,6 +891,8 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ AggPath *agg_path;
+ RelAggInfo *agg_info = rel->agg_info;
/*
* Check that index supports the desired scan type(s)
@@ -1029,6 +1046,8 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
* in the current clauses, OR the index ordering is potentially useful for
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
+ *
+ * This is where grouped path start to be considered.
*/
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
@@ -1046,7 +1065,53 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
outer_relids,
loop_count,
false);
- result = lappend(result, ipath);
+
+ if (!IS_GROUPED_REL(rel))
+ result = lappend(result, ipath);
+ else
+ {
+ /*
+ * Try to create the grouped paths if caller is interested in
+ * them.
+ *
+ * Do not waste cycles if there's no uniquekeys to be assigned to
+ * the AggPath.
+ */
+ if (useful_pathkeys != NIL && agg_info->uniquekeys != NIL)
+ {
+ agg_path = create_agg_sorted_path(root,
+ (Path *) ipath,
+ true,
+ ipath->path.rows);
+
+ if (agg_path != NULL)
+ {
+ agg_path->path.uniquekeys = agg_info->uniquekeys;
+ result = lappend(result, agg_path);
+ }
+ }
+
+ /*
+ * Hashed aggregation should not be parameterized: the cost of
+ * repeated creation of the hashtable (for different parameter
+ * values) is probably not worth.
+ *
+ * Do not waste cycles if there's no uniquekeys to be assigned to
+ * the AggPath.
+ */
+ if (outer_relids != NULL && agg_info->uniquekeys != NIL)
+ {
+ agg_path = create_agg_hashed_path(root,
+ (Path *) ipath,
+ ipath->path.rows);
+
+ if (agg_path != NULL)
+ {
+ agg_path->path.uniquekeys = agg_info->uniquekeys;
+ result = lappend(result, agg_path);
+ }
+ }
+ }
/*
* If appropriate, consider parallel index scan. We don't allow
@@ -1103,7 +1168,34 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
outer_relids,
loop_count,
false);
- result = lappend(result, ipath);
+
+ if (!IS_GROUPED_REL(rel))
+ result = lappend(result, ipath);
+
+ /*
+ * Do not waste cycles if there's no uniquekeys to be assigned to
+ * the AggPath.
+ */
+ else if (agg_info->uniquekeys != NIL)
+ {
+ /*
+ * As the input set ordering does not matter to AGG_HASHED,
+ * only AGG_SORTED makes sense here. (The AGG_HASHED path we'd
+ * create here should already exist.)
+ *
+ * pathkeys are new, so check them.
+ */
+ agg_path = create_agg_sorted_path(root,
+ (Path *) ipath,
+ true,
+ ipath->path.rows);
+
+ if (agg_path != NULL)
+ {
+ agg_path->path.uniquekeys = agg_info->uniquekeys;
+ result = lappend(result, agg_path);
+ }
+ }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
@@ -1127,7 +1219,10 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
- add_partial_path(rel, (Path *) ipath);
+ {
+ if (!IS_GROUPED_REL(rel))
+ add_partial_path(rel, (Path *) ipath);
+ }
else
pfree(ipath);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 642f951093..6eb385c01b 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -54,7 +54,8 @@ static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
- JoinType jointype, JoinPathExtraData *extra);
+ JoinType jointype, JoinPathExtraData *extra,
+ bool do_aggregate);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
@@ -67,10 +68,12 @@ static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
- Path *inner_cheapest_total);
+ Path *inner_cheapest_total,
+ bool do_aggregate);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
- JoinType jointype, JoinPathExtraData *extra);
+ JoinType jointype, JoinPathExtraData *extra,
+ bool do_aggregate);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
@@ -120,7 +123,8 @@ add_paths_to_joinrel(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist,
+ bool do_aggregate)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
@@ -143,6 +147,7 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.do_aggregate = do_aggregate;
/*
* See if the inner relation is provably unique for this outer rel.
@@ -278,7 +283,7 @@ add_paths_to_joinrel(PlannerInfo *root,
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
- jointype, &extra);
+ jointype, &extra, do_aggregate);
#ifdef NOT_USED
@@ -305,7 +310,7 @@ add_paths_to_joinrel(PlannerInfo *root,
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
- jointype, &extra);
+ jointype, &extra, do_aggregate);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
@@ -376,6 +381,7 @@ try_nestloop_path(PlannerInfo *root,
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ bool success = false;
/*
* Paths are parameterized by top-level parents, so run parameterization
@@ -422,10 +428,37 @@ try_nestloop_path(PlannerInfo *root,
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * If the join output should be aggregated, the precheck is skipped
+ * because it makes little sense to compare the new join path to existing,
+ * already aggregated paths. Since we don't have row count estimate yet,
+ * it's hard to involve AggPath in the precheck.
+ */
+ if ((!extra->do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ extra->do_aggregate)
{
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!extra->do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ /*
+ * If the join output is subject to aggregation, the path must
+ * generate aggregation input.
+ */
+ target = joinrel->agg_info->input;
+ parent_rel = joinrel->agg_info->plain_rel;
+ }
+
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
@@ -447,21 +480,76 @@ try_nestloop_path(PlannerInfo *root,
}
}
- add_path(joinrel, (Path *)
- create_nestloop_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer));
+ path = (Path *) create_nestloop_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer);
+ if (extra->do_aggregate)
+ {
+ /*
+ * Non-grouped rel had to be passed above so that row estimate
+ * reflects the input data for aggregation, but the join path
+ * actually belongs to joinrel as well.
+ */
+ path->parent = joinrel;
+ }
+
+ if (!extra->do_aggregate)
+ {
+ set_join_uniquekeys(root, path, outer_path, inner_path);
+
+ /*
+ * Grouped join which does not have the suitable uniquekeys is not
+ * worth further processing: if the target contains uniquekey not
+ * present in the query groupkeys, it won't disappear if we join
+ * any other relation to it.
+ */
+ if (path->uniquekeys != NIL || !IS_GROUPED_REL(joinrel))
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ }
+ else
+ {
+ /*
+ * Do not waste cycles if there's no uniquekeys to be assigned to
+ * the AggPath.
+ */
+ if (joinrel->agg_info->uniquekeys != NIL)
+ {
+ /*
+ * Try both AGG_HASHED and AGG_SORTED aggregation.
+ *
+ * AGG_HASHED should not be parameterized because we don't
+ * want to create the hashtable again for each set of
+ * parameters.
+ */
+ if (required_outer == NULL)
+ success = add_grouped_path(root, joinrel, path,
+ AGG_HASHED);
+
+ /*
+ * Don't try AGG_SORTED if create_grouped_path() would reject
+ * it anyway.
+ */
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root, joinrel, path, AGG_SORTED);
+ }
+ }
}
- else
+
+ if (!success)
{
- /* Waste no memory when we reject a path here */
+ /* Waste no memory when we reject path(s) here */
bms_free(required_outer);
}
}
@@ -538,6 +626,7 @@ try_partial_nestloop_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_nestloop_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -568,8 +657,9 @@ try_mergejoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool success = false;
- if (is_partial)
+ if (!IS_GROUPED_REL(joinrel) && is_partial)
{
try_partial_mergejoin_path(root,
joinrel,
@@ -617,26 +707,80 @@ try_mergejoin_path(PlannerInfo *root,
outersortkeys, innersortkeys,
extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!extra->do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ extra->do_aggregate)
{
- add_path(joinrel, (Path *)
- create_mergejoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer,
- mergeclauses,
- outersortkeys,
- innersortkeys));
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!extra->do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = joinrel->agg_info->input;
+ parent_rel = joinrel->agg_info->plain_rel;
+ }
+
+ path = (Path *) create_mergejoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer,
+ mergeclauses,
+ outersortkeys,
+ innersortkeys);
+ /* See comments in try_nestloop_path() */
+ if (extra->do_aggregate)
+ path->parent = joinrel;
+
+ /* Regarding uniquekeys, see comments in try_nestloop_path() */
+ if (!extra->do_aggregate)
+ {
+ set_join_uniquekeys(root, path, outer_path, inner_path);
+
+ if (path->uniquekeys != NIL || !IS_GROUPED_REL(joinrel))
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ }
+ else
+ {
+ if (joinrel->agg_info->uniquekeys != NIL)
+ {
+ if (required_outer == NULL)
+ success = add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_HASHED);
+
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_SORTED);
+ }
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -700,6 +844,7 @@ try_partial_mergejoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_mergejoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -729,6 +874,7 @@ try_hashjoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool success = false;
/*
* Check to see if proposed path is still parameterized, and reject if the
@@ -745,30 +891,85 @@ try_hashjoin_path(PlannerInfo *root,
}
/*
+ * Parameterized execution of grouped path would mean repeated hashing of
+ * the output of the hashjoin output, so forget about AGG_HASHED if there
+ * are any parameters. And AGG_SORTED makes no sense because the hash join
+ * output is not sorted.
+ */
+ if (required_outer && IS_GROUPED_REL(joinrel))
+ return;
+
+ /*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- NIL, required_outer))
+
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!extra->do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ NIL, required_outer)) ||
+ extra->do_aggregate)
{
- add_path(joinrel, (Path *)
- create_hashjoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- false, /* parallel_hash */
- extra->restrictlist,
- required_outer,
- hashclauses));
+ PathTarget *target;
+ Path *path = NULL;
+ RelOptInfo *parent_rel;
+
+ if (!extra->do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = joinrel->agg_info->input;
+ parent_rel = joinrel->agg_info->plain_rel;
+ }
+
+ path = (Path *) create_hashjoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ false, /* parallel_hash */
+ extra->restrictlist,
+ required_outer,
+ hashclauses);
+ /* See comments in try_nestloop_path() */
+ if (extra->do_aggregate)
+ path->parent = joinrel;
+
+ /* Regarding uniquekeys, see comments in try_nestloop_path() */
+ if (!extra->do_aggregate)
+ {
+ set_join_uniquekeys(root, path, outer_path, inner_path);
+
+ if (path->uniquekeys != NIL || !IS_GROUPED_REL(joinrel))
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ }
+ else
+ {
+ /*
+ * As the hashjoin path is not sorted, only try AGG_HASHED.
+ */
+ if (joinrel->agg_info->uniquekeys != NIL &&
+ add_grouped_path(root, joinrel, path, AGG_HASHED))
+ success = true;
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -824,6 +1025,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_hashjoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -1051,7 +1253,7 @@ sort_inner_and_outer(PlannerInfo *root,
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
- if (cheapest_partial_outer && cheapest_safe_inner)
+ if (!IS_GROUPED_REL(joinrel) && cheapest_partial_outer && cheapest_safe_inner)
try_partial_mergejoin_path(root,
joinrel,
cheapest_partial_outer,
@@ -1333,7 +1535,8 @@ match_unsorted_outer(PlannerInfo *root,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
- JoinPathExtraData *extra)
+ JoinPathExtraData *extra,
+ bool do_aggregate)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
@@ -1516,7 +1719,8 @@ match_unsorted_outer(PlannerInfo *root,
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
- if (joinrel->consider_parallel &&
+ if (!IS_GROUPED_REL(joinrel) &&
+ joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
@@ -1545,7 +1749,8 @@ match_unsorted_outer(PlannerInfo *root,
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
- inner_cheapest_total);
+ inner_cheapest_total,
+ do_aggregate);
}
}
@@ -1568,7 +1773,8 @@ consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
- Path *inner_cheapest_total)
+ Path *inner_cheapest_total,
+ bool do_aggregate)
{
ListCell *lc1;
@@ -1679,7 +1885,8 @@ hash_inner_and_outer(PlannerInfo *root,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
- JoinPathExtraData *extra)
+ JoinPathExtraData *extra,
+ bool do_aggregate)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index d3d21fed5d..d165bbc246 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -16,13 +16,16 @@
#include "miscadmin.h"
#include "optimizer/clauses.h"
+#include "optimizer/cost.h"
#include "optimizer/joininfo.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/prep.h"
+#include "optimizer/tlist.h"
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -31,15 +34,24 @@ static void make_rels_by_clause_joins(PlannerInfo *root,
static void make_rels_by_clauseless_joins(PlannerInfo *root,
RelOptInfo *old_rel,
ListCell *other_rels);
+static void set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, bool do_aggregate);
+static void make_join_rel_common_grouped(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, bool do_aggregate);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist);
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ bool do_aggregate);
static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
@@ -47,7 +59,6 @@ static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
-
/*
* join_search_one_level
* Consider ways to produce join relations containing exactly 'level'
@@ -322,6 +333,49 @@ make_rels_by_clauseless_joins(PlannerInfo *root,
}
}
+/*
+ * Set joinrel's reltarget according to agg_info and estimate the number of
+ * rows.
+ */
+static void
+set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info)
+{
+ Assert(agg_info != NULL);
+
+ /*
+ * build_join_rel() does not create the target for grouped relation.
+ */
+ Assert(joinrel->reltarget == NULL);
+ Assert(joinrel->agg_info == NULL);
+
+ joinrel->reltarget = agg_info->target;
+
+ /*
+ * The rest of agg_info will be needed at aggregation time.
+ */
+ joinrel->agg_info = agg_info;
+
+ /*
+ * Now that we have the target, compute the estimates.
+ */
+ set_joinrel_size_estimates(root, joinrel, rel1, rel2, sjinfo,
+ restrictlist);
+
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied to
+ * one of the input rels. Is this worth extra effort, e.g. maintaining a
+ * separate RelOptInfo for each case (one difficulty that would introduce
+ * is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, joinrel->agg_info->group_exprs,
+ joinrel->rows, NULL);
+}
/*
* join_is_legal
@@ -651,21 +705,29 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
+ * make_join_rel_common
* Find or create a join RelOptInfo that represents the join of
* the two given rels, and add to it path information for paths
* created with the two rels as outer and inner rel.
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation
+ * should not be grouped.
+ *
+ * 'do_aggregate' tells that two non-grouped rels should be grouped and
+ * aggregation should be applied to all their paths.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. NULL is also returned if caller is interested in a
+ * grouped relation but there's no useful grouped input relation.
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, bool do_aggregate)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -673,10 +735,14 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
SpecialJoinInfo sjinfo_data;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
+ /* do_aggregate implies the output to be grouped. */
+ Assert(!do_aggregate || grouped);
+
/* Construct Relids set that identifies the joinrel. */
joinrelids = bms_union(rel1->relids, rel2->relids);
@@ -726,7 +792,37 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, false);
+
+ if (grouped)
+ {
+ /*
+ * Make sure there's a grouped join relation.
+ */
+ if (joinrel->grouped == NULL)
+ joinrel->grouped = build_join_rel(root,
+ joinrelids,
+ rel1,
+ rel2,
+ sjinfo,
+ &restrictlist,
+ true);
+
+ /*
+ * The grouped join is what we need to return.
+ */
+ joinrel = joinrel->grouped;
+
+
+ /*
+ * Make sure the grouped joinrel has reltarget initialized. Caller
+ * should supply the target for group relation, so build_join_rel()
+ * should have omitted its creation.
+ */
+ if (joinrel->reltarget == NULL)
+ set_grouped_joinrel_target(root, joinrel, rel1, rel2, sjinfo,
+ restrictlist, agg_info);
+ }
/*
* If we've already proven this join is empty, we needn't consider any
@@ -738,15 +834,161 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return joinrel;
}
- /* Add paths to the join relation. */
+ /*
+ * Do not apply the join -> aggregate strategy if this the topmost join.
+ * create_grouping_paths() will do so anyway.
+ */
+ if (do_aggregate && bms_equal(joinrel->relids, root->all_baserels))
+ {
+ /*
+ * The aggregation input target should not have been initialized in
+ * this case.
+ */
+ Assert(agg_info->input == NULL);
+ return joinrel;
+ }
+
+ /*
+ * Add paths to the join relation.
+ */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
bms_free(joinrelids);
return joinrel;
}
+static void
+make_join_rel_common_grouped(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, bool do_aggregate)
+{
+ RelOptInfo *rel1_grouped = NULL;
+ RelOptInfo *rel2_grouped = NULL;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /*
+ * Retrieve the grouped relations.
+ *
+ * Dummy rel indicates join relation able to generate grouped paths as
+ * such (i.e. it has valid agg_info), but for which the path actually
+ * could not be created (e.g. only AGG_HASHED strategy was possible but
+ * work_mem was not sufficient for hash table).
+ */
+ if (rel1->grouped)
+ rel1_grouped = rel1->grouped;
+ if (rel2->grouped)
+ rel2_grouped = rel2->grouped;
+
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /*
+ * Nothing else to do?
+ */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info,
+ do_aggregate);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info,
+ do_aggregate);
+}
+
+/*
+ * Front-end to make_join_rel_common(). Generates plain (non-grouped) join and
+ * then uses all the possible strategies to generate the grouped one.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ Relids joinrelids;
+ RelAggInfo *agg_info;
+ RelOptInfo *joinrel;
+ RelOptInfo *result;
+
+ /* 1) form the plain join. */
+ result = make_join_rel_common(root, rel1, rel2, NULL, false);
+
+ if (result == NULL)
+ return result;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return result;
+
+ /*
+ * If the same joinrel was already formed, just with the base rels divided
+ * between rel1 and rel2 in a different way, we might already have the
+ * matching agg_info.
+ */
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_join_rel(root, joinrelids);
+
+ /*
+ * At the moment we know that non-grouped join exists, so it should have
+ * been fetched.
+ */
+ Assert(joinrel != NULL);
+
+ if (joinrel->grouped != NULL)
+ {
+ Assert(IS_GROUPED_REL(joinrel->grouped));
+
+ agg_info = joinrel->grouped->agg_info;
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, result);
+ }
+
+ /*
+ * Cannot we build grouped join?
+ */
+ if (agg_info == NULL)
+ return result;
+
+ /*
+ * 2) join two plain rels and aggregate the join paths.
+ */
+ result->grouped = make_join_rel_common(root, rel1, rel2, agg_info, true);
+ Assert(result->grouped != NULL);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_common_grouped(root, rel1, rel2, agg_info, false);
+
+ return result;
+}
+
/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
@@ -757,7 +999,8 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist)
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ bool do_aggregate)
{
/*
* Consider paths using each rel as both outer and inner. Depending on
@@ -788,10 +1031,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
@@ -805,10 +1048,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
@@ -819,10 +1062,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
/*
* If there are join quals that aren't mergeable or hashable, we
@@ -855,7 +1098,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
}
/*
@@ -878,10 +1121,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
}
break;
case JOIN_ANTI:
@@ -896,7 +1139,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
- restrictlist);
+ restrictlist, do_aggregate);
break;
default:
/* other values not expected here */
@@ -904,8 +1147,20 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
}
- /* Apply partitionwise join technique, if possible. */
- try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
+ /*
+ * The aggregate push-down feature currently does not support
+ * partition-wise aggregation. It should be fixed soon.
+ *
+ * Note: As for AGGSPLIT_SIMPLE strategy, we will only allow
+ * partition-wise aggregation if no group can be emitted by multiple
+ * partitions.
+ */
+ if (IS_GROUPED_REL(joinrel))
+ return;
+
+ /* Apply partition-wise join technique, if possible. */
+ try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
}
@@ -1413,7 +1668,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
- child_restrictlist);
+ child_restrictlist, false);
}
}
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
index 3bb5b8def6..b04aa357c8 100644
--- a/src/backend/optimizer/path/tidpath.c
+++ b/src/backend/optimizer/path/tidpath.c
@@ -250,10 +250,11 @@ TidQualFromBaseRestrictinfo(RelOptInfo *rel)
* Candidate paths are added to the rel's pathlist (using add_path).
*/
void
-create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
+create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Relids required_outer;
List *tidquals;
+ Path *tidpath;
/*
* We don't support pushing join clauses into the quals of a tidscan, but
@@ -263,8 +264,25 @@ create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
required_outer = rel->lateral_relids;
tidquals = TidQualFromBaseRestrictinfo(rel);
+ if (!tidquals)
+ return;
- if (tidquals)
- add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
- required_outer));
+ tidpath = (Path *) create_tidscan_path(root, rel, tidquals,
+ required_outer);
+
+ if (!grouped)
+ add_path(rel, tidpath);
+
+ /*
+ * Do not waste cycles if there's no uniquekeys to be assigned to the
+ * AggPath.
+ */
+ else if (required_outer == NULL && rel->agg_info->uniquekeys != NIL)
+ {
+ /*
+ * Only AGG_HASHED is suitable here as it does not expect the input
+ * set to be sorted.
+ */
+ add_grouped_path(root, rel, tidpath, AGG_HASHED);
+ }
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index ae41c9efa0..96c910732c 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -824,6 +824,12 @@ use_physical_tlist(PlannerInfo *root, Path *path, int flags)
return false;
/*
+ * Grouped relation's target list contains Aggrefs.
+ */
+ if (IS_GROUPED_REL(rel))
+ return false;
+
+ /*
* If a bitmap scan's tlist is empty, keep it as-is. This may allow the
* executor to skip heap page fetches, and in any case, the benefit of
* using a physical tlist instead would be minimal.
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 01335db511..aa6c76aedc 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "catalog/pg_class.h"
#include "nodes/nodeFuncs.h"
@@ -27,6 +28,7 @@
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
@@ -46,6 +48,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -96,10 +100,9 @@ static void check_hashjoinable(RestrictInfo *restrictinfo);
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
- * every non-join RTE that is used in the query. Therefore, this routine
- * is the only place that should call build_simple_rel with reloptkind
- * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
- * "other rel" RelOptInfos for the members of any appendrels we find here.)
+ * every non-grouped non-join RTE that is used in the query. (Note:
+ * build_simple_rel recurses internally to build "other rel" RelOptInfos for
+ * the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
@@ -241,6 +244,331 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression and setup RelOptInfo for each base or
+ * 'other' relation that can product grouped paths.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+add_grouped_base_rels_to_query(PlannerInfo *root)
+{
+ int i;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /*
+ * TODO Consider if this is a real limitation.
+ */
+ if (root->parse->hasWindowFuncs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+ RangeTblEntry *rte;
+ RelAggInfo *agg_info;
+
+ /* NULL should mean a join relation. */
+ if (rel == NULL)
+ continue;
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ continue;;
+
+ /*
+ * Grouped append relation, see set_rel_pathlist().
+ */
+ if (rte->inh)
+ continue;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel->reloptkind != RELOPT_BASEREL)
+ continue;
+
+ /*
+ * Retrieve the information we need for aggregation of the rel
+ * contents.
+ */
+ Assert(rel->agg_info == NULL);
+ agg_info = create_rel_agg_info(root, rel);
+ if (agg_info == NULL)
+ continue;
+
+ /*
+ * Create the grouped counterpart of "rel".
+ */
+ Assert(rel->grouped == NULL);
+ rel->grouped = build_simple_grouped_rel(root, rel);
+
+ /*
+ * Assign it the aggregation-specific info.
+ *
+ * The aggregation paths will get their input target from agg_info, so
+ * store it too.
+ */
+ rel->grouped->reltarget = agg_info->target;
+ rel->grouped->agg_info = agg_info;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ /*
+ * If the aggregation is pushed down, the equality operator must
+ * compare the physical (stored) values of the grouping key so that no
+ * information is lost.
+ *
+ * For example, 0.0 and 0.00 are logically equal values of the numeric
+ * data type but they are physically different. The pushed-down
+ * aggregation must not put them into the same group because that way
+ * scale() function in the WHERE or JOIN/ON expressions (now above the
+ * aggregate) would receive different input data than they do without
+ * aggregation push-down.
+ *
+ * The aggregate push-down feature currently supports only 1-stage
+ * (AGGSPLIT_SIMPLE) aggregation, so we only accept operators for
+ * which logical equality is the same like physical equality.
+ *
+ * Once the we support 2-stage aggregation, we can use different
+ * operators: the physical for AGGSPLIT_INITIAL_SERIAL and the logical
+ * for AGGSPLIT_FINAL_DESERIAL.
+ */
+ if (get_oprphys(sgClause->eqop) != sgClause->eqop)
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index b05adc70c4..82b8a514d0 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -66,6 +66,8 @@ query_planner(PlannerInfo *root, List *tlist,
*/
if (parse->jointree->fromlist == NIL)
{
+ RelOptInfo *final_rel;
+
/* We need a dummy joinrel to describe the empty set of baserels */
final_rel = build_empty_join_rel(root);
@@ -114,6 +116,7 @@ query_planner(PlannerInfo *root, List *tlist,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -199,6 +202,11 @@ query_planner(PlannerInfo *root, List *tlist,
joinlist = remove_useless_joins(root, joinlist);
/*
+ * No rels should disappear now, so we can initialize all_baserels.
+ */
+ setup_all_baserels(root);
+
+ /*
* Also, reduce any semijoins with unique inner rels to plain inner joins.
* Likewise, this can't be done until now for lack of needed info.
*/
@@ -232,6 +240,16 @@ query_planner(PlannerInfo *root, List *tlist,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup root->grouped_var_list
+ * and create RelOptInfo for base relations capable to do the grouping.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ add_grouped_base_rels_to_query(root);
+
+ /*
* We should now have size estimates for every actual table involved in
* the query, and we also know which if any have been deleted from the
* query by join removal; so we can compute total_table_pages.
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index a5fbf5f8af..25c5746a05 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -3693,6 +3693,7 @@ create_grouping_paths(PlannerInfo *root,
{
int flags = 0;
GroupPathExtraData extra;
+ List *agg_pushdown_paths = NIL;
/*
* Determine whether it's possible to perform sort-based
@@ -3760,6 +3761,30 @@ create_grouping_paths(PlannerInfo *root,
create_ordinary_grouping_paths(root, input_rel, grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
+
+ /*
+ * Process paths generated by aggregation push-down feature.
+ */
+ if (input_rel->grouped)
+ {
+ RelOptInfo *agg_pushdown_rel;
+ ListCell *lc;
+
+ agg_pushdown_rel = input_rel->grouped;
+ agg_pushdown_paths = agg_pushdown_rel->pathlist;
+
+ /*
+ * See create_grouped_path().
+ */
+ Assert(agg_pushdown_rel->partial_pathlist == NIL);
+
+ foreach(lc, agg_pushdown_paths)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(grouped_rel, path);
+ }
+ }
}
set_cheapest(grouped_rel);
@@ -3965,11 +3990,11 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
bool force_rel_creation;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregation push-down took place, force creation of a
+ * partially_grouped_rel so we can add the related paths to it.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4002,10 +4027,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -5974,7 +6003,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
comparisonCost = 2.0 * (indexExprCost.startup + indexExprCost.per_tuple);
/* Estimate the cost of seq scan + sort */
- seqScanPath = create_seqscan_path(root, rel, NULL, 0);
+ seqScanPath = create_seqscan_path(root, rel, NULL, NULL, 0);
cost_sort(&seqScanAndSortPath, root, NIL,
seqScanPath->total_cost, rel->tuples, rel->reltarget->width,
comparisonCost, maintenance_work_mem, -1.0);
@@ -7090,6 +7119,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
if (partially_grouped_rel && partial_grouping_valid)
{
Assert(partially_grouped_live_children != NIL);
+ Assert(partially_grouped_rel->agg_info == NULL);
add_paths_to_append_rel(root, partially_grouped_rel,
partially_grouped_live_children);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index f66f39d8c6..1412ec7bf0 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2287,6 +2287,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_SIMPLE)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c5aaaf5c22..7667db54c5 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -17,6 +17,8 @@
#include <math.h>
#include "miscadmin.h"
+#include "access/sysattr.h"
+#include "catalog/pg_constraint.h"
#include "foreign/fdwapi.h"
#include "nodes/extensible.h"
#include "nodes/nodeFuncs.h"
@@ -27,6 +29,7 @@
#include "optimizer/planmain.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+/* TODO Remove this if create_grouped_path ends up in another module. */
#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
@@ -56,7 +59,8 @@ static int append_startup_cost_compare(const void *a, const void *b);
static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
-
+static EquivalenceClass *get_uniquekey_for_expr(PlannerInfo *root,
+ Expr *expr);
/*****************************************************************************
* MISC. PATH UTILITIES
@@ -951,14 +955,14 @@ add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
* pathnode.
*/
Path *
-create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
+create_seqscan_path(PlannerInfo *root, RelOptInfo *rel, PathTarget *target,
Relids required_outer, int parallel_workers)
{
Path *pathnode = makeNode(Path);
pathnode->pathtype = T_SeqScan;
pathnode->parent = rel;
- pathnode->pathtarget = rel->reltarget;
+ pathnode->pathtarget = target ? target : rel->reltarget;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
@@ -1041,7 +1045,11 @@ create_index_path(PlannerInfo *root,
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
pathnode->path.parent = rel;
- pathnode->path.pathtarget = rel->reltarget;
+ /* For grouped relation only generate the aggregation input. */
+ if (!IS_GROUPED_REL(rel))
+ pathnode->path.pathtarget = rel->reltarget;
+ else
+ pathnode->path.pathtarget = rel->agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
@@ -1192,7 +1200,11 @@ create_tidscan_path(PlannerInfo *root, RelOptInfo *rel, List *tidquals,
pathnode->path.pathtype = T_TidScan;
pathnode->path.parent = rel;
- pathnode->path.pathtarget = rel->reltarget;
+ /* For grouped relation only generate the aggregation input. */
+ if (!IS_GROUPED_REL(rel))
+ pathnode->path.pathtarget = rel->reltarget;
+ else
+ pathnode->path.pathtarget = rel->agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
@@ -1229,9 +1241,11 @@ create_append_path(PlannerInfo *root,
Assert(!parallel_aware || parallel_workers > 0);
pathnode->path.pathtype = T_Append;
- pathnode->path.parent = rel;
+
pathnode->path.pathtarget = rel->reltarget;
+ pathnode->path.parent = rel;
+
/*
* When generating an Append path for a partitioned table, there may be
* parameters that are useful so we can eliminate certain partitions
@@ -1341,7 +1355,8 @@ append_startup_cost_compare(const void *a, const void *b)
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
- * pathnode.
+ * pathnode. target can be supplied by caller. If NULL is passed, the field
+ * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
@@ -1495,6 +1510,7 @@ create_material_path(RelOptInfo *rel, Path *subpath)
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = subpath->pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
@@ -1528,7 +1544,9 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
MemoryContext oldcontext;
int numCols;
- /* Caller made a mistake if subpath isn't cheapest_total ... */
+ /*
+ * Caller made a mistake if subpath isn't cheapest_total.
+ */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
@@ -2149,6 +2167,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
* relations.
*
* 'joinrel' is the join relation.
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_nestloop
* 'extra' contains various information about the join
@@ -2163,6 +2182,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
NestPath *
create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2203,7 +2223,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
- pathnode->path.pathtarget = joinrel->reltarget;
+ pathnode->path.pathtarget = target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2235,6 +2255,7 @@ create_nestloop_path(PlannerInfo *root,
* two relations
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_mergejoin
* 'extra' contains various information about the join
@@ -2251,6 +2272,7 @@ create_nestloop_path(PlannerInfo *root,
MergePath *
create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2267,7 +2289,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2303,6 +2325,7 @@ create_mergejoin_path(PlannerInfo *root,
* Creates a pathnode corresponding to a hash join between two relations.
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_hashjoin
* 'extra' contains various information about the join
@@ -2317,6 +2340,7 @@ create_mergejoin_path(PlannerInfo *root,
HashPath *
create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2331,7 +2355,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2413,8 +2437,8 @@ create_projection_path(PlannerInfo *root,
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
- if (is_projection_capable_path(subpath) ||
- equal(oldtarget->exprs, target->exprs))
+ if ((is_projection_capable_path(subpath) ||
+ equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
@@ -2647,6 +2671,7 @@ create_sort_path(PlannerInfo *root,
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
pathnode->path.pathkeys = pathkeys;
+ pathnode->path.uniquekeys = subpath->uniquekeys;
pathnode->subpath = subpath;
@@ -2799,8 +2824,7 @@ create_agg_path(PlannerInfo *root,
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
@@ -2833,6 +2857,179 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * check_pathkeys can be passed FALSE if the function was already called for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool check_pathkeys, double input_rows)
+{
+ RelOptInfo *rel;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ Node *qual = NULL;
+ AggPath *result = NULL;
+ RelAggInfo *agg_info;
+
+ rel = subpath->parent;
+ agg_info = rel->agg_info;
+ Assert(agg_info != NULL);
+
+ aggsplit = AGGSPLIT_SIMPLE;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ if (check_pathkeys)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ if (root->parse->havingQual)
+ {
+ qual = root->parse->havingQual;
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+ }
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ Assert(agg_info->group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ (List *) qual, &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ double input_rows)
+{
+ RelOptInfo *rel;
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ Node *qual = NULL;
+ AggPath *result = NULL;
+ RelAggInfo *agg_info;
+
+ rel = subpath->parent;
+ agg_info = rel->agg_info;
+ Assert(agg_info != NULL);
+
+ aggsplit = AGGSPLIT_SIMPLE;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ if (parse->havingQual)
+ {
+ qual = parse->havingQual;
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+ }
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the aggregation path.
+ */
+ Assert(agg_info->group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ (List *) qual,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
@@ -3519,7 +3716,7 @@ reparameterize_path(PlannerInfo *root, Path *path,
switch (path->pathtype)
{
case T_SeqScan:
- return create_seqscan_path(root, rel, required_outer, 0);
+ return create_seqscan_path(root, rel, NULL, required_outer, 0);
case T_SampleScan:
return (Path *) create_samplescan_path(root, rel, required_outer);
case T_IndexScan:
@@ -3956,3 +4153,353 @@ reparameterize_pathlist_by_child(PlannerInfo *root,
return result;
}
+
+/*
+ * Construct the common uniquekeys produced by paths of base relation.
+ *
+ * If special cases like UniquePath are implemented someday, they'll have to
+ * be implemented separate.
+ */
+List *
+make_baserel_uniquekeys(PlannerInfo *root, RelOptInfo *rel)
+{
+ RangeTblEntry *rte;
+ Bitmapset *pkattnos;
+ Oid constraintOid;
+ List *pk_vars = NIL;
+ int pkattno;
+ AttrNumber attno;
+ ListCell *l;
+ List *result = NIL;
+
+ /*
+ * The unique keys are not interesting if there's no chance to push
+ * aggregation down to base relations.
+ */
+ if (root->grouped_var_list == NIL)
+ return NIL;
+
+ /*
+ * Called on the correct reloptkind?
+ */
+ Assert(rel->reloptkind == RELOPT_BASEREL);
+
+ rte = root->simple_rte_array[rel->relid];
+
+ /*
+ * uniquekeys are currently not supported for inheritance parent relation.
+ */
+ if (rte->inh)
+ return NIL;
+
+ /*
+ * Check if the rel emits all attributes of the PK. While doing so,
+ * collect the vars for further processing.
+ *
+ * XXX Currently we only honor primary key. As for the relation unique
+ * constraint, we'd have to pay special attention to nullable columns, and
+ * also we'd have to be ready to accept multiple uniquekeys lists per
+ * path.
+ */
+ pkattnos = get_primary_key_attnos(rte->relid, false, &constraintOid);
+ if (pkattnos == NULL)
+ return NIL;
+
+ while ((pkattno = bms_first_member(pkattnos)) >= 0)
+ {
+ Var *pk_var = NULL;
+
+ attno = pkattno + FirstLowInvalidHeapAttributeNumber;
+
+ foreach(l, rel->reltarget->exprs)
+ {
+ if (IsA(lfirst(l), Var))
+ {
+ pk_var = lfirst_node(Var, l);
+
+ /*
+ * XXX It's mentioned elsewhere in the planner code that varno
+ * is not guaranteed to be equal to relid due to lateral
+ * references. So test it as well.
+ */
+ if (pk_var->varno == rel->relid && pk_var->varattno == attno)
+ break;
+ }
+ }
+ if (l)
+ pk_vars = lappend(pk_vars, pk_var);
+ else
+ {
+ /*
+ * At least one PK attribute not emitted by the relation.
+ */
+ list_free(pk_vars);
+ return NIL;
+ }
+ }
+
+ /*
+ * Try to find the EC for each PK attribute.
+ */
+ foreach(l, pk_vars)
+ {
+ EquivalenceClass *uniquekey;
+
+ uniquekey = get_uniquekey_for_expr(root,
+ (Expr *) lfirst_node(Var, l));
+
+ if (uniquekey)
+ {
+ /*
+ * Avoid adding duplicate values to uniquekeys (the duplicates
+ * shouldn't break matchingw to the grouping expression, but would
+ * make it less efficient).
+ */
+ result = list_append_unique_ptr(result, uniquekey);
+ }
+ else
+ {
+ list_free(pk_vars);
+ list_free(result);
+ return NIL;
+ }
+ }
+
+ list_free(pk_vars);
+ return result;
+}
+
+/*
+ * Construct the uniquekeys for join path.
+ *
+ * This function should only be applied to join of two non-grouped paths or to
+ * join of a grouped path to non-grouped one. In contrast, create_grouped_path
+ * takes care of cases where two non-grouped relations are joined and the
+ * result is aggregated.
+ *
+ * Returns NIL if uniquekeys could not be built.
+ */
+void
+set_join_uniquekeys(PlannerInfo *root, Path *joinpath, Path *outer_path,
+ Path *inner_path)
+{
+ RelOptInfo *joinrel = joinpath->parent;
+ ListCell *l;
+ EquivalenceClass *uniquekey;
+ List *result = NIL;
+
+ /* The function should not be called more than once on the same path. */
+ Assert(joinpath->uniquekeys == NIL);
+
+ /*
+ * The unique keys are not interesting if there's no chance to push
+ * aggregation down to join relations.
+ */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * uniquekeys are currently not supported for inheritance parent relation.
+ */
+ if (joinrel->reloptkind == RELOPT_OTHER_JOINREL)
+ return;
+
+ /*
+ * If one input path has no uniquekeys, it can duplicate rows of the
+ * other.
+ */
+ if (outer_path->uniquekeys == NIL || inner_path->uniquekeys == NIL)
+ return;
+
+ /*
+ * To form uniquekeys of a join we only need to combine unique keys of the
+ * input paths. Note that we should not need to care about outer joins
+ * because there should be no nullable expressions in the input
+ * uniquekeys.
+ */
+ foreach(l, outer_path->uniquekeys)
+ {
+ uniquekey = lfirst_node(EquivalenceClass, l);
+
+ /*
+ * It's likely for a join to contain multiple members of the same EC.
+ * We do not need the duplicate keys for the evaluation below, so
+ * eliminate them right away.
+ */
+ result = list_append_unique_ptr(result, uniquekey);
+ }
+ foreach(l, inner_path->uniquekeys)
+ {
+ uniquekey = lfirst_node(EquivalenceClass, l);
+ result = list_append_unique_ptr(result, uniquekey);
+ }
+
+ /*
+ * Only return the keys if they are useful.
+ */
+ if (!match_uniquekeys_to_groupkeys(root, result))
+ {
+ list_free(result);
+ result = NIL;
+ }
+
+ joinpath->uniquekeys = result;
+}
+
+/*
+ * Return true if path having given uniquekeys cannot duplicate the query
+ * grouping expressions.
+ *
+ * TODO Does this need to be in a separate function?
+ */
+bool
+match_uniquekeys_to_groupkeys(PlannerInfo *root, List *uniquekeys)
+{
+ ListCell *l;
+
+ /*
+ * Empty uniquekeys means that we should return false, but no caller
+ * should pass such a value.
+ */
+ Assert(uniquekeys != NIL);
+
+ /*
+ * The join must not duplicate grouping keys. In other words, there must
+ * be no uniquekey that is not present in the grouping keys.
+ */
+ foreach(l, uniquekeys)
+ {
+ ListCell *l2;
+ EquivalenceClass *uniquekey = lfirst_node(EquivalenceClass, l);
+
+ foreach(l2, root->group_pathkeys)
+ {
+ PathKey *pk = lfirst_node(PathKey, l2);
+
+ if (pk->pk_eclass == uniquekey)
+ break;
+ }
+ if (l2 == NULL)
+ {
+ /*
+ * Found an uniquekey that is not a grouping expression. This is
+ * sufficient to duplicate the grouping expressions.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
+
+
+/*
+ * Construct the common uniquekeys produced by paths of a grouped base or join
+ * relation.
+ */
+List *
+make_grouped_rel_uniquekeys(PlannerInfo *root, PathTarget *target)
+{
+ ListCell *l;
+ List *result = NIL;
+
+ foreach(l, target->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(l);
+ EquivalenceClass *uniquekey;
+
+ /*
+ * Aggrefs should be located at the end of the target. Once we hit the
+ * first one, we're done with grouping expressions.
+ */
+ if (IsA(expr, Aggref))
+ break;
+
+ uniquekey = get_uniquekey_for_expr(root, expr);
+
+ if (uniquekey)
+ result = list_append_unique_ptr(result, uniquekey);
+ else
+ {
+ list_free(result);
+ return NIL;
+ }
+ }
+
+ return result;
+}
+
+/*
+ * Return an EC eligible to be a "unique key", which contains given
+ * expression.
+ */
+static EquivalenceClass *
+get_uniquekey_for_expr(PlannerInfo *root, Expr *expr)
+{
+ ListCell *l;
+ EquivalenceClass *result = NULL;
+
+ /*
+ * Since the uniquekeys will eventually be matched to group_pathkeys,
+ * search for the EC there rather than in eq_classes. By using
+ * group_pathkeys we also make sure that pointer equality is sufficient to
+ * match the final join uniquekeys to group_pathkeys, although the planner
+ * tries to avoid duplicating the ECs anyway.
+ */
+ foreach(l, root->group_pathkeys)
+ {
+ ListCell *l2;
+ PathKey *pk = lfirst_node(PathKey, l);
+ EquivalenceClass *ec = pk->pk_eclass;
+
+ /*
+ * TODO Exclude any other ECs?
+ */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * Do not iterate the EC members if pk_var cannot be there.
+ */
+ if (IsA(expr, Var) &&
+ !bms_is_member(((Var *) expr)->varno, ec->ec_relids))
+ continue;
+
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+
+ /*
+ * expr should originate from RELOPT_BASEREL.
+ */
+ if (em->em_is_child)
+ continue;
+
+ /*
+ * If the attribute can become NULL above the base relation, the
+ * PK values are not guaranteed to be unique in the upper
+ * relations.
+ */
+ if (!bms_is_empty(em->em_nullable_relids))
+ continue;
+
+ if (equal(em->em_expr, expr))
+ {
+ /*
+ * Accept this EC.
+ */
+ result = ec;
+ break;
+ }
+ }
+ if (l2 != NULL)
+ {
+ /*
+ * This EC is ok, so no need to check the following ones.
+ */
+ break;
+ }
+ }
+
+ return result;
+}
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 39f5729b91..c9d21d3143 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,6 +17,7 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
#include "optimizer/pathnode.h"
@@ -26,6 +27,8 @@
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "optimizer/var.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
@@ -62,6 +65,10 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis);
+static bool have_all_group_exprs(PlannerInfo *root, PathTarget *target);
/*
@@ -131,6 +138,34 @@ setup_append_rel_array(PlannerInfo *root)
}
/*
+ * setup_all_baserels
+ * Construct the all_baserels Relids set.
+ */
+void
+setup_all_baserels(PlannerInfo *root)
+{
+ Index rti;
+
+ root->all_baserels = NULL;
+ for (rti = 1; rti < root->simple_rel_array_size; rti++)
+ {
+ RelOptInfo *brel = root->simple_rel_array[rti];
+
+ /* there may be empty slots corresponding to non-baserel RTEs */
+ if (brel == NULL)
+ continue;
+
+ Assert(brel->relid == rti); /* sanity check on array */
+
+ /* ignore RTEs that are "other rels" */
+ if (brel->reloptkind != RELOPT_BASEREL)
+ continue;
+
+ root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
+ }
+}
+
+/*
* build_simple_rel
* Construct a new RelOptInfo for a base relation or 'other' relation.
*/
@@ -153,7 +188,14 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->reloptkind = parent ? RELOPT_OTHER_MEMBER_REL : RELOPT_BASEREL;
rel->relids = bms_make_singleton(relid);
rel->rows = 0;
- /* cheap startup cost is interesting iff not all tuples to be retrieved */
+
+ /*
+ * Cheap startup cost is interesting iff not all tuples to be retrieved.
+ * XXX As for grouped relation, the startup cost might be interesting for
+ * AGG_SORTED (if it can produce the ordering that matches
+ * root->query_pathkeys) but not in general (other kinds of aggregation
+ * need the whole relation). Yet it seems worth trying.
+ */
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
@@ -167,6 +209,8 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->agg_info = NULL;
+ rel->grouped = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
@@ -315,6 +359,55 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation.
+ *
+ * Flat copy ensures that we do not miss any information that the non-grouped
+ * rel already contains. XXX Do we need to copy any Node field?
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, RelOptInfo *rel)
+{
+ ListCell *l;
+ List *indexlist = NIL;
+ RelOptInfo *result;
+
+ result = makeNode(RelOptInfo);
+ memcpy(result, rel, sizeof(RelOptInfo));
+
+ /*
+ * The new relation is grouped itself.
+ */
+ result->grouped = NULL;
+
+ /*
+ * The target to generate aggregation input will be initialized later.
+ */
+ result->reltarget = NULL;
+
+ /*
+ * Make sure that index paths have access to the parent rel's agg_info,
+ * which is used to indicate that the rel should produce grouped paths.
+ */
+ foreach(l, result->indexlist)
+ {
+ IndexOptInfo *src,
+ *dst;
+
+ src = lfirst_node(IndexOptInfo, l);
+ dst = makeNode(IndexOptInfo);
+ memcpy(dst, src, sizeof(IndexOptInfo));
+
+ dst->rel = result;
+ indexlist = lappend(indexlist, dst);
+ }
+ result->indexlist = indexlist;
+
+ return result;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -487,7 +580,9 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
static void
add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
{
- /* GEQO requires us to append the new joinrel to the end of the list! */
+ /*
+ * GEQO requires us to append the new joinrel to the end of the list!
+ */
root->join_rel_list = lappend(root->join_rel_list, joinrel);
/* store it into the auxiliary hashtable if there is one. */
@@ -517,6 +612,9 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'grouped' forces creation of a "standalone" object, i.e. w/o search in the
+ * join list and without adding the result to the list. Caller is
+ * responsible for setup of reltarget in such a case.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -527,10 +625,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ bool grouped)
{
- RelOptInfo *joinrel;
+ RelOptInfo *joinrel = NULL;
List *restrictlist;
+ bool create_target = !grouped;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -538,7 +638,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ if (!grouped)
+ joinrel = find_join_rel(root, joinrelids);
if (joinrel)
{
@@ -561,11 +662,11 @@ build_join_rel(PlannerInfo *root,
joinrel->reloptkind = RELOPT_JOINREL;
joinrel->relids = bms_copy(joinrelids);
joinrel->rows = 0;
- /* cheap startup cost is interesting iff not all tuples to be retrieved */
+ /* See the comment in build_simple_rel(). */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
- joinrel->reltarget = create_empty_pathtarget();
+ joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
@@ -579,6 +680,8 @@ build_join_rel(PlannerInfo *root,
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->agg_info = NULL;
+ joinrel->grouped = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
@@ -630,9 +733,13 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (create_target)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -669,31 +776,39 @@ build_join_rel(PlannerInfo *root,
/*
* Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
*
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
+ * XXX The function claims to need reltarget but it does not seem to
+ * actually use it. Should we call it unconditionally so that callers of
+ * build_join_rel() do not have to care?
*/
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (create_target)
+ {
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
/*
* Also, if dynamic-programming join search is active, add the new joinrel
@@ -701,7 +816,7 @@ build_join_rel(PlannerInfo *root,
* of members should be for equality, but some of the level 1 rels might
* have been joinrels already, so we can only assert <=.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
@@ -1771,3 +1886,680 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ /*
+ * The source relation has nothing to do with grouping.
+ */
+ Assert(rel->agg_info == NULL);
+ result = makeNode(RelAggInfo);
+
+ if (bms_equal(rel->relids, root->all_baserels))
+ {
+ /*
+ * The topmost scan/join rel is a special case, as the suitable target
+ * should already be available, so don't waste cycles constructing it.
+ *
+ * By using an existing target we also ensure that ordering of
+ * expressions matches that of the query. Thus we ensure that subquery
+ * rel target does match the target of the subquery plan.
+ */
+ result->target = root->upper_targets[UPPERREL_GROUP_AGG];
+ Assert(result->target);
+
+ /*
+ * Since aggregation makes little sense for the topmost scan/join rel
+ * (create_grouping_paths() will do the same anyway), no other fields
+ * of the RelAggInfo should be needed.
+ */
+ return result;
+ }
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if (list_length(target->exprs) == 0)
+ return NULL;
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ /*
+ * Check if target for 1-stage aggregation can be setup.
+ */
+ if (!have_all_group_exprs(root, target))
+ return NULL;
+
+ result->plain_rel = rel;
+ result->target = target;
+ result->input = agg_input;
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, there no grouping expressions should
+ * follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ cl = get_sortgroupref_clause(sortgroupref,
+ root->parse->groupClause);
+
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = castNode(Aggref, lfirst(lc));
+
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /*
+ * Calculate uniquekeys for AggPaths.
+ */
+ result->uniquekeys = make_grouped_rel_uniquekeys(root, target);
+ if (result->uniquekeys &&
+ !match_uniquekeys_to_groupkeys(root, result->uniquekeys))
+ result->uniquekeys = NIL;
+
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var. (As for generic grouping expressions,
+ * add_grouped_vars_to_target will retrieve them from the query
+ * targetlist and add them to "target" outside this function.)
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. (For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.)
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by upper join. This includes one that is
+ * referenced by a generic grouping expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as grouping expression on its
+ * own at the top of the loop, so we can add it to the
+ * grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * We can't use this as a grouping expression at relation
+ * level because the aggregation push-down feature does not
+ * involve 2-stage replication so far: there's no final
+ * aggregation that would get rid of the additional groups.
+ * Thus we need to reject aggregation of the relation
+ * altogether.
+ *
+ * (We could just ignore this case if we were sure that the
+ * join clause only contains GROUP BY expressions containing
+ * this var, as opposed to the var alone, and let
+ * create_rel_agg_info() add those expressions to the target.
+ * However it does not seem worth the effort.)
+ */
+ return false;
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by generic grouping expression
+ * but not referenced by any join.
+ *
+ * create_rel_agg_info() could add the whole generic expression
+ * but there doesn't seem to be an obvious use case for this.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/*
+ * Check if relation target contains all the grouping expressions of the
+ * query.
+ *
+ * EC-derived grouping expression has the same sortgroupref as the original
+ * expression, so by checking sortgroupref we can easily match the derived
+ * expressions too.
+ */
+static bool
+have_all_group_exprs(PlannerInfo *root, PathTarget *target)
+{
+ Bitmapset *sgr_set = NULL;
+ Bitmapset *sgr_query_set = NULL;
+ ListCell *lc;
+ int i = 0;
+
+ /*
+ *
+ * TODO Take into account the fact that grouping expressions can be
+ * derived using ECs, so a single expression of the target can correspond
+ * to multiple expressions of the query target. (There's no reason to put
+ * GROUP BY usually multiple EC members in the GROUP BY clause, but if
+ * user does, it should not disable aggregation push-down.)
+ *
+ * First, collect sortgrouprefs of the relation target.
+ */
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+
+ Assert(target->sortgrouprefs != NULL);
+ sortgroupref = target->sortgrouprefs[i++];
+ if (sortgroupref > 0)
+ sgr_set = bms_add_member(sgr_set, sortgroupref);
+ }
+
+ /*
+ * Collect those of the query.
+ */
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+ ListCell *lc2;
+
+ if (te->ressortgroupref > 0)
+ {
+ bool accept = false;
+
+ /*
+ * Ignore target entries that contain aggregates.
+ */
+ if (IsA(te->expr, Var))
+ accept = true;
+ else
+ {
+ List *l = pull_var_clause((Node *) te->expr,
+ PVC_INCLUDE_AGGREGATES);
+
+ foreach(lc2, l)
+ {
+ Expr *expr = lfirst(lc2);
+
+ if (IsA(expr, Aggref))
+ break;
+ }
+
+ /*
+ * Accept the target entry if no Aggref was found.
+ */
+ if (lc2 == NULL)
+ accept = true;
+ list_free(l);
+ }
+ if (accept)
+ sgr_query_set = bms_add_member(sgr_query_set,
+ te->ressortgroupref);
+ }
+ }
+
+ /*
+ * Evaluate.
+ */
+ return bms_equal(sgr_set, sgr_query_set);
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 5500f33e63..aa12859eaf 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -426,7 +426,6 @@ get_sortgrouplist_exprs(List *sgClauses, List *targetList)
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
@@ -801,6 +800,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->gvexpr,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 0625eff219..bf70c6f61c 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -952,6 +952,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 697d3d7a5f..389199a761 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -223,6 +223,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -266,6 +267,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index adb4265047..0107aebcb8 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -193,6 +193,7 @@ typedef struct PlannerInfo
* unreferenced view RTE; or if the RelOptInfo hasn't been made yet.
*/
struct RelOptInfo **simple_rel_array; /* All 1-rel RelOptInfos */
+
int simple_rel_array_size; /* allocated size of array */
/*
@@ -247,6 +248,7 @@ typedef struct PlannerInfo
* join_rel_level is NULL if not in use.
*/
List **join_rel_level; /* lists of join-relation RelOptInfos */
+
int join_cur_level; /* index of list being extended */
List *init_plans; /* init SubPlans for query */
@@ -279,6 +281,8 @@ typedef struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -612,6 +616,9 @@ typedef enum RelOptKind
(rel)->reloptkind == RELOPT_OTHER_JOINREL || \
(rel)->reloptkind == RELOPT_OTHER_UPPER_REL)
+/* Does the given relation contain only grouped paths? */
+#define IS_GROUPED_REL(rel) ((rel)->agg_info != NULL)
+
typedef struct RelOptInfo
{
NodeTag type;
@@ -646,6 +653,16 @@ typedef struct RelOptInfo
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to apply aggregation to this rel's paths. */
+ struct RelAggInfo *agg_info;
+
+ /*
+ * If the relation can produce grouped paths, store them here.
+ *
+ * If "grouped" is valid then "agg_info" must be NULL and vice versa.
+ */
+ struct RelOptInfo *grouped;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
@@ -1053,6 +1070,61 @@ typedef struct ParamPathInfo
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+/*
+ * RelAggInfo
+ *
+ * RelOptInfo needs information contained here if its paths should be
+ * aggregated.
+ *
+ * "target" will be used as pathtarget for aggregation if "explicit
+ * aggregation" is applied to base relation or join. The same target will also
+ * --- if the relation is a join --- be used to joinin grouped path to a
+ * non-grouped one.
+ *
+ * These targets contain plain-Var grouping expressions and Aggrefs which.
+ * Once Aggref is evaluated, its value is passed to the upper paths w/o being
+ * evaluated again.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "input" contains Vars used either as grouping expressions or aggregate
+ * arguments, plus those used in grouping expressions which are not plain Vars
+ * themselves. Paths providing the aggregation plan with input data should use
+ * this target.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "plain_rel" is pointer to the non-grouped relation whose grouping this
+ * structure describes.
+ *
+ * "uniquekeys" is what we assign to Path.uniquekeys if it's AggPath. In
+ * contrast, if joining grouped path to non-grouped, we need to calculate
+ * uniquekeys per-path.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ PathTarget *target; /* Target for grouped paths.. */
+
+ PathTarget *input; /* pathtarget of paths that generate input for
+ * aggregation paths. */
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *plain_rel;
+
+ List *uniquekeys;
+} RelAggInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
@@ -1082,6 +1154,12 @@ typedef struct ParamPathInfo
*
* "pathkeys" is a List of PathKey nodes (see above), describing the sort
* ordering of the path's output rows.
+ *
+ * "uniquekeys" is a List of EquivalenceClass objects. A row composed of items
+ * of this list is unique across the output of this path. This is often
+ * identical for all paths of the same relations, but not always. For
+ * example, grouped join can have uniquekeys if it's an output of Agg plan
+ * but no uniquekeys if it's a join of grouped and non-grouped relation.
*/
typedef struct Path
{
@@ -1105,6 +1183,9 @@ typedef struct Path
List *pathkeys; /* sort ordering of path's output */
/* pathkeys is a List of PathKey nodes; see above */
+
+ List *uniquekeys; /* Description of an unique output row of this
+ * path. */
} Path;
/* Macro for extracting a path's parameterization relids; beware double eval */
@@ -2207,6 +2288,24 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
@@ -2313,6 +2412,7 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * do_aggregate tells whether the join output should be aggregated.
*/
typedef struct JoinPathExtraData
{
@@ -2322,6 +2422,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ bool do_aggregate;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index ed854fdd40..f9f3d14b0b 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -88,4 +88,6 @@ extern Query *inline_set_returning_function(PlannerInfo *root,
extern List *expand_function_arguments(List *args, Oid result_type,
HeapTuple func_tuple);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 77ca7ff837..bb6ec0f4e1 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -72,6 +72,7 @@ extern PGDLLIMPORT bool enable_partitionwise_aggregate;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
extern PGDLLIMPORT bool enable_partition_pruning;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int constraint_exclusion;
extern double clamp_row_est(double nrows);
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 7c5ff22650..a901212cc7 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -35,6 +35,7 @@ extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target,
Relids required_outer, int parallel_workers);
extern Path *create_samplescan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer);
@@ -123,6 +124,7 @@ extern Relids calc_non_nestloop_required_outer(Path *outer_path, Path *inner_pat
extern NestPath *create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -134,6 +136,7 @@ extern NestPath *create_nestloop_path(PlannerInfo *root,
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -148,6 +151,7 @@ extern MergePath *create_mergejoin_path(PlannerInfo *root,
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -196,6 +200,13 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool check_pathkeys,
+ double input_rows);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -255,14 +266,21 @@ extern Path *reparameterize_path(PlannerInfo *root, Path *path,
double loop_count);
extern Path *reparameterize_path_by_child(PlannerInfo *root, Path *path,
RelOptInfo *child_rel);
+extern List *make_baserel_uniquekeys(PlannerInfo *root, RelOptInfo *rel);
+extern void set_join_uniquekeys(PlannerInfo *root, Path *joinpath, Path *outer_path,
+ Path *inner_path);
+extern bool match_uniquekeys_to_groupkeys(PlannerInfo *root, List *uniquekeys);
+extern List *make_grouped_rel_uniquekeys(PlannerInfo *root, PathTarget *target);
/*
* prototypes for relnode.c
*/
extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
+extern void setup_all_baserels(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, RelOptInfo *rel);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
@@ -270,7 +288,8 @@ extern RelOptInfo *build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ bool grouped);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -297,5 +316,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index cafde307ad..bb689ff4b5 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -50,11 +51,15 @@ extern PGDLLIMPORT join_search_hook_type join_search_hook;
extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
-extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
+extern RelOptInfo *standard_join_search(PlannerInfo *root,
+ int levels_needed,
List *initial_rels);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+
+extern bool add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -92,7 +97,8 @@ extern Expr *adjust_rowcompare_for_index(RowCompareExpr *clause,
* tidpath.h
* routines to generate tid paths
*/
-extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
+extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel,
+ bool grouped);
/*
* joinpath.c
@@ -101,7 +107,8 @@ extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist,
+ bool do_aggregate);
/*
* joinrels.c
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index c8ab0280d2..ac76375b31 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -76,6 +76,8 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void add_grouped_base_rels_to_query(PlannerInfo *root);
+extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 9fa52e1278..69f14a6477 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -16,7 +16,6 @@
#include "nodes/relation.h"
-
extern TargetEntry *tlist_member(Expr *node, List *targetlist);
extern TargetEntry *tlist_member_ignore_relabel(Expr *node, List *targetlist);
@@ -41,7 +40,6 @@ extern Node *get_sortgroupclause_expr(SortGroupClause *sgClause,
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
-
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
extern SortGroupClause *get_sortgroupref_clause_noerr(Index sortref,
@@ -65,6 +63,13 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 540aba98b3..4589427747 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -199,7 +199,7 @@ pg_GSS_startup(PGconn *conn, int payloadlen)
min_stat;
int maxlen;
gss_buffer_desc temp_gbuf;
- char *host = conn->connhost[conn->whichhost].host;
+ char *host = PQhost(conn);
if (!(host && host[0] != '\0'))
{
@@ -414,7 +414,7 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
{
SECURITY_STATUS r;
TimeStamp expire;
- char *host = conn->connhost[conn->whichhost].host;
+ char *host = PQhost(conn);
if (conn->sspictx)
{
diff --git a/src/interfaces/libpq/fe-secure-common.c b/src/interfaces/libpq/fe-secure-common.c
index b3f580f595..40203f3b64 100644
--- a/src/interfaces/libpq/fe-secure-common.c
+++ b/src/interfaces/libpq/fe-secure-common.c
@@ -88,17 +88,10 @@ pq_verify_peer_name_matches_certificate_name(PGconn *conn,
{
char *name;
int result;
- char *host = conn->connhost[conn->whichhost].host;
+ char *host = PQhost(conn);
*store_name = NULL;
- if (!(host && host[0] != '\0'))
- {
- printfPQExpBuffer(&conn->errorMessage,
- libpq_gettext("host name must be specified\n"));
- return -1;
- }
-
/*
* There is no guarantee the string returned from the certificate is
* NULL-terminated, so make a copy that is.
@@ -152,7 +145,7 @@ pq_verify_peer_name_matches_certificate_name(PGconn *conn,
bool
pq_verify_peer_name_matches_certificate(PGconn *conn)
{
- char *host = conn->connhost[conn->whichhost].host;
+ char *host = PQhost(conn);
int rc;
int names_examined = 0;
char *first_name = NULL;
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..0cbe1a7dd8
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,128 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+INSERT INTO agg_pushdown_parent(i)
+SELECT n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 64 * i + n, i, random()
+FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 64 * i + n, i, random()
+FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+-- Perform scan of a table and aggregate the result. In addition, check that
+-- functionally dependent column (c.x) can be referenced by SELECT although
+-- GROUP BY references p.i.
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------
+ Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(7 rows)
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------
+ Nested Loop
+ -> GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(6 rows)
+
+SET enable_seqscan TO on;
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------
+ Nested Loop
+ -> HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------
+ Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(12 rows)
+
+ROLLBACK;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 16f979c8d9..6d406c65cc 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -98,6 +98,9 @@ test: rules psql_crosstab amutils
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 42632be675..f480c7aaa0 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -139,6 +139,7 @@ test: rules
test: psql_crosstab
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..94c4606c81
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,81 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+INSERT INTO agg_pushdown_parent(i)
+SELECT n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 64 * i + n, i, random()
+FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 64 * i + n, i, random()
+FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+-- Perform scan of a table and aggregate the result. In addition, check that
+-- functionally dependent column (c.x) can be referenced by SELECT although
+-- GROUP BY references p.i.
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ROLLBACK;
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-11-18 21:43 Tom Lane <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Tom Lane @ 2018-11-18 21:43 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
Antonin Houska <[email protected]> writes:
> [ agg_pushdown_v8.tgz ]
I spent a few hours looking at this today. It seems to me that at no
point has there been a clear explanation of what the patch is trying to
accomplish, so let me see whether I've got it straight or not. (I suggest
that something like this ought to be included in optimizer/README; the
patch's lack of internal documentation is a serious deficiency.)
--
Conceptually, we'd like to be able to push aggregation down below joins
when that yields a win, which it could do by reducing the number of rows
that have to be processed at the join stage. Thus consider
SELECT agg(a.x) FROM a, b WHERE a.y = b.z;
We can't simply aggregate during the scan of "a", because some values of
"x" might not appear at all in the input of the naively-computed aggregate
(if their rows have no join partner in "b"), or might appear multiple
times (if their rows have multiple join partners). So at first glance,
aggregating before the join is impossible. The key insight of the patch
is that we can make some progress by considering only grouped aggregation:
if we can group "a" into sets of rows that must all have the same join
partners, then we can do preliminary aggregation within each such group,
and take care of the number-of-repetitions problem when we join. If the
groups are large then this reduces the number of rows processed by the
join, at the cost that we might spend time computing the aggregate for
some row sets that will just be discarded by the join. So now we consider
SELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY ?;
and ask what properties the grouping column(s) "?" must have to make this
work. I believe we can say that it's OK to push down through a join if
its join clauses are all of the form "a.y = b.z", where either a.y or b.z
is listed as a GROUP BY column *and* the join operator is equality for
the btree opfamily specified by the SortGroupClause. (Note: actually,
SortGroupClause per se mentions an equality operator, although I think the
planner quickly reduces that to an opfamily specification.) The concerns
Robert had about equality semantics are certainly vital, but I believe
that the logic goes through correctly as long as the grouping equality
operator and the join equality operator belong to the same opfamily.
Case 1: the GROUP BY column is a.y. Two rows of "a" whose y values are
equal per the grouping operator must join to exactly the same set of "b"
rows, else transitivity is failing.
Case 2: the GROUP BY column is b.z. It still works, because the set of
"a" rows that are equal to a given z value is well-defined, and those
rows must join to exactly the "b" rows whose z entries are equal to
the given value, else transitivity is failing.
Robert postulated cases like "citext = text", but that doesn't apply
here because no cross-type citext = text operator could be part of a
well-behaved opfamily. What we'd actually be looking at is either
"citextvar::text texteq textvar" or "citextvar citexteq textvar::citext",
and the casts prevent these expressions from matching GROUP BY entries
that have the wrong semantics.
However: what we have proven here is only that we can aggregate across
a set of rows that must share the same join partners. We still have
to be able to handle the case that the rowset has more than one join
partner, which AFAICS means that we must use partial aggregation and
then apply an "aggmultifn" (or else apply the aggcombinefn N times).
We can avoid that and use plain aggregation when we can prove the "b"
side of the join is unique, so that no sets of rows will have to be merged
post-join; but ISTM that that reduces the set of use cases to be too small
to be worth such a complex patch. So I'm really doubtful that we should
proceed forward with only that case available.
Also, Tomas complained in the earlier thread that he didn't think
grouping on the join column was a very common use-case in the first
place. I share that concern, but I think we could extend the logic
to the case that Tomas posited as being useful:
SELECT agg(a.x) FROM a, b WHERE a.y = b.id GROUP BY b.z;
where the join column b.id is unique. If we group on a.y (using semantics
compatible with the join operator and the uniqueness constraint), then all
"a" rows in a given group will join to exactly one "b" row that
necessarily has exactly one grouping value, so this group can safely be
aggregated together. We might need to combine it post-join with other "b"
rows that have equal "z" values, but we can do that as long as we're okay
with partial aggregation. I think this example shows why the idea is far
more powerful with partial aggregation than without.
--
In short, then, I don't have much use for the patch as presented in this
thread, without "aggmultifn". That might be OK as the first phase in a
multi-step patch, but not as a production feature. I also have no use
for the stuff depending on bitwise equality rather than the user-visible
operators; that's just a hack, and an ugly one.
As far as the patch details go, I've not looked through it in great
detail, but it appears to me that you are still trying to cram the same
stuff into base relations as before; shoving it into a subsidiary struct
doesn't improve that. Multiple people have said that's the wrong thing,
and I agree. Conceptually it's a disaster: a single RelOptInfo should
represent one well-defined set of result rows, not more than one. This
approach also cannot be extended to handle doing aggregations partway up
the join tree, which is at least theoretically interesting (though I'm
fine with not tackling it right away). I think the right representation
is to create UPPERREL_GROUP_AGG RelOptInfos whose relids show the set
of baserels joined before aggregating. Currently there's only one of
those and its relids is equal to all_baserels (or should be, anyway).
This patch would add instances of UPPERREL_GROUP_AGG RelOptInfos with
singleton relids, and later we might put in the ability to consider
aggregation across other relid subsets, and in any case we'd run join
planning on those level-one rels and create new UPPERREL_GROUP_AGG
RelOptInfos that represent the intermediate join steps leading up to
the final join. The correct indexing structure for this collection of
RelOptInfos is certainly not simple_rel_array; it should look more like
the way we index the various regular join rels that we consider. (Maybe
an appropriate preliminary patch is to refactor the support code
associated with join_rel_list + join_rel_hash so that we can treat those
fields as one instance of a structure we'll replicate later.) Note that
I'm envisioning that we'd basically run join planning a second time on
this tree of join rels, rather than try to merge it with the primary
join planning. Since the numbers of rows to be processed will be
completely different in this join tree, merging it with the standard one
seems hopeless.
BTW, I noticed some noise in the v8 patch: what's it doing touching
src/interfaces/libpq/fe-auth.c or src/interfaces/libpq/fe-secure-common.c?
regards, tom lane
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2018-12-17 15:58 Antonin Houska <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2018-12-17 15:58 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: pgsql-hackers
Tom Lane <[email protected]> wrote:
> Antonin Houska <[email protected]> writes:
> > [ agg_pushdown_v8.tgz ]
>
> I spent a few hours looking at this today.
Thanks!
> It seems to me that at no point has there been a clear explanation of what
> the patch is trying to accomplish, so let me see whether I've got it
> straight or not. (I suggest that something like this ought to be included
> in optimizer/README; the patch's lack of internal documentation is a serious
> deficiency.)
Earlier version of the patch did update optimizer/README but I forgot to merge
it into the current one. I'll fix that in the next version. (Also some more
will be added, especially header comments of some functions.)
> Conceptually, we'd like to be able to push aggregation down below joins
> when that yields a win, which it could do by reducing the number of rows
> that have to be processed at the join stage. Thus consider
>
> SELECT agg(a.x) FROM a, b WHERE a.y = b.z;
>
> We can't simply aggregate during the scan of "a", because some values of
> "x" might not appear at all in the input of the naively-computed aggregate
> (if their rows have no join partner in "b"), or might appear multiple
> times (if their rows have multiple join partners). So at first glance,
> aggregating before the join is impossible. The key insight of the patch
> is that we can make some progress by considering only grouped aggregation:
Yes, the patch does not handle queries with no GROUP BY clause. Primarily
because I don't know how the grouped "a" table in this example could emit the
"a.y" column w/o using it as a grouping key.
> if we can group "a" into sets of rows that must all have the same join
> partners, then we can do preliminary aggregation within each such group,
> and take care of the number-of-repetitions problem when we join. If the
> groups are large then this reduces the number of rows processed by the
> join, at the cost that we might spend time computing the aggregate for
> some row sets that will just be discarded by the join. So now we consider
>
> SELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY ?;
>
> and ask what properties the grouping column(s) "?" must have to make this
> work. I believe we can say that it's OK to push down through a join if
> its join clauses are all of the form "a.y = b.z", where either a.y or b.z
> is listed as a GROUP BY column *and* the join operator is equality for
> the btree opfamily specified by the SortGroupClause. (Note: actually,
> SortGroupClause per se mentions an equality operator, although I think the
> planner quickly reduces that to an opfamily specification.)
I suppose you mean make_pathkey_from_sortop().
> The concerns Robert had about equality semantics are certainly vital, but I
> believe that the logic goes through correctly as long as the grouping
> equality operator and the join equality operator belong to the same
> opfamily.
ok, generic approach like this is always better. I just could not devise it,
nor can I prove that your theory is wrong.
> Robert postulated cases like "citext = text", but that doesn't apply
> here because no cross-type citext = text operator could be part of a
> well-behaved opfamily.
The reason I can see now is that the GROUP BY operator (opfamily) essentially
has the grouping column on both sides, so it does not match the cross-type
operator.
> What we'd actually be looking at is either "citextvar::text texteq textvar"
> or "citextvar citexteq textvar::citext", and the casts prevent these
> expressions from matching GROUP BY entries that have the wrong semantics.
Can you please explain this? If the expression is "citextvar::text texteq
textvar", then both operands of the operator should be of "text" type (because
the operator receives the output of the cast), so I'd expect a match to
SortGroupClause.eqop of clause like "GROUP BY <text expression>".
> However: what we have proven here is only that we can aggregate across
> a set of rows that must share the same join partners. We still have
> to be able to handle the case that the rowset has more than one join
> partner, which AFAICS means that we must use partial aggregation and
> then apply an "aggmultifn" (or else apply the aggcombinefn N times).
> We can avoid that and use plain aggregation when we can prove the "b"
> side of the join is unique, so that no sets of rows will have to be merged
> post-join; but ISTM that that reduces the set of use cases to be too small
> to be worth such a complex patch. So I'm really doubtful that we should
> proceed forward with only that case available.
I tried to follow Heikki's proposal in [1] and separated the functionality
that does not need the partial aggregation. I was not able to foresee that the
patch does not get much smaller this way. Also because I had to introduce
functions that check whether particular join does duplicate aggregated values
or not. (Even if we use partial aggregation, we can use this functionality to
detect that we only need to call aggfinalfn() in some cases, as opposed to
setting up regular Agg node for the final aggregation. However that's an
optimization that can probably be moved to later position in the patch
series.)
I'll re-introduce the parallel aggregation in the next patch version.
> Also, Tomas complained in the earlier thread that he didn't think
> grouping on the join column was a very common use-case in the first
> place. I share that concern, but I think we could extend the logic
> to the case that Tomas posited as being useful:
>
> SELECT agg(a.x) FROM a, b WHERE a.y = b.id GROUP BY b.z;
>
> where the join column b.id is unique. If we group on a.y (using semantics
> compatible with the join operator and the uniqueness constraint), then all
> "a" rows in a given group will join to exactly one "b" row that
> necessarily has exactly one grouping value, so this group can safely be
> aggregated together. We might need to combine it post-join with other "b"
> rows that have equal "z" values, but we can do that as long as we're okay
> with partial aggregation. I think this example shows why the idea is far
> more powerful with partial aggregation than without.
Good idea. I'll try to incorporate it in the next version.
> In short, then, I don't have much use for the patch as presented in this
> thread, without "aggmultifn". That might be OK as the first phase in a
> multi-step patch, but not as a production feature.
The version 8 actually implements only part of the functionality that earlier
versions did. I wanted to have more feedback from the community on the concept
before I rework the whole series again. So for version 9 I'm going to include
the partial aggregation. On the other hand, support of things like parallel
processing, append relation or postgres_fdw doesn't seem to be necessary for
the inital version of the feature.
> I also have no use for the stuff depending on bitwise equality rather than
> the user-visible operators; that's just a hack, and an ugly one.
The purpose of this was to avoid aggregation push-down for data types like
numeric, see the example
SELECT one.a
FROM one, two
WHERE one.a = two.a AND scale(two.a) > 1
GROUP BY 1
in [2]. The problem is that aggregation can make some information lost, which
is needed above by WHERE/ON clauses. I appreciate any suggestions how to
address this problem.
> As far as the patch details go, I've not looked through it in great
> detail, but it appears to me that you are still trying to cram the same
> stuff into base relations as before; shoving it into a subsidiary struct
> doesn't improve that. Multiple people have said that's the wrong thing,
> and I agree. Conceptually it's a disaster: a single RelOptInfo should
> represent one well-defined set of result rows, not more than one.
The first version of my patch did what you reject here, but then I started to
use a separate RelOptInfo for the grouped relations. First I introduced
simple_grouped_rel_array, but then (again per Heikki's suggestion) made the
"plain" RelOptInfo point to the grouped one.
> I think the right representation is to create UPPERREL_GROUP_AGG RelOptInfos
(Just a detail: since UPPERREL_PARTIAL_GROUP_AGG was added at some point, I
wonder if this is more appropriate than UPPERREL_GROUP_AGG for base relations
and joins. create_grouping_paths() can then add the paths to
UPPERREL_GROUP_AGG.)
> whose relids show the set of baserels joined before aggregating. Currently
> there's only one of those and its relids is equal to all_baserels (or should
> be, anyway). This patch would add instances of UPPERREL_GROUP_AGG
> RelOptInfos with singleton relids, and later we might put in the ability to
> consider aggregation across other relid subsets, and in any case we'd run
> join planning on those level-one rels and create new UPPERREL_GROUP_AGG
> RelOptInfos that represent the intermediate join steps leading up to the
> final join. The correct indexing structure for this collection of
> RelOptInfos is certainly not simple_rel_array; it should look more like the
> way we index the various regular join rels that we consider.
Nothing like simple_rel_array (e.g. simple_grouped_rel_array) even for the
base relations? I think the fact that the extra base relations are grouped
should not affect the way they are retrieved.
> (Maybe an appropriate preliminary patch is to refactor the support code
> associated with join_rel_list + join_rel_hash so that we can treat those
> fields as one instance of a structure we'll replicate later.)
Do you mean that, instead of a single RelOptInfo, join_rel_list would contain
a structure that points to two RelOptInfo structures, where one is the "plain"
one and the other is the "grouped" one?
> Note that I'm envisioning that we'd basically run join planning a second
> time on this tree of join rels, rather than try to merge it with the primary
> join planning. Since the numbers of rows to be processed will be completely
> different in this join tree, merging it with the standard one seems
> hopeless.
A separate run of make_join_rel() for the grouped relations is what I tried to
do so far.
> BTW, I noticed some noise in the v8 patch: what's it doing touching
> src/interfaces/libpq/fe-auth.c or src/interfaces/libpq/fe-secure-common.c?
It's definitely just a noise. Something went wrong when I was extracting the
diff from my private repository.
[1] https://www.postgresql.org/message-id/[email protected]
[2] https://www.postgresql.org/message-id/20239.1516971866%40localhost
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-01-15 14:06 Antonin Houska <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2019-01-15 14:06 UTC (permalink / raw)
To: [email protected]; +Cc: pgsql-hackers
This is the next version. A few more comments below.
Antonin Houska <[email protected]> wrote:
> Tom Lane <[email protected]> wrote:
>
> > Antonin Houska <[email protected]> writes:
>
> Thanks!
>
> > Conceptually, we'd like to be able to push aggregation down below joins
> > when that yields a win, which it could do by reducing the number of rows
> > that have to be processed at the join stage. Thus consider
> >
> > SELECT agg(a.x) FROM a, b WHERE a.y = b.z;
> >
> > We can't simply aggregate during the scan of "a", because some values of
> > "x" might not appear at all in the input of the naively-computed aggregate
> > (if their rows have no join partner in "b"), or might appear multiple
> > times (if their rows have multiple join partners). So at first glance,
> > aggregating before the join is impossible. The key insight of the patch
> > is that we can make some progress by considering only grouped aggregation:
>
> Yes, the patch does not handle queries with no GROUP BY clause. Primarily
> because I don't know how the grouped "a" table in this example could emit the
> "a.y" column w/o using it as a grouping key.
>
> > if we can group "a" into sets of rows that must all have the same join
> > partners, then we can do preliminary aggregation within each such group,
> > and take care of the number-of-repetitions problem when we join. If the
> > groups are large then this reduces the number of rows processed by the
> > join, at the cost that we might spend time computing the aggregate for
> > some row sets that will just be discarded by the join. So now we consider
> >
> > SELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY ?;
> >
> > and ask what properties the grouping column(s) "?" must have to make this
> > work. I believe we can say that it's OK to push down through a join if
> > its join clauses are all of the form "a.y = b.z", where either a.y or b.z
> > is listed as a GROUP BY column *and* the join operator is equality for
> > the btree opfamily specified by the SortGroupClause.
I think the requirement for the join clauses to be of the form "a.y = b.z"
would only be valid if we wanted to avoid the 2-stage aggregation. For
example, in this query
SELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY b.z
uniqueness of "b.z" ensures that groups resulting from the pushed-down
aggregation (SELECT a.y, agg(a.x) FROM a GROUP BY a.y) do not get duplicated
by the join. However if there's a consensus that the 2-stage aggregation will
be used (because avoidance of it would limit usefulness of the feature too
much), I think we can accept generic join clauses. (Note that the patch does
not try to push aggregate down below nullable side of outer join - handling of
generated NULL values by join clauses would be another problem.)
> > The concerns Robert had about equality semantics are certainly vital, but I
> > believe that the logic goes through correctly as long as the grouping
> > equality operator and the join equality operator belong to the same
> > opfamily.
>
> ok, generic approach like this is always better. I just could not devise it,
> nor can I prove that your theory is wrong.
>
> > Robert postulated cases like "citext = text", but that doesn't apply
> > here because no cross-type citext = text operator could be part of a
> > well-behaved opfamily.
>
> The reason I can see now is that the GROUP BY operator (opfamily) essentially
> has the grouping column on both sides, so it does not match the cross-type
> operator.
Even if my statement that the join clause can be of other kind than "<leftarg>
= <rightarg>" was right, it might still be worth insisting on "<leftargt> <op>
<rightarg>" where <op> is of the same operator family as the grouping
operator?
> > Also, Tomas complained in the earlier thread that he didn't think
> > grouping on the join column was a very common use-case in the first
> > place. I share that concern, but I think we could extend the logic
> > to the case that Tomas posited as being useful:
> >
> > SELECT agg(a.x) FROM a, b WHERE a.y = b.id GROUP BY b.z;
> >
> > where the join column b.id is unique. If we group on a.y (using semantics
> > compatible with the join operator and the uniqueness constraint), then all
> > "a" rows in a given group will join to exactly one "b" row that
> > necessarily has exactly one grouping value, so this group can safely be
> > aggregated together. We might need to combine it post-join with other "b"
> > rows that have equal "z" values, but we can do that as long as we're okay
> > with partial aggregation. I think this example shows why the idea is far
> > more powerful with partial aggregation than without.
>
> Good idea. I'll try to incorporate it in the next version.
The previous version of the patch (before I removed the partial aggregation)
could actually handle this case, so I restored it. It just does not check the
uniqueness of the b.id column because: the final aggregation will eliminate
the duplicated groups anyway.
The patch views the problem from this perspective: "a.y" is needed by join
clause, and the only way to retrieve it from the partially aggregated relation
"a" is to use it as a grouping column. Such an "artificial" grouping
expression can only be used if the plan contains the final aggregate node,
which uses only the original GROUP BY clause.
> > I also have no use for the stuff depending on bitwise equality rather than
> > the user-visible operators; that's just a hack, and an ugly one.
>
> The purpose of this was to avoid aggregation push-down for data types like
> numeric, see the example
>
> SELECT one.a
> FROM one, two
> WHERE one.a = two.a AND scale(two.a) > 1
> GROUP BY 1
>
> in [2]. The problem is that aggregation can make some information lost, which
> is needed above by WHERE/ON clauses. I appreciate any suggestions how to
> address this problem.
The new patch version does not contain this hack, but the problem is still
open. Alternatively I think of a boolean flag to be added to pg_opfamily or
pg_opclass catalog, saying whether equality also means "bitwise equality". If
the value is false (the default) for the GROUP BY operator, no aggregate
push-down can take place.
> > I think the right representation is to create UPPERREL_GROUP_AGG
> > RelOptInfos whose relids show the set of baserels joined before
> > aggregating. Currently there's only one of those and its relids is equal
> > to all_baserels (or should be, anyway). This patch would add instances of
> > UPPERREL_GROUP_AGG RelOptInfos with singleton relids, and later we might
> > put in the ability to consider aggregation across other relid subsets, and
> > in any case we'd run join planning on those level-one rels and create new
> > UPPERREL_GROUP_AGG RelOptInfos that represent the intermediate join steps
> > leading up to the final join. The correct indexing structure for this
> > collection of RelOptInfos is certainly not simple_rel_array; it should
> > look more like the way we index the various regular join rels that we
> > consider.
> > (Maybe an appropriate preliminary patch is to refactor the support code
> > associated with join_rel_list + join_rel_hash so that we can treat those
> > fields as one instance of a structure we'll replicate later.)
> Do you mean that, instead of a single RelOptInfo, join_rel_list would contain
> a structure that points to two RelOptInfo structures, where one is the "plain"
> one and the other is the "grouped" one?
Eventually I thought that you propose a separate instance of join_rel_list
(e.g. join_rel_list_grouped) for the 2nd run of the joining process, however
that would mean that some functions need to receive RelOptInfos as arguments,
and *also* search in the the join_rel_list_grouped when joining grouped
relation to non-grouped one. So I decided to wrap both grouped and non-grouped
relation into a new structure (RelOptInfoSet). Thus all the relations to be
joined are passed as function arguments. make_join_rel() shows best how
instances of the new structure is used.
> [1] https://www.postgresql.org/message-id/[email protected]
>
> [2] https://www.postgresql.org/message-id/20239.1516971866%40localhost
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] v09-001-Export_estimate_hashagg_tablesize.patch (4.5K, ../../10529.1547561178@localhost/2-v09-001-Export_estimate_hashagg_tablesize.patch)
download | inline diff:
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b849ae03b8..6f3a4a063c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -146,9 +146,6 @@ static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd,
List *target_list);
-static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
@@ -3634,40 +3631,6 @@ get_number_of_groups(PlannerInfo *root,
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
-static Size
-estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
-{
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
-}
-
-/*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index e8f51d2d0d..be059f0de0 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -114,6 +114,7 @@
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+#include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -3902,6 +3903,39 @@ estimate_hash_bucket_stats(PlannerInfo *root, Node *hashkey, double nbuckets,
ReleaseVariableStats(vardata);
}
+/*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+Size
+estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+{
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+}
/*-------------------------------------------------------------------------
*
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
index 5cc4cf15e2..e65ab13b1a 100644
--- a/src/include/utils/selfuncs.h
+++ b/src/include/utils/selfuncs.h
@@ -213,6 +213,9 @@ extern void estimate_hash_bucket_stats(PlannerInfo *root,
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
[text/x-diff] v09-002-Introduce_make_join_rel_common.patch (2.5K, ../../10529.1547561178@localhost/3-v09-002-Introduce_make_join_rel_common.patch)
download | inline diff:
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 38eeb23d81..0caf67a916 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -37,6 +37,7 @@ static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -651,21 +652,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -748,6 +740,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen
+ * when working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
+/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
[text/x-diff] v09-003-Introduce_estimate_join_rows.patch (2.0K, ../../10529.1547561178@localhost/4-v09-003-Introduce_estimate_join_rows.patch)
download | inline diff:
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 99c5ad9b4a..353dc116f4 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2291,6 +2291,20 @@ cost_group(Path *path, PlannerInfo *root,
}
/*
+ * estimate_join_rows
+ * Set rows of a join path according to its parent relation or according
+ * to parameters.
+ */
+static void
+estimate_join_rows(PlannerInfo *root, Path *path)
+{
+ if (path->param_info)
+ path->rows = path->param_info->ppi_rows;
+ else
+ path->rows = path->parent->rows;
+}
+
+/*
* initial_cost_nestloop
* Preliminary estimate of the cost of a nestloop join path.
*
@@ -2411,10 +2425,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->path.param_info)
- path->path.rows = path->path.param_info->ppi_rows;
- else
- path->path.rows = path->path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2857,10 +2868,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3287,10 +3295,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
[text/x-diff] v09-004-Agg_pushdown_basic.patch (189.5K, ../../10529.1547561178@localhost/5-v09-004-Agg_pushdown_basic.patch)
download | inline diff:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 006a3d1772..d46885de93 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2198,8 +2198,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* relation.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2340,6 +2340,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5107,6 +5121,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0fde876c77..8c91355931 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2198,6 +2198,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2205,6 +2206,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2416,6 +2418,22 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+/*
+ * Note we do NOT print plain_rel, else we'd be in infinite recursion.
+ */
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2504,6 +2522,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4039,6 +4069,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4054,6 +4087,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 9c852a15ef..52e31404b0 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1118,3 +1118,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index e07bab831e..4beac45fdf 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -182,13 +182,15 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
for (rel_count = 0; rel_count < num_gene; rel_count++)
{
int cur_rel_index;
+ RelOptInfoSet *cur_relset;
RelOptInfo *cur_rel;
Clump *cur_clump;
/* Get the next input relation */
cur_rel_index = (int) tour[rel_count];
- cur_rel = (RelOptInfo *) list_nth(private->initial_rels,
- cur_rel_index - 1);
+ cur_relset = (RelOptInfoSet *) list_nth(private->initial_rels,
+ cur_rel_index - 1);
+ cur_rel = cur_relset->rel_plain;
/* Make it into a single-rel clump */
cur_clump = (Clump *) palloc(sizeof(Clump));
@@ -251,16 +253,24 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
desirable_join(root, old_clump->joinrel, new_clump->joinrel))
{
RelOptInfo *joinrel;
+ RelOptInfoSet *oldset,
+ *newset;
/*
* Construct a RelOptInfo representing the join of these two input
* relations. Note that we expect the joinrel not to exist in
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
+ *
+ * TODO Consider using make_join_rel_common() (possibly renamed)
+ * here instead of wrapping the joinrels into RelOptInfoSet.
*/
- joinrel = make_join_rel(root,
- old_clump->joinrel,
- new_clump->joinrel);
+ oldset = makeNode(RelOptInfoSet);
+ oldset->rel_plain = old_clump->joinrel;
+ newset = makeNode(RelOptInfoSet);
+ newset->rel_plain = new_clump->joinrel;
+
+ joinrel = make_join_rel(root, oldset, newset);
/* Keep searching if join order is not valid */
if (joinrel)
diff --git a/src/backend/optimizer/geqo/geqo_main.c b/src/backend/optimizer/geqo/geqo_main.c
index dc51829dec..b4a4f4ff27 100644
--- a/src/backend/optimizer/geqo/geqo_main.c
+++ b/src/backend/optimizer/geqo/geqo_main.c
@@ -63,7 +63,7 @@ static int gimme_number_generations(int pool_size);
* similar to a constrained Traveling Salesman Problem (TSP)
*/
-RelOptInfo *
+RelOptInfoSet *
geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
{
GeqoPrivateData private;
@@ -74,6 +74,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
Pool *pool;
int pool_size,
number_generations;
+ RelOptInfoSet *result;
#ifdef GEQO_DEBUG
int status_interval;
@@ -296,7 +297,9 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
/* ... clear root pointer to our private storage */
root->join_search_private = NULL;
- return best_rel;
+ result = makeNode(RelOptInfoSet);
+ result->rel_plain = best_rel;
+ return result;
}
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index bc389b52e4..54bd229daf 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -59,6 +59,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -74,16 +75,18 @@ static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
static void set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelAggInfo *agg_info);
static void create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel);
static void set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
@@ -119,7 +122,8 @@ static void set_namedtuplestore_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
-static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
+static RelOptInfoSet *make_rel_from_joinlist(PlannerInfo *root,
+ List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
@@ -143,10 +147,10 @@ static void remove_unused_subquery_outputs(Query *subquery, RelOptInfo *rel);
* Finds all possible access paths for executing a query, returning a
* single rel that represents the join of all base rels in the query.
*/
-RelOptInfo *
+RelOptInfoSet *
make_one_rel(PlannerInfo *root, List *joinlist)
{
- RelOptInfo *rel;
+ RelOptInfoSet *rel;
Index rti;
double total_pages;
@@ -224,7 +228,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
/*
* The result should join all and only the query's base rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->rel_plain->relids, root->all_baserels));
return rel;
}
@@ -315,7 +319,7 @@ set_base_rel_sizes(PlannerInfo *root)
if (root->glob->parallelModeOK)
set_rel_consider_parallel(root, rel, rte);
- set_rel_size(root, rel, rti, rte);
+ set_rel_size(root, rel, rti, rte, NULL);
}
}
@@ -344,18 +348,29 @@ set_base_rel_pathlists(PlannerInfo *root)
if (rel->reloptkind != RELOPT_BASEREL)
continue;
- set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti]);
+ set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti],
+ NULL, NULL);
}
}
/*
* set_rel_size
* Set size estimates for a base relation
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info)
{
+ bool grouped = agg_info != NULL;
+
+ /*
+ * Aggregate push-down is currently used only for relations.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (rel->reloptkind == RELOPT_BASEREL &&
relation_excluded_by_constraints(root, rel, rte))
{
@@ -374,7 +389,13 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support grouped
+ * append relation.
+ */
+ Assert(!grouped);
set_append_rel_size(root, rel, rti, rte);
}
else
@@ -384,7 +405,12 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Grouped foreign table is not supported.
+ */
+ Assert(!grouped);
set_foreign_size(root, rel, rte);
}
else if (rte->relkind == RELKIND_PARTITIONED_TABLE)
@@ -392,18 +418,27 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* A partitioned table without any partitions is marked as
* a dummy rel.
+ *
+ * Grouped partitioned table is not supported.
*/
+ Assert(!grouped);
+
set_dummy_rel_pathlist(rel);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation
+ *
+ * Grouped tablesample rel is not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_size(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_size(root, rel, rte);
+ set_plain_rel_size(root, rel, rte, agg_info);
}
break;
case RTE_SUBQUERY:
@@ -454,18 +489,49 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* set_rel_pathlist
* Build access paths for a base relation
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
+ bool grouped = rel_grouped != NULL;
+
+ /*
+ * Aggregate push-down is currently implemented for RTE_RELATION only.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
+ if (grouped)
+ {
+ /*
+ * Do not apply AggPath if this is the only relation of the query:
+ * create_grouping_paths() will do so anyway.
+ */
+ if (bms_equal(rel->relids, root->all_baserels))
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
+ }
+
if (IS_DUMMY_REL(rel))
{
/* We already proved the relation empty, so nothing more to do */
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support append
+ * relation.
+ */
+ Assert(!grouped);
set_append_rel_pathlist(root, rel, rti, rte);
}
else
@@ -475,18 +541,29 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_foreign_pathlist(root, rel, rte);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation.
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_pathlist(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_pathlist(root, rel, rte);
+ set_plain_rel_pathlist(root, rel, rte, rel_grouped,
+ agg_info);
}
break;
case RTE_SUBQUERY:
@@ -528,9 +605,12 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* Also, if this is the topmost scan/join rel (that is, the only baserel),
* we postpone this until the final scan/join targelist is available (see
* grouping_planner).
+ *
+ * Note on aggregation push-down: parallel paths are not supported so far.
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON &&
+ !grouped)
generate_gather_paths(root, rel, false);
/*
@@ -539,10 +619,24 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* add_path(), or delete or modify paths added by the core code.
*/
if (set_rel_pathlist_hook)
- (*set_rel_pathlist_hook) (root, rel, rti, rte);
+ (*set_rel_pathlist_hook) (root, rel, rti, rte, rel_grouped,
+ agg_info);
+
+ /*
+ * The grouped relation is not guaranteed to have any paths. If the
+ * pathlist is empty, there's no point in calling set_cheapest().
+ */
+ if (rel_grouped && rel_grouped->pathlist == NIL)
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
/* Now find the cheapest of the paths for this rel */
- set_cheapest(rel);
+ if (!grouped)
+ set_cheapest(rel);
+ else
+ set_cheapest(rel_grouped);
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -552,9 +646,13 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_size
* Set size estimates for a plain relation (no subquery, no inheritance)
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
-set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
+ RelAggInfo *agg_info)
{
/*
* Test any partial indexes of rel for applicability. We must do this
@@ -563,7 +661,7 @@ set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
check_index_predicates(root, rel);
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, agg_info);
}
/*
@@ -740,11 +838,19 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_pathlist
* Build access paths for a plain relation (no subquery, no inheritance)
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
-set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
Relids required_outer;
+ Path *seq_path;
+ bool grouped = rel_grouped != NULL;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
@@ -753,18 +859,38 @@ set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
*/
required_outer = rel->lateral_relids;
- /* Consider sequential scan */
- add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
+ /* Consider sequential scan. */
+ seq_path = create_seqscan_path(root, rel, required_outer, 0, rel_grouped,
+ agg_info);
+
+ if (!grouped)
+ add_path(rel, seq_path);
+
+ /*
+ * It's probably not good idea to repeat hashed aggregation with different
+ * parameters, so check if there are no parameters.
+ */
+ else if (required_outer == NULL)
+ {
+ /*
+ * Only AGG_HASHED is suitable here as it does not expect the input
+ * set to be sorted.
+ */
+ add_grouped_path(root, rel_grouped, seq_path, AGG_HASHED, agg_info);
+ }
/* If appropriate, consider parallel sequential scan */
- if (rel->consider_parallel && required_outer == NULL)
+ if (rel->consider_parallel && required_outer == NULL && !grouped)
create_plain_partial_paths(root, rel);
- /* Consider index scans */
- create_index_paths(root, rel);
+ /*
+ * Consider index scans, possibly including the grouped paths.
+ */
+ create_index_paths(root, rel, rel_grouped, agg_info);
/* Consider TID scans */
- create_tidscan_paths(root, rel);
+ if (!grouped)
+ create_tidscan_paths(root, rel);
}
/*
@@ -776,15 +902,55 @@ create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
- parallel_workers = compute_parallel_worker(rel, rel->pages, -1,
- max_parallel_workers_per_gather);
+ parallel_workers = compute_parallel_worker(rel, rel->pages, -1, max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
- add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
+ add_partial_path(rel, create_seqscan_path(root, rel, NULL,
+ parallel_workers, NULL, NULL));
+}
+
+/*
+ * Apply aggregation to a subpath and add the AggPath to the pathlist.
+ *
+ * The return value tells whether the path was added to the pathlist.
+ *
+ * XXX Pass the plain rel and fetch the grouped from rel->grouped? How about
+ * subroutines then?
+ */
+bool
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+ /*
+ * Repeated creation of hash table does not sound like a good idea. Caller
+ * should avoid asking us to do so.
+ */
+ Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ {
+ add_path(rel, (Path *) agg_path);
+
+ return true;
+ }
+
+ return false;
}
/*
@@ -824,7 +990,7 @@ set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
rel->tuples = tuples;
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -1222,7 +1388,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's size.
*/
- set_rel_size(root, childrel, childRTindex, childRTE);
+ set_rel_size(root, childrel, childRTindex, childRTE, NULL);
/*
* It is possible that constraint exclusion detected a contradiction
@@ -1371,7 +1537,7 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's access paths.
*/
- set_rel_pathlist(root, childrel, childRTindex, childRTE);
+ set_rel_pathlist(root, childrel, childRTindex, childRTE, NULL, NULL);
/*
* If child is dummy, ignore it.
@@ -2631,7 +2797,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
-static RelOptInfo *
+static RelOptInfoSet *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
@@ -2657,13 +2823,35 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
- RelOptInfo *thisrel;
+ RelOptInfoSet *thisrel;
if (IsA(jlnode, RangeTblRef))
{
- int varno = ((RangeTblRef *) jlnode)->rtindex;
+ RangeTblRef *rtref = castNode(RangeTblRef, jlnode);
+ int varno = rtref->rtindex;
+ RangeTblEntry *rte = root->simple_rte_array[varno];
+ RelOptInfo *rel_grouped;
+
+ thisrel = makeNode(RelOptInfoSet);
+ thisrel->rel_plain = find_base_rel(root, varno);
- thisrel = find_base_rel(root, varno);
+ /*
+ * Create the grouped counterpart of rel_plain if possible.
+ */
+ build_simple_grouped_rel(root, thisrel);
+ rel_grouped = thisrel->rel_grouped;
+
+ /*
+ * If succeeded, do what should already have been done for the
+ * plain relation.
+ */
+ if (rel_grouped)
+ {
+ set_rel_size(root, rel_grouped, varno, rte,
+ thisrel->agg_info);
+ set_rel_pathlist(root, thisrel->rel_plain, varno, rte,
+ rel_grouped, thisrel->agg_info);
+ }
}
else if (IsA(jlnode, List))
{
@@ -2685,7 +2873,7 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
/*
* Single joinlist node, so we're done.
*/
- return (RelOptInfo *) linitial(initial_rels);
+ return (RelOptInfoSet *) linitial(initial_rels);
}
else
{
@@ -2699,11 +2887,19 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
root->initial_rels = initial_rels;
if (join_search_hook)
- return (*join_search_hook) (root, levels_needed, initial_rels);
+ return (*join_search_hook) (root, levels_needed,
+ initial_rels);
else if (enable_geqo && levels_needed >= geqo_threshold)
+ {
+ /*
+ * TODO Teach GEQO about grouped relations. Don't forget that
+ * pathlist can be NIL before set_cheapest() gets called.
+ */
return geqo(root, levels_needed, initial_rels);
+ }
else
- return standard_join_search(root, levels_needed, initial_rels);
+ return standard_join_search(root, levels_needed,
+ initial_rels);
}
}
@@ -2736,11 +2932,11 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
-RelOptInfo *
+RelOptInfoSet *
standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
{
int lev;
- RelOptInfo *rel;
+ RelOptInfoSet *result;
/*
* This function cannot be invoked recursively within any one planning
@@ -2782,13 +2978,20 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
+ *
+ * Neither partitionwise join nor parallel processing is supported for
+ * grouped relations so far.
*/
foreach(lc, root->join_rel_level[lev])
{
- rel = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *rel_set;
+ RelOptInfo *rel_plain;
+
+ rel_set = (RelOptInfoSet *) lfirst(lc);
+ rel_plain = rel_set->rel_plain;
/* Create paths for partitionwise joins. */
- generate_partitionwise_join_paths(root, rel);
+ generate_partitionwise_join_paths(root, rel_plain);
/*
* Except for the topmost scan/join rel, consider gathering
@@ -2796,10 +2999,33 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* once we know the final targetlist (see grouping_planner).
*/
if (lev < levels_needed)
- generate_gather_paths(root, rel, false);
+ generate_gather_paths(root, rel_plain, false);
/* Find and save the cheapest paths for this rel */
- set_cheapest(rel);
+ set_cheapest(rel_plain);
+
+ if (rel_set->rel_grouped)
+ {
+ RelOptInfo *rel_grouped;
+
+ rel_grouped = rel_set->rel_grouped;
+
+ /* Partial paths not supported yet. */
+ Assert(rel_grouped->partial_pathlist == NIL);
+
+ if (rel_grouped->pathlist != NIL)
+ set_cheapest(rel_grouped);
+ else
+ {
+ /*
+ * When checking whether the grouped relation can supply
+ * any input paths to join, test for existence of the
+ * relation will be easier than a test of pathlist.
+ */
+ pfree(rel_grouped);
+ rel_set->rel_grouped = NULL;
+ }
+ }
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -2814,11 +3040,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
Assert(list_length(root->join_rel_level[levels_needed]) == 1);
- rel = (RelOptInfo *) linitial(root->join_rel_level[levels_needed]);
+ result = (RelOptInfoSet *) linitial(root->join_rel_level[levels_needed]);
root->join_rel_level = NULL;
- return rel;
+ return result;
}
/*****************************************************************************
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 353dc116f4..7bca52daad 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -88,6 +88,7 @@
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
@@ -2293,13 +2294,23 @@ cost_group(Path *path, PlannerInfo *root,
/*
* estimate_join_rows
* Set rows of a join path according to its parent relation or according
- * to parameters.
+ * to parameters. If agg_info is passed, the join path is grouped.
*/
static void
-estimate_join_rows(PlannerInfo *root, Path *path)
+estimate_join_rows(PlannerInfo *root, Path *path, RelAggInfo *agg_info)
{
if (path->param_info)
+ {
path->rows = path->param_info->ppi_rows;
+ if (agg_info)
+ {
+ double nrows;
+
+ nrows = estimate_num_groups(root, agg_info->group_exprs,
+ path->rows, NULL);
+ path->rows = clamp_row_est(nrows);
+ }
+ }
else
path->rows = path->parent->rows;
}
@@ -2425,7 +2436,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2868,7 +2879,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3295,7 +3306,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -4300,27 +4311,59 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
* restriction clauses).
* width: the estimated average output tuple width in bytes.
* baserestrictcost: estimated cost of evaluating baserestrictinfo clauses.
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
void
-set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
+set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info)
{
double nrows;
+ bool grouped = agg_info != NULL;
/* Should only be applied to base relations */
Assert(rel->relid > 0);
- nrows = rel->tuples *
- clauselist_selectivity(root,
- rel->baserestrictinfo,
- 0,
- JOIN_INNER,
- NULL);
+ if (!grouped)
+ {
+ nrows = rel->tuples *
+ clauselist_selectivity(root,
+ rel->baserestrictinfo,
+ 0,
+ JOIN_INNER,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
- rel->rows = clamp_row_est(nrows);
+ /*
+ * Only set the estimate for grouped base rel if aggregation can take
+ * place. (Aggregation is the only way to build grouped base relation.)
+ */
+ else if (!bms_equal(rel->relids, root->all_baserels))
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ nrows = estimate_num_groups(root,
+ agg_info->group_exprs,
+ agg_info->input_rows,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
- set_rel_width(root, rel);
+ /*
+ * The grouped target should have the cost and width set immediately on
+ * creation, see create_rel_agg_info().
+ */
+ if (!grouped)
+ set_rel_width(root, rel);
+#ifdef USE_ASSERT_CHECKING
+ else
+ Assert(rel->reltarget->width > 0);
+#endif
}
/*
@@ -4884,7 +4927,7 @@ set_subquery_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4922,7 +4965,7 @@ set_function_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4944,7 +4987,7 @@ set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 100;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4975,7 +5018,7 @@ set_values_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = list_length(rte->values_lists);
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5013,7 +5056,7 @@ set_cte_size_estimates(PlannerInfo *root, RelOptInfo *rel, double cte_rows)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5046,7 +5089,7 @@ set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 1000;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5270,11 +5313,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5305,6 +5348,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 6e134ae1d2..25f1b7fb44 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -65,7 +65,6 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
-
/*
* process_equivalence
* The given clause has a mergejoinable operator and can be applied without
@@ -2511,3 +2510,137 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index f8e674c9c4..23e5ecf93e 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -33,6 +33,7 @@
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
@@ -78,13 +79,14 @@ typedef struct
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
-
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths);
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -93,7 +95,9 @@ static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -101,19 +105,24 @@ static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static bool eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
List *indexjoinclauses);
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths);
+ List **bitindexpaths, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop);
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -217,6 +226,10 @@ static Const *string_to_const(const char *str, Oid datatype);
*
* 'rel' is the relation for which we want to generate index paths
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Note: check_index_predicates() must have been run previously for this rel.
*
* Note: in cases involving LATERAL references in the relation's tlist, it's
@@ -229,7 +242,8 @@ static Const *string_to_const(const char *str, Oid datatype);
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
-create_index_paths(PlannerInfo *root, RelOptInfo *rel)
+create_index_paths(PlannerInfo *root, RelOptInfo *rel, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
List *bitindexpaths;
@@ -239,6 +253,7 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
IndexClauseSet jclauseset;
IndexClauseSet eclauseset;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/* Skip the whole mess if no indexes */
if (rel->indexlist == NIL)
@@ -274,8 +289,8 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
- get_index_paths(root, rel, index, &rclauseset,
- &bitindexpaths);
+ get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
+ rel_grouped, agg_info);
/*
* Identify the join clauses that can match the index. For the moment
@@ -304,15 +319,24 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
&rclauseset,
&jclauseset,
&eclauseset,
- &bitjoinpaths);
+ &bitjoinpaths,
+ rel_grouped,
+ agg_info);
}
/*
+ * It does not seem too efficient to aggregate the individual paths and
+ * then AND them together.
+ */
+ if (grouped)
+ return;
+
+ /*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
- indexpaths = generate_bitmap_or_paths(root, rel,
- rel->baserestrictinfo, NIL);
+ indexpaths = generate_bitmap_or_paths(root, rel, rel->baserestrictinfo,
+ NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
@@ -434,6 +458,10 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* 'jclauseset' is the collection of indexable simple join clauses
* 'eclauseset' is the collection of indexable clauses from EquivalenceClasses
* '*bitindexpaths' is the list to add bitmap paths to
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
@@ -441,7 +469,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
int considered_clauses = 0;
List *considered_relids = NIL;
@@ -477,7 +507,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
jclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
/* Consider each applicable eclass join clause */
considered_clauses += list_length(eclauseset->indexclauses[indexcol]);
consider_index_join_outer_rels(root, rel, index,
@@ -485,7 +517,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
eclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -500,6 +534,10 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
* 'indexjoinclauses' is a list of RestrictInfos for join clauses
* 'considered_clauses' is the total number of clauses considered (so far)
* '*considered_relids' is a list of all relids sets already considered
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
@@ -510,7 +548,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
ListCell *lc;
@@ -577,7 +617,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
bms_union(clause_relids, oldrelids),
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
/* Also try this set of relids by itself */
@@ -585,7 +627,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
clause_relids,
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -601,6 +645,10 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
* 'bitindexpaths', 'considered_relids' as above
* 'relids' is the current set of relids to consider (the target rel plus
* one or more outer rels)
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -610,7 +658,9 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexClauseSet clauseset;
int indexcol;
@@ -667,7 +717,8 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
- get_index_paths(root, rel, index, &clauseset, bitindexpaths);
+ get_index_paths(root, rel, index, &clauseset, bitindexpaths,
+ rel_grouped, agg_info);
/*
* Remember we considered paths for this set of relids. We use lcons not
@@ -717,7 +768,6 @@ bms_equal_any(Relids relids, List *relids_list)
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
@@ -732,16 +782,23 @@ bms_equal_any(Relids relids, List *relids_list)
* paths, and then make a separate attempt to include them in bitmap paths.
* Furthermore, we should consider excluding lower-order ScalarArrayOpExpr
* quals so as to create ordered paths.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
bool skip_nonnative_saop = false;
bool skip_lower_saop = false;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/*
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
@@ -754,7 +811,40 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- &skip_lower_saop);
+ &skip_lower_saop,
+ rel_grouped,
+ agg_info);
+
+ /*
+ * If the relation is grouped, apply the partial aggregation.
+ *
+ * Only AGG_SORTED strategy is used, so we ignore bitmap paths, as well as
+ * indexes that can only produce them.
+ */
+ if (grouped && index->amhasgettuple)
+ {
+ foreach(lc, indexpaths)
+ {
+ IndexPath *ipath = (IndexPath *) lfirst(lc);
+ Path *subpath = &ipath->path;
+
+ if (subpath->pathkeys != NIL)
+ {
+ AggPath *agg_path;
+
+ agg_path = create_agg_sorted_path(root,
+ subpath,
+ agg_info);
+ if (agg_path)
+ add_path(rel_grouped, (Path *) agg_path);
+ }
+ }
+
+ /*
+ * Nothing more to do for grouped relation.
+ */
+ return;
+ }
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
@@ -769,7 +859,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- NULL));
+ NULL,
+ rel_grouped,
+ agg_info));
}
/*
@@ -809,7 +901,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
false,
ST_BITMAPSCAN,
NULL,
- NULL);
+ NULL,
+ rel_grouped,
+ agg_info);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
@@ -853,7 +947,15 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
- * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
+ * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
+ * XXX When enabling the aggregate push-down for partial paths, we'll need the
+ * function to return the aggregation input paths in a separate list instead
+ * of calling add_partial_path() on them immediately.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -861,7 +963,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop)
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *result = NIL;
IndexPath *ipath;
@@ -878,6 +982,12 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool include_partial;
+
+ /*
+ * Grouped partial paths are not supported yet.
+ */
+ include_partial = rel_grouped == NULL;
/*
* Check that index supports the desired scan type(s)
@@ -1047,14 +1157,16 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
*/
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1070,7 +1182,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth using
@@ -1104,11 +1218,13 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1122,7 +1238,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth
@@ -1244,6 +1362,8 @@ build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
useful_predicate,
ST_BITMAPSCAN,
NULL,
+ NULL,
+ NULL,
NULL);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d8ff4bf432..f99d6b98a9 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -120,7 +120,9 @@ add_paths_to_joinrel(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
@@ -143,6 +145,8 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.agg_info = agg_info;
+ extra.rel_agg_input = rel_agg_input;
/*
* See if the inner relation is provably unique for this outer rel.
@@ -376,6 +380,8 @@ try_nestloop_path(PlannerInfo *root,
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Paths are parameterized by top-level parents, so run parameterization
@@ -422,10 +428,37 @@ try_nestloop_path(PlannerInfo *root,
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * If the join output should be aggregated, the precheck is skipped
+ * because it makes little sense to compare the new join path to existing,
+ * already aggregated paths. Since we don't have row count estimate yet,
+ * it's hard to involve AggPath in the precheck.
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ /*
+ * If the join output is subject to aggregation, the path must
+ * generate aggregation input.
+ */
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
@@ -447,21 +480,57 @@ try_nestloop_path(PlannerInfo *root,
}
}
- add_path(joinrel, (Path *)
- create_nestloop_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer));
+ path = (Path *) create_nestloop_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /*
+ * Non-grouped rel had to be passed to create_nestloop_path() so
+ * that row estimate reflects the aggregation input data, however
+ * even the aggregation input path should eventually be owned by
+ * the grouped joinrel.
+ */
+ path->parent = joinrel;
+
+ /*
+ * Try both AGG_HASHED and AGG_SORTED aggregation.
+ *
+ * AGG_HASHED should not be parameterized because we don't want to
+ * create the hashtable again for each set of parameters.
+ */
+ if (required_outer == NULL)
+ success = add_grouped_path(root, joinrel, path,
+ AGG_HASHED, extra->agg_info);
+
+ /*
+ * Don't try AGG_SORTED if add_grouped_path() would reject it
+ * anyway.
+ */
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root, joinrel, path, AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
- /* Waste no memory when we reject a path here */
+ /* Waste no memory when we reject path(s) here */
bms_free(required_outer);
}
}
@@ -538,6 +607,7 @@ try_partial_nestloop_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_nestloop_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -568,8 +638,11 @@ try_mergejoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
- if (is_partial)
+ if (!grouped && is_partial)
{
try_partial_mergejoin_path(root,
joinrel,
@@ -617,26 +690,73 @@ try_mergejoin_path(PlannerInfo *root,
outersortkeys, innersortkeys,
extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_mergejoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer,
- mergeclauses,
- outersortkeys,
- innersortkeys));
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_mergejoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer,
+ mergeclauses,
+ outersortkeys,
+ innersortkeys);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ if (required_outer == NULL)
+ success = add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_HASHED,
+ extra->agg_info);
+
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -700,6 +820,7 @@ try_partial_mergejoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_mergejoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -729,6 +850,9 @@ try_hashjoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Check to see if proposed path is still parameterized, and reject if the
@@ -745,30 +869,79 @@ try_hashjoin_path(PlannerInfo *root,
}
/*
+ * Parameterized execution of grouped path would mean repeated hashing of
+ * the output of the hashjoin output, so forget about AGG_HASHED if there
+ * are any parameters. And AGG_SORTED makes no sense because the hash join
+ * output is not sorted.
+ */
+ if (required_outer && grouped)
+ return;
+
+ /*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- NIL, required_outer))
+
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ NIL, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_hashjoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- false, /* parallel_hash */
- extra->restrictlist,
- required_outer,
- hashclauses));
+ PathTarget *target;
+ Path *path = NULL;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_hashjoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ false, /* parallel_hash */
+ extra->restrictlist,
+ required_outer,
+ hashclauses);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ /*
+ * As the hashjoin path is not sorted, only try AGG_HASHED.
+ */
+ if (add_grouped_path(root, joinrel, path, AGG_HASHED,
+ extra->agg_info))
+ success = true;
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -824,6 +997,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_hashjoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -892,6 +1066,7 @@ sort_inner_and_outer(PlannerInfo *root,
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped = extra->agg_info != NULL;
/*
* We only consider the cheapest-total-cost input paths, since we are
@@ -1051,7 +1226,7 @@ sort_inner_and_outer(PlannerInfo *root,
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
- if (cheapest_partial_outer && cheapest_safe_inner)
+ if (!grouped && cheapest_partial_outer && cheapest_safe_inner)
try_partial_mergejoin_path(root,
joinrel,
cheapest_partial_outer,
@@ -1341,6 +1516,7 @@ match_unsorted_outer(PlannerInfo *root,
Path *inner_cheapest_total = innerrel->cheapest_total_path;
Path *matpath = NULL;
ListCell *lc1;
+ bool grouped = extra->agg_info != NULL;
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
@@ -1516,7 +1692,8 @@ match_unsorted_outer(PlannerInfo *root,
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
- if (joinrel->consider_parallel &&
+ if (!grouped &&
+ joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 0caf67a916..cc2e32bd81 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -17,30 +17,41 @@
#include "miscadmin.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
+#include "optimizer/cost.h"
#include "optimizer/joininfo.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
+#include "optimizer/tlist.h"
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
static void make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
+static void set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
-static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input);
+static void make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist);
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
@@ -48,7 +59,6 @@ static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
-
/*
* join_search_one_level
* Consider ways to produce join relations containing exactly 'level'
@@ -83,7 +93,15 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel;
+
+ /*
+ * Both rel_plain and rel_grouped should have the same clauses, so it
+ * does not matter which one we use here. The only difference is that
+ * rel_grouped is not guaranteed to exist, so use rel_plain.
+ */
+ old_rel = old_relset->rel_plain;
if (old_rel->joininfo != NIL || old_rel->has_eclass_joins ||
has_join_restriction(root, old_rel))
@@ -109,7 +127,7 @@ join_search_one_level(PlannerInfo *root, int level)
other_rels = list_head(joinrels[1]);
make_rels_by_clause_joins(root,
- old_rel,
+ old_relset,
other_rels);
}
else
@@ -127,7 +145,7 @@ join_search_one_level(PlannerInfo *root, int level)
* avoid the duplicated effort.
*/
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
}
@@ -153,7 +171,8 @@ join_search_one_level(PlannerInfo *root, int level)
foreach(r, joinrels[k])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel = old_relset->rel_plain;
ListCell *other_rels;
ListCell *r2;
@@ -173,7 +192,8 @@ join_search_one_level(PlannerInfo *root, int level)
for_each_cell(r2, other_rels)
{
- RelOptInfo *new_rel = (RelOptInfo *) lfirst(r2);
+ RelOptInfoSet *new_relset = (RelOptInfoSet *) lfirst(r2);
+ RelOptInfo *new_rel = new_relset->rel_plain;
if (!bms_overlap(old_rel->relids, new_rel->relids))
{
@@ -185,7 +205,7 @@ join_search_one_level(PlannerInfo *root, int level)
if (have_relevant_joinclause(root, old_rel, new_rel) ||
have_join_order_restriction(root, old_rel, new_rel))
{
- (void) make_join_rel(root, old_rel, new_rel);
+ (void) make_join_rel(root, old_relset, new_relset);
}
}
}
@@ -219,10 +239,10 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
@@ -268,25 +288,31 @@ join_search_one_level(PlannerInfo *root, int level)
* 'other_rels': the first cell in a linked list containing the other
* rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it
* will work for joining to joinrels too.
*/
static void
make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(old_rel->relids, other_rel->relids) &&
(have_relevant_joinclause(root, old_rel, other_rel) ||
have_join_order_restriction(root, old_rel, other_rel)))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
@@ -302,27 +328,62 @@ make_rels_by_clause_joins(PlannerInfo *root,
* 'other_rels': the first cell of a linked list containing the
* other rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it would
* work for joining to joinrels too.
*/
static void
make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(other_rel->relids, old_rel->relids))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
+/*
+ * Set joinrel's reltarget according to agg_info and estimate the number of
+ * rows.
+ */
+static void
+set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info)
+{
+ Assert(agg_info != NULL);
+
+ /*
+ * build_join_rel() does not create the target for grouped relation.
+ */
+ Assert(joinrel->reltarget == NULL);
+
+ joinrel->reltarget = agg_info->target;
+
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied to
+ * one of the input rels. Is this worth extra effort, e.g. maintaining a
+ * separate RelOptInfo for each case (one difficulty that would introduce
+ * is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+}
/*
* join_is_legal
@@ -655,9 +716,18 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation
+ * should not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -665,10 +735,15 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
SpecialJoinInfo sjinfo_data;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool do_aggregate = rel_agg_input != NULL;
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
+ /* do_aggregate implies the output to be grouped. */
+ Assert(!do_aggregate || grouped);
+
/* Construct Relids set that identifies the joinrel. */
joinrelids = bms_union(rel1->relids, rel2->relids);
@@ -718,7 +793,18 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
+
+ if (grouped)
+ {
+ /*
+ * Make sure the grouped joinrel has reltarget initialized. Caller
+ * should supply the target for grouped relation, so build_join_rel()
+ * should have omitted its creation.
+ */
+ if (joinrel->reltarget == NULL)
+ set_grouped_joinrel_target(root, joinrel, agg_info);
+ }
/*
* If we've already proven this join is empty, we needn't consider any
@@ -732,13 +818,72 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
bms_free(joinrelids);
return joinrel;
}
+static void
+make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped = NULL;
+ RelOptInfo *rel2_grouped = NULL;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /*
+ * Retrieve the grouped relations.
+ *
+ * Dummy rel may indicates a join relation that is able to generate
+ * grouped paths as such (i.e. it has valid agg_info), but for which the
+ * path actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ if (relset1->rel_grouped)
+ rel1_grouped = relset1->rel_grouped;
+ if (relset2->rel_grouped)
+ rel2_grouped = relset2->rel_grouped;
+
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /*
+ * Nothing to do?
+ */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, relset2->rel_plain, agg_info,
+ NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, relset1->rel_plain, rel2_grouped, agg_info,
+ NULL);
+}
+
/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
@@ -747,14 +892,101 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but there's no useful grouped input
+ * relation.
+ *
+ * Only the plain relation is returned.
+ *
+ * TODO geqo is the only caller interested in the result. We'll need to return
+ * RelOptInfoSet if geqo adopts the aggregate pushdown technique.
*/
RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info;
+ RelOptInfoSet *joinrelset;
+ RelOptInfo *joinrel_plain;
+ RelOptInfo *rel1 = relset1->rel_plain;
+ RelOptInfo *rel2 = relset2->rel_plain;
+
+ /* 1) form the plain join. */
+ joinrel_plain = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ /*
+ * If the same grouped joinrel was already formed, just with the base rels
+ * divided between rel1 and rel2 in a different way, we should already
+ * have the matching agg_info.
+ */
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrelset = find_join_rel(root, joinrelids);
+
+ /*
+ * At the moment we know that non-grouped join exists, so the containing
+ * joinrelset should have been fetched.
+ */
+ Assert(joinrelset != NULL);
+
+ if (joinrelset->rel_grouped != NULL)
+ {
+ /*
+ * The same agg_info should be used for all the rels consisting of
+ * exactly joinrelids.
+ */
+ agg_info = joinrelset->agg_info;
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ if (agg_info != NULL)
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * Cannot we build grouped join?
+ */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the final one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_common_grouped(root, relset1, relset2, agg_info);
+
+ return joinrel_plain;
}
/*
@@ -767,8 +999,11 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist)
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
+ bool grouped = agg_info != NULL;
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -798,10 +1033,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
@@ -815,10 +1050,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
@@ -829,10 +1064,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
/*
* If there are join quals that aren't mergeable or hashable, we
@@ -865,7 +1100,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
/*
@@ -888,10 +1123,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
break;
case JOIN_ANTI:
@@ -906,7 +1141,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
default:
/* other values not expected here */
@@ -914,8 +1149,16 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
}
- /* Apply partitionwise join technique, if possible. */
- try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
+ /*
+ * The aggregate push-down feature currently does not support
+ * partition-wise aggregation.
+ */
+ if (grouped)
+ return;
+
+ /* Apply partition-wise join technique, if possible. */
+ try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
}
@@ -1115,7 +1358,8 @@ has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel)
foreach(lc, root->initial_rels)
{
- RelOptInfo *rel2 = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *relset2 = lfirst_node(RelOptInfoSet, lc);
+ RelOptInfo *rel2 = relset2->rel_plain;
/* ignore rels that are already in "rel" */
if (bms_overlap(rel->relids, rel2->relids))
@@ -1423,7 +1667,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
- child_restrictlist);
+ child_restrictlist, NULL, NULL);
}
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 97d0c28132..1bc73a8110 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -887,6 +887,18 @@ use_physical_tlist(PlannerInfo *root, Path *path, int flags)
}
}
+ /*
+ * If aggregate was pushed down, the target can contain aggregates. The
+ * original target must be preserved then.
+ */
+ foreach(lc, path->pathtarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, Aggref))
+ return false;
+ }
+
return true;
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index a66374094f..965d993ba5 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "catalog/pg_class.h"
#include "nodes/nodeFuncs.h"
@@ -27,6 +28,7 @@
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
@@ -46,6 +48,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -96,10 +100,9 @@ static void check_hashjoinable(RestrictInfo *restrictinfo);
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
- * every non-join RTE that is used in the query. Therefore, this routine
- * is the only place that should call build_simple_rel with reloptkind
- * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
- * "other rel" RelOptInfos for the members of any appendrels we find here.)
+ * every non-grouped non-join RTE that is used in the query. (Note:
+ * build_simple_rel recurses internally to build "other rel" RelOptInfos for
+ * the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
@@ -241,6 +244,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index d98cb89984..6dd3a34a99 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -348,6 +348,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
List *tlist;
NullTest *ntest;
SortGroupClause *sortcl;
+ RelOptInfoSet *final_relset;
RelOptInfo *final_rel;
Path *sorted_path;
Cost path_cost;
@@ -441,7 +442,8 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_relset = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_rel = final_relset->rel_plain;
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index fc97a1bb50..7130a05659 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -52,13 +52,15 @@
* qp_callback once we have completed merging the query's equivalence classes.
* (We cannot construct canonical pathkeys until that's done.)
*/
-RelOptInfo *
+RelOptInfoSet *
query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra)
{
Query *parse = root->parse;
List *joinlist;
+ RelOptInfoSet *final_rel_set;
RelOptInfo *final_rel;
+ RelOptInfoSet *final_relset;
/*
* If the query has an empty join tree, then it's something easy like
@@ -66,6 +68,8 @@ query_planner(PlannerInfo *root, List *tlist,
*/
if (parse->jointree->fromlist == NIL)
{
+ RelOptInfo *final_rel;
+
/* We need a dummy joinrel to describe the empty set of baserels */
final_rel = build_empty_join_rel(root);
@@ -95,7 +99,9 @@ query_planner(PlannerInfo *root, List *tlist,
root->canon_pathkeys = NIL;
(*qp_callback) (root, qp_extra);
- return final_rel;
+ final_relset = makeNode(RelOptInfoSet);
+ final_relset->rel_plain = final_rel;
+ return final_relset;
}
/*
@@ -114,6 +120,7 @@ query_planner(PlannerInfo *root, List *tlist,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -199,6 +206,11 @@ query_planner(PlannerInfo *root, List *tlist,
joinlist = remove_useless_joins(root, joinlist);
/*
+ * No rels should disappear now, so we can initialize all_baserels.
+ */
+ /* setup_all_baserels(root); */
+
+ /*
* Also, reduce any semijoins with unique inner rels to plain inner joins.
* Likewise, this can't be done until now for lack of needed info.
*/
@@ -232,14 +244,25 @@ query_planner(PlannerInfo *root, List *tlist,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
+ /*
* Ready to do the primary planning.
*/
- final_rel = make_one_rel(root, joinlist);
+ final_rel_set = make_one_rel(root, joinlist);
+ final_rel = final_rel_set->rel_plain;
/* Check that we got at least one usable path */
if (!final_rel || !final_rel->cheapest_total_path ||
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
- return final_rel;
+ return final_rel_set;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 6f3a4a063c..3ef806fd75 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -147,7 +147,7 @@ static double get_number_of_groups(PlannerInfo *root,
grouping_sets_data *gd,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -160,7 +160,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -627,6 +627,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1659,6 +1660,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ RelOptInfoSet *current_relset;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
ListCell *lc;
@@ -1878,8 +1880,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, tlist,
- standard_qp_callback, &qp_extra);
+ current_relset = query_planner(root, tlist,
+ standard_qp_callback, &qp_extra);
+ current_rel = current_relset->rel_plain;
/*
* Convert the query's result tlist into PathTarget format.
@@ -2019,7 +2022,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
if (have_grouping)
{
current_rel = create_grouping_paths(root,
- current_rel,
+ current_relset,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3650,13 +3653,14 @@ get_number_of_groups(PlannerInfo *root,
*/
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd)
{
Query *parse = root->parse;
+ RelOptInfo *input_rel = input_relset->rel_plain;
RelOptInfo *grouped_rel;
RelOptInfo *partially_grouped_rel;
@@ -3741,7 +3745,7 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_relset, grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -3897,13 +3901,14 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
* function creates, or to NULL if it doesn't create one.
*/
static void
-create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
GroupPathExtraData *extra,
RelOptInfo **partially_grouped_rel_p)
{
+ RelOptInfo *input_rel = input_relset->rel_plain;
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *partially_grouped_rel = NULL;
double dNumGroups;
@@ -3947,13 +3952,25 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ RelOptInfo *input_rel_grouped = input_relset->rel_grouped;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ input_rel_grouped->pathlist != NIL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -3962,6 +3979,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -3986,10 +4020,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -6068,7 +6106,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
comparisonCost = 2.0 * (indexExprCost.startup + indexExprCost.per_tuple);
/* Estimate the cost of seq scan + sort */
- seqScanPath = create_seqscan_path(root, rel, NULL, 0);
+ seqScanPath = create_seqscan_path(root, rel, NULL, 0, NULL, NULL);
cost_sort(&seqScanAndSortPath, root, NIL,
seqScanPath->total_cost, rel->tuples, rel->reltarget->width,
comparisonCost, maintenance_work_mem, -1.0);
@@ -6077,7 +6115,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
indexScanPath = create_index_path(root, indexInfo,
NIL, NIL, NIL, NIL, NIL,
ForwardScanDirection, false,
- NULL, 1.0, false);
+ NULL, 1.0, false, NULL, NULL);
return (seqScanAndSortPath.total_cost < indexScanPath->path.total_cost);
}
@@ -7090,6 +7128,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
for (cnt_parts = 0; cnt_parts < nparts; cnt_parts++)
{
RelOptInfo *child_input_rel = input_rel->part_rels[cnt_parts];
+ RelOptInfoSet *child_input_relset;
PathTarget *child_target = copy_pathtarget(target);
AppendRelInfo **appinfos;
int nappinfos;
@@ -7148,12 +7187,17 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
continue;
}
+ child_input_relset = makeNode(RelOptInfoSet);
+ child_input_relset->rel_plain = child_input_rel;
+
/* Create grouping paths for this child relation. */
- create_ordinary_grouping_paths(root, child_input_rel,
+ create_ordinary_grouping_paths(root, child_input_relset,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ pfree(child_input_relset);
+
if (child_partially_grouped_rel)
{
partially_grouped_live_children =
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 5d363edab8..52f970ae71 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2299,6 +2299,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 77dbf4eba3..a07e41169e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -914,6 +914,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index b2637d0e89..899d4b4c6b 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -17,6 +17,8 @@
#include <math.h>
#include "miscadmin.h"
+#include "access/sysattr.h"
+#include "catalog/pg_constraint.h"
#include "foreign/fdwapi.h"
#include "nodes/extensible.h"
#include "nodes/nodeFuncs.h"
@@ -58,7 +60,6 @@ static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
-
/*****************************************************************************
* MISC. PATH UTILITIES
*****************************************************************************/
@@ -953,13 +954,15 @@ add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
*/
Path *
create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers)
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info)
{
Path *pathnode = makeNode(Path);
pathnode->pathtype = T_SeqScan;
- pathnode->parent = rel;
- pathnode->pathtarget = rel->reltarget;
+ pathnode->parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
@@ -1019,6 +1022,10 @@ create_samplescan_path(PlannerInfo *root, RelOptInfo *rel, Relids required_outer
* estimates of caching behavior.
* 'partial_path' is true if constructing a parallel index scan path.
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Returns the new path node.
*/
IndexPath *
@@ -1033,7 +1040,9 @@ create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path)
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexPath *pathnode = makeNode(IndexPath);
RelOptInfo *rel = index->rel;
@@ -1041,8 +1050,9 @@ create_index_path(PlannerInfo *root,
*indexqualcols;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
- pathnode->path.parent = rel;
- pathnode->path.pathtarget = rel->reltarget;
+ pathnode->path.parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->path.pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
@@ -1186,8 +1196,8 @@ create_bitmap_or_path(PlannerInfo *root,
* Creates a path corresponding to a scan by TID, returning the pathnode.
*/
TidPath *
-create_tidscan_path(PlannerInfo *root, RelOptInfo *rel, List *tidquals,
- Relids required_outer)
+create_tidscan_path(PlannerInfo *root, RelOptInfo *rel,
+ List *tidquals, Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
@@ -1342,7 +1352,8 @@ append_startup_cost_compare(const void *a, const void *b)
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
- * pathnode.
+ * pathnode. target can be supplied by caller. If NULL is passed, the field
+ * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
@@ -1529,7 +1540,9 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
MemoryContext oldcontext;
int numCols;
- /* Caller made a mistake if subpath isn't cheapest_total ... */
+ /*
+ * Caller made a mistake if subpath isn't cheapest_total.
+ */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
@@ -2150,6 +2163,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
* relations.
*
* 'joinrel' is the join relation.
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_nestloop
* 'extra' contains various information about the join
@@ -2164,6 +2178,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
NestPath *
create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2204,7 +2219,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
- pathnode->path.pathtarget = joinrel->reltarget;
+ pathnode->path.pathtarget = target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2236,6 +2251,7 @@ create_nestloop_path(PlannerInfo *root,
* two relations
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_mergejoin
* 'extra' contains various information about the join
@@ -2252,6 +2268,7 @@ create_nestloop_path(PlannerInfo *root,
MergePath *
create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2268,7 +2285,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2304,6 +2321,7 @@ create_mergejoin_path(PlannerInfo *root,
* Creates a pathnode corresponding to a hash join between two relations.
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_hashjoin
* 'extra' contains various information about the join
@@ -2318,6 +2336,7 @@ create_mergejoin_path(PlannerInfo *root,
HashPath *
create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2332,7 +2351,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2414,8 +2433,8 @@ create_projection_path(PlannerInfo *root,
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
- if (is_projection_capable_path(subpath) ||
- equal(oldtarget->exprs, target->exprs))
+ if ((is_projection_capable_path(subpath) ||
+ equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
@@ -2800,8 +2819,7 @@ create_agg_path(PlannerInfo *root,
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
@@ -2834,6 +2852,152 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
@@ -3518,7 +3682,8 @@ reparameterize_path(PlannerInfo *root, Path *path,
switch (path->pathtype)
{
case T_SeqScan:
- return create_seqscan_path(root, rel, required_outer, 0);
+ return create_seqscan_path(root, rel, required_outer, 0, NULL,
+ NULL);
case T_SampleScan:
return (Path *) create_samplescan_path(root, rel, required_outer);
case T_IndexScan:
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 8141160c37..5d9c2a09e3 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,6 +17,8 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
@@ -24,9 +26,12 @@
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "optimizer/var.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
@@ -34,7 +39,7 @@
typedef struct JoinHashEntry
{
Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
+ RelOptInfoSet *join_relset;
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
@@ -54,7 +59,7 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
-static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
+static void add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
@@ -63,6 +68,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -316,6 +324,97 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation.
+ */
+void
+build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *relset)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ rel_plain = relset->rel_plain;
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return;;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, relset->rel_plain);
+ if (agg_info == NULL)
+ return;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * The aggregation paths will get their input target from agg_info, so
+ * store it too.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ relset->rel_grouped = rel_grouped;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+ relset->agg_info = agg_info;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -364,16 +463,16 @@ build_join_rel_hash(PlannerInfo *root)
/* Insert all the already-existing joinrels */
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *relset = (RelOptInfoSet *) lfirst(l);
JoinHashEntry *hentry;
bool found;
hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
+ &(relset->rel_plain->relids),
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->join_relset = relset;
}
root->join_rel_hash = hashtab;
@@ -384,7 +483,7 @@ build_join_rel_hash(PlannerInfo *root)
* Returns relation entry corresponding to 'relids' (a set of RT indexes),
* or NULL if none exists. This is for join relations.
*/
-RelOptInfo *
+RelOptInfoSet *
find_join_rel(PlannerInfo *root, Relids relids)
{
/*
@@ -412,7 +511,13 @@ find_join_rel(PlannerInfo *root, Relids relids)
HASH_FIND,
NULL);
if (hentry)
- return hentry->join_rel;
+ {
+ RelOptInfoSet *result = hentry->join_relset;;
+
+ /* The plain relation should always be there. */
+ Assert(result->rel_plain != NULL);
+ return result;
+ }
}
else
{
@@ -420,10 +525,10 @@ find_join_rel(PlannerInfo *root, Relids relids)
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *result = (RelOptInfoSet *) lfirst(l);
- if (bms_equal(rel->relids, relids))
- return rel;
+ if (bms_equal(result->rel_plain->relids, relids))
+ return result;
}
}
@@ -486,23 +591,26 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset)
{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
+ /*
+ * GEQO requires us to append the new joinrel to the end of the list!
+ */
+ root->join_rel_list = lappend(root->join_rel_list, joinrelset);
/* store it into the auxiliary hashtable if there is one. */
if (root->join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
+ Relids relids = joinrelset->rel_plain->relids;
hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
+ &relids,
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = joinrel;
+ hentry->join_relset = joinrelset;
}
}
@@ -518,6 +626,8 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' contains information needed for the join to form grouped
+ * paths. If NULL, the join is not grouped.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -528,10 +638,20 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
+ RelOptInfoSet *joinrelset;
+ bool new_set = false;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool create_target;
+
+ /*
+ * Target for grouped relation will be supplied by caller.
+ */
+ create_target = !grouped;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -539,7 +659,19 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrelset = find_join_rel(root, joinrelids);
+ if (joinrelset == NULL)
+ {
+ /*
+ * The plain joinrel should be the first one to be added to the set.
+ */
+ Assert(!grouped);
+
+ joinrelset = makeNode(RelOptInfoSet);
+ new_set = true;
+ }
+
+ joinrel = !grouped ? joinrelset->rel_plain : joinrelset->rel_grouped;
if (joinrel)
{
@@ -566,7 +698,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
- joinrel->reltarget = create_empty_pathtarget();
+ joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
@@ -631,9 +763,13 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (create_target)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -669,45 +805,68 @@ build_join_rel(PlannerInfo *root,
sjinfo->jointype);
/*
- * Set estimates of the joinrel's size.
+ * Assign the joinrel to the set.
*/
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
+ if (!grouped)
+ joinrelset->rel_plain = joinrel;
+ else
+ {
+ joinrelset->rel_grouped = joinrel;
+ joinrelset->agg_info = agg_info;
+ }
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (new_set)
+ {
+ /* Add the joinrelset to the PlannerInfo. */
+ add_join_rel(root, joinrelset);
- /* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ /*
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only
+ * assert <=.
+ */
+ if (root->join_rel_level)
+ {
+ Assert(root->join_cur_level > 0);
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
+ root->join_rel_level[root->join_cur_level] =
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrelset);
+ }
+ }
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Set estimates of the joinrel's size.
+ *
+ * XXX set_joinrel_size_estimates() claims to need reltarget but it does
+ * not seem to actually use it. Should we call it unconditionally so that
+ * callers of build_join_rel() do not have to care?
*/
- if (root->join_rel_level)
+ if (create_target)
{
- Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
- root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
}
return joinrel;
@@ -734,6 +893,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
JoinType jointype)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
+ RelOptInfoSet *joinrelset = makeNode(RelOptInfoSet);
AppendRelInfo **appinfos;
int nappinfos;
@@ -845,7 +1005,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
Assert(!find_join_rel(root, joinrel->relids));
/* Add the relation to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ joinrelset->rel_plain = joinrel;
+ add_join_rel(root, joinrelset);
return joinrel;
}
@@ -1772,3 +1933,626 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->target = target;
+ result->input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /*
+ * The "input_rows" field should be set by caller.
+ */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index d0cc14f11d..eaa57fea0f 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -426,7 +426,6 @@ get_sortgrouplist_exprs(List *sgClauses, List *targetList)
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
@@ -801,6 +800,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index be059f0de0..22f7e1222b 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -4876,8 +4876,12 @@ examine_variable(PlannerInfo *root, Node *node, int varRelid,
case BMS_MULTIPLE:
if (varRelid == 0)
{
+ RelOptInfoSet *relset;
+
/* treat it as a variable of a join relation */
- vardata->rel = find_join_rel(root, varnos);
+ relset = find_join_rel(root, varnos);
+ if (relset)
+ vardata->rel = relset->rel_plain;
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
@@ -5735,7 +5739,13 @@ find_join_input_rel(PlannerInfo *root, Relids relids)
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
- rel = find_join_rel(root, relids);
+ {
+ RelOptInfoSet *relset;
+
+ relset = find_join_rel(root, relids);
+ if (relset)
+ rel = relset->rel_plain;
+ }
break;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index ae925c1650..15a0b194c5 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -986,6 +986,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 10dac60cd3..d88fd6d71d 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -223,6 +223,8 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
+ T_RelOptInfoSet,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -266,6 +268,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 3430061361..4623d8b761 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -278,6 +278,8 @@ typedef struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -290,7 +292,7 @@ typedef struct PlannerInfo
List *part_schemes; /* Canonicalised partition schemes used in the
* query. */
- List *initial_rels; /* RelOptInfos we are now trying to join */
+ List *initial_rels; /* RelOptInfoSets we are now trying to join */
/* Use fetch_upper_rel() to get any particular upper rel */
List *upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
@@ -304,6 +306,12 @@ typedef struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -729,6 +737,82 @@ typedef struct RelOptInfo
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
/*
+ * RelAggInfo
+ *
+ * RelOptInfo needs information contained here if its paths should be
+ * aggregated.
+ *
+ * "target" will be used as pathtarget for aggregation if "explicit
+ * aggregation" is applied to base relation or join. The same target will also
+ * --- if the relation is a join --- be used to joinin grouped path to a
+ * non-grouped one.
+ *
+ * These targets contain plain-Var grouping expressions and Aggrefs which.
+ * Once Aggref is evaluated, its value is passed to the upper paths w/o being
+ * evaluated again.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ struct PathTarget *target; /* Target for grouped paths.. */
+
+ struct PathTarget *input; /* pathtarget of paths that generate input for
+ * aggregation paths. */
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+} RelAggInfo;
+
+/*
+ * RelOptInfoSet
+ *
+ * Structure to use where RelOptInfo for other UpperRelationKind than
+ * UPPERREL_FINAL may be needed. Currently we only need UPPERREL_FINAL
+ * and UPPERREL_PARTIAL_GROUP_AGG.
+ *
+ * rel_plain should always be initialized. Code paths that do not
+ * distinguish between plain and grouped relation use rel_plain,
+ * e.g. has_legal_joinclause().
+ *
+ * agg_info is initialized iff rel_grouped is.
+ *
+ * XXX Is RelOptInfoPair more appropriate name?
+ */
+typedef struct RelOptInfoSet
+{
+ NodeTag type;
+
+ RelOptInfo *rel_plain; /* UPPERREL_FINAL */
+ RelOptInfo *rel_grouped; /* UPPERREL_PARTIAL_GROUP_AGG */
+
+ RelAggInfo *agg_info; /* Information needed to create rel_grouped
+ * and its paths. It seems just convenient to
+ * store it here. */
+} RelOptInfoSet;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
@@ -2205,6 +2289,26 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
@@ -2311,6 +2415,11 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * agg_info passes information necessary for joins to produce partially
+ * grouped data.
+ * rel_agg_input describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
typedef struct JoinPathExtraData
{
@@ -2320,6 +2429,8 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ RelAggInfo *agg_info;
+ RelOptInfo *rel_agg_input;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 5c8580e478..64b5e94048 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -88,4 +88,6 @@ extern Query *inline_set_returning_function(PlannerInfo *root,
extern List *expand_function_arguments(List *args, Oid result_type,
HeapTuple func_tuple);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index e7005b4a0c..32dffc7cc7 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -72,6 +72,7 @@ extern PGDLLIMPORT bool enable_partitionwise_aggregate;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
extern PGDLLIMPORT bool enable_partition_pruning;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int constraint_exclusion;
extern double clamp_row_est(double nrows);
@@ -174,7 +175,8 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
SpecialJoinInfo *sjinfo,
List *restrictlist,
SemiAntiJoinFactors *semifactors);
-extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
+extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
List *param_clauses);
diff --git a/src/include/optimizer/geqo.h b/src/include/optimizer/geqo.h
index dc88fbbc1b..701aeac7bb 100644
--- a/src/include/optimizer/geqo.h
+++ b/src/include/optimizer/geqo.h
@@ -78,7 +78,7 @@ typedef struct
/* routines in geqo_main.c */
-extern RelOptInfo *geqo(PlannerInfo *root,
+extern RelOptInfoSet *geqo(PlannerInfo *root,
int number_of_rels, List *initial_rels);
/* routines in geqo_eval.c */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index bd905d3328..f420a2e13a 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -35,7 +35,8 @@ extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers);
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern Path *create_samplescan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer);
extern IndexPath *create_index_path(PlannerInfo *root,
@@ -49,7 +50,9 @@ extern IndexPath *create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path);
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern BitmapHeapPath *create_bitmap_heap_path(PlannerInfo *root,
RelOptInfo *rel,
Path *bitmapqual,
@@ -123,6 +126,7 @@ extern Relids calc_non_nestloop_required_outer(Path *outer_path, Path *inner_pat
extern NestPath *create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -134,6 +138,7 @@ extern NestPath *create_nestloop_path(PlannerInfo *root,
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -148,6 +153,7 @@ extern MergePath *create_mergejoin_path(PlannerInfo *root,
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -196,6 +202,12 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -263,14 +275,16 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern void build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *rel);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
-extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern RelOptInfoSet *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -297,5 +311,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 666217c189..1df5dfebad 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -29,7 +30,9 @@ extern PGDLLIMPORT int min_parallel_index_scan_size;
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
RelOptInfo *rel,
Index rti,
- RangeTblEntry *rte);
+ RangeTblEntry *rte,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
/* Hook for plugins to get control in add_paths_to_joinrel() */
@@ -42,19 +45,24 @@ typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
-typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
- int levels_needed,
- List *initial_rels);
+typedef RelOptInfoSet *(*join_search_hook_type) (PlannerInfo *root,
+ int levels_needed,
+ List *initial_rels);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
-extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
+extern RelOptInfoSet *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
-extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
+extern RelOptInfoSet *standard_join_search(PlannerInfo *root,
+ int levels_needed,
List *initial_rels);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+
+extern bool add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -70,7 +78,8 @@ extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
* indxpath.c
* routines to generate index paths
*/
-extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
+extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
@@ -101,7 +110,9 @@ extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
/*
* joinrels.c
@@ -109,7 +120,7 @@ extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
*/
extern void join_search_one_level(PlannerInfo *root, int level);
extern RelOptInfo *make_join_rel(PlannerInfo *root,
- RelOptInfo *rel1, RelOptInfo *rel2);
+ RelOptInfoSet *relset1, RelOptInfoSet *relset2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_dangerous_phv(PlannerInfo *root,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index bec0c38617..dc050aec4f 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -37,7 +37,7 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
/*
* prototypes for plan/planmain.c
*/
-extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
+extern RelOptInfoSet *query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra);
/*
@@ -78,6 +78,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 8b967f9583..b39ef1ac1c 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -16,7 +16,6 @@
#include "nodes/relation.h"
-
extern TargetEntry *tlist_member(Expr *node, List *targetlist);
extern TargetEntry *tlist_member_ignore_relabel(Expr *node, List *targetlist);
@@ -41,7 +40,6 @@ extern Node *get_sortgroupclause_expr(SortGroupClause *sgClause,
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
-
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
extern SortGroupClause *get_sortgroupref_clause_noerr(Index sortref,
@@ -65,6 +63,13 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..b3a97f86d6
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,217 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index cc0bbf5db9..74a22fd83b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -98,6 +98,9 @@ test: rules psql_crosstab amutils
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0c10c7100c..af387fd9e3 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -135,6 +135,7 @@ test: rules
test: psql_crosstab
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-02-04 06:03 Michael Paquier <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Michael Paquier @ 2019-02-04 06:03 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: [email protected]; pgsql-hackers
On Tue, Jan 15, 2019 at 03:06:18PM +0100, Antonin Houska wrote:
> This is the next version. A few more comments below.
Latest patch set fails to apply, so moved to next CF, waiting on
author.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-02-05 09:40 Antonin Houska <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2019-02-05 09:40 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: [email protected]; pgsql-hackers
Michael Paquier <[email protected]> wrote:
> On Tue, Jan 15, 2019 at 03:06:18PM +0100, Antonin Houska wrote:
> > This is the next version. A few more comments below.
>
> Latest patch set fails to apply, so moved to next CF, waiting on
> author.
Rebased.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] v10-001-Export_estimate_hashagg_tablesize.patch (4.5K, ../../25767.1549359615@localhost/2-v10-001-Export_estimate_hashagg_tablesize.patch)
download | inline diff:
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b849ae03b8..6f3a4a063c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -146,9 +146,6 @@ static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd,
List *target_list);
-static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
@@ -3634,40 +3631,6 @@ get_number_of_groups(PlannerInfo *root,
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
-static Size
-estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
-{
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
-}
-
-/*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index e8f51d2d0d..be059f0de0 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -114,6 +114,7 @@
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+#include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -3902,6 +3903,39 @@ estimate_hash_bucket_stats(PlannerInfo *root, Node *hashkey, double nbuckets,
ReleaseVariableStats(vardata);
}
+/*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+Size
+estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+{
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+}
/*-------------------------------------------------------------------------
*
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
index 5cc4cf15e2..e65ab13b1a 100644
--- a/src/include/utils/selfuncs.h
+++ b/src/include/utils/selfuncs.h
@@ -213,6 +213,9 @@ extern void estimate_hash_bucket_stats(PlannerInfo *root,
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
[text/x-diff] v10-002-Introduce_make_join_rel_common.patch (2.5K, ../../25767.1549359615@localhost/3-v10-002-Introduce_make_join_rel_common.patch)
download | inline diff:
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 38eeb23d81..0caf67a916 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -37,6 +37,7 @@ static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -651,21 +652,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -748,6 +740,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen
+ * when working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
+/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
[text/x-diff] v10-003-Introduce_estimate_join_rows.patch (2.0K, ../../25767.1549359615@localhost/4-v10-003-Introduce_estimate_join_rows.patch)
download | inline diff:
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 99c5ad9b4a..353dc116f4 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2291,6 +2291,20 @@ cost_group(Path *path, PlannerInfo *root,
}
/*
+ * estimate_join_rows
+ * Set rows of a join path according to its parent relation or according
+ * to parameters.
+ */
+static void
+estimate_join_rows(PlannerInfo *root, Path *path)
+{
+ if (path->param_info)
+ path->rows = path->param_info->ppi_rows;
+ else
+ path->rows = path->parent->rows;
+}
+
+/*
* initial_cost_nestloop
* Preliminary estimate of the cost of a nestloop join path.
*
@@ -2411,10 +2425,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->path.param_info)
- path->path.rows = path->path.param_info->ppi_rows;
- else
- path->path.rows = path->path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2857,10 +2868,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3287,10 +3295,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
[text/x-diff] v10-004-Agg_pushdown_basic.patch (188.6K, ../../25767.1549359615@localhost/5-v10-004-Agg_pushdown_basic.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 7fcac81e2e..5752eba563 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -665,7 +665,7 @@ postgresGetForeignRelSize(PlannerInfo *root,
}
/* Estimate baserel size as best we can with local statistics. */
- set_baserel_size_estimates(root, baserel);
+ set_baserel_size_estimates(root, baserel, NULL);
/* Fill in basically-bogus cost estimates for use later. */
estimate_path_cost_size(root, baserel, NIL, NIL,
@@ -2068,7 +2068,10 @@ postgresPlanDirectModify(PlannerInfo *root,
/* Safe to fetch data about the target foreign rel */
if (fscan->scan.scanrelid == 0)
{
- foreignrel = find_join_rel(root, fscan->fs_relids);
+ RelOptInfoSet *foreignrelset;
+
+ foreignrelset = find_join_rel(root, fscan->fs_relids);
+ foreignrel = foreignrelset->rel_plain;
/* We should have a rel for this foreign join. */
Assert(foreignrel);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index b44ead269f..eabea5dce1 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2198,8 +2198,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2340,6 +2340,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5108,6 +5122,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f97cf37f1f..a859d60f43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2198,6 +2198,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2205,6 +2206,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2415,6 +2417,22 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+/*
+ * Note we do NOT print plain_rel, else we'd be in infinite recursion.
+ */
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2503,6 +2521,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4041,6 +4071,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4056,6 +4089,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..4633fb8768 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index e07bab831e..4beac45fdf 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -182,13 +182,15 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
for (rel_count = 0; rel_count < num_gene; rel_count++)
{
int cur_rel_index;
+ RelOptInfoSet *cur_relset;
RelOptInfo *cur_rel;
Clump *cur_clump;
/* Get the next input relation */
cur_rel_index = (int) tour[rel_count];
- cur_rel = (RelOptInfo *) list_nth(private->initial_rels,
- cur_rel_index - 1);
+ cur_relset = (RelOptInfoSet *) list_nth(private->initial_rels,
+ cur_rel_index - 1);
+ cur_rel = cur_relset->rel_plain;
/* Make it into a single-rel clump */
cur_clump = (Clump *) palloc(sizeof(Clump));
@@ -251,16 +253,24 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
desirable_join(root, old_clump->joinrel, new_clump->joinrel))
{
RelOptInfo *joinrel;
+ RelOptInfoSet *oldset,
+ *newset;
/*
* Construct a RelOptInfo representing the join of these two input
* relations. Note that we expect the joinrel not to exist in
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
+ *
+ * TODO Consider using make_join_rel_common() (possibly renamed)
+ * here instead of wrapping the joinrels into RelOptInfoSet.
*/
- joinrel = make_join_rel(root,
- old_clump->joinrel,
- new_clump->joinrel);
+ oldset = makeNode(RelOptInfoSet);
+ oldset->rel_plain = old_clump->joinrel;
+ newset = makeNode(RelOptInfoSet);
+ newset->rel_plain = new_clump->joinrel;
+
+ joinrel = make_join_rel(root, oldset, newset);
/* Keep searching if join order is not valid */
if (joinrel)
diff --git a/src/backend/optimizer/geqo/geqo_main.c b/src/backend/optimizer/geqo/geqo_main.c
index dc51829dec..b4a4f4ff27 100644
--- a/src/backend/optimizer/geqo/geqo_main.c
+++ b/src/backend/optimizer/geqo/geqo_main.c
@@ -63,7 +63,7 @@ static int gimme_number_generations(int pool_size);
* similar to a constrained Traveling Salesman Problem (TSP)
*/
-RelOptInfo *
+RelOptInfoSet *
geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
{
GeqoPrivateData private;
@@ -74,6 +74,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
Pool *pool;
int pool_size,
number_generations;
+ RelOptInfoSet *result;
#ifdef GEQO_DEBUG
int status_interval;
@@ -296,7 +297,9 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
/* ... clear root pointer to our private storage */
root->join_search_private = NULL;
- return best_rel;
+ result = makeNode(RelOptInfoSet);
+ result->rel_plain = best_rel;
+ return result;
}
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 55b871c02c..7f3ac0a6de 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -59,6 +59,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -74,16 +75,18 @@ static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
static void set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelAggInfo *agg_info);
static void create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel);
static void set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
@@ -118,10 +121,11 @@ static void set_cte_pathlist(PlannerInfo *root, RelOptInfo *rel,
static void set_namedtuplestore_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
-static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
+static RelOptInfoSet *make_rel_from_joinlist(PlannerInfo *root,
+ List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
@@ -148,10 +152,10 @@ static bool apply_child_basequals(PlannerInfo *root, RelOptInfo *rel,
* Finds all possible access paths for executing a query, returning a
* single rel that represents the join of all base rels in the query.
*/
-RelOptInfo *
+RelOptInfoSet *
make_one_rel(PlannerInfo *root, List *joinlist)
{
- RelOptInfo *rel;
+ RelOptInfoSet *rel;
Index rti;
double total_pages;
@@ -229,7 +233,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
/*
* The result should join all and only the query's base rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->rel_plain->relids, root->all_baserels));
return rel;
}
@@ -320,7 +324,7 @@ set_base_rel_sizes(PlannerInfo *root)
if (root->glob->parallelModeOK)
set_rel_consider_parallel(root, rel, rte);
- set_rel_size(root, rel, rti, rte);
+ set_rel_size(root, rel, rti, rte, NULL);
}
}
@@ -349,18 +353,29 @@ set_base_rel_pathlists(PlannerInfo *root)
if (rel->reloptkind != RELOPT_BASEREL)
continue;
- set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti]);
+ set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti],
+ NULL, NULL);
}
}
/*
* set_rel_size
* Set size estimates for a base relation
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info)
{
+ bool grouped = agg_info != NULL;
+
+ /*
+ * Aggregate push-down is currently used only for relations.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (rel->reloptkind == RELOPT_BASEREL &&
relation_excluded_by_constraints(root, rel, rte))
{
@@ -379,7 +394,13 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support grouped
+ * append relation.
+ */
+ Assert(!grouped);
set_append_rel_size(root, rel, rti, rte);
}
else
@@ -389,7 +410,12 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Grouped foreign table is not supported.
+ */
+ Assert(!grouped);
set_foreign_size(root, rel, rte);
}
else if (rte->relkind == RELKIND_PARTITIONED_TABLE)
@@ -397,18 +423,27 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* A partitioned table without any partitions is marked as
* a dummy rel.
+ *
+ * Grouped partitioned table is not supported.
*/
+ Assert(!grouped);
+
set_dummy_rel_pathlist(rel);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation
+ *
+ * Grouped tablesample rel is not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_size(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_size(root, rel, rte);
+ set_plain_rel_size(root, rel, rte, agg_info);
}
break;
case RTE_SUBQUERY:
@@ -464,18 +499,49 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* set_rel_pathlist
* Build access paths for a base relation
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
+ bool grouped = rel_grouped != NULL;
+
+ /*
+ * Aggregate push-down is currently implemented for RTE_RELATION only.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
+ if (grouped)
+ {
+ /*
+ * Do not apply AggPath if this is the only relation of the query:
+ * create_grouping_paths() will do so anyway.
+ */
+ if (bms_equal(rel->relids, root->all_baserels))
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
+ }
+
if (IS_DUMMY_REL(rel))
{
/* We already proved the relation empty, so nothing more to do */
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support append
+ * relation.
+ */
+ Assert(!grouped);
set_append_rel_pathlist(root, rel, rti, rte);
}
else
@@ -485,18 +551,29 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_foreign_pathlist(root, rel, rte);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation.
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_pathlist(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_pathlist(root, rel, rte);
+ set_plain_rel_pathlist(root, rel, rte, rel_grouped,
+ agg_info);
}
break;
case RTE_SUBQUERY:
@@ -541,9 +618,12 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* Also, if this is the topmost scan/join rel (that is, the only baserel),
* we postpone this until the final scan/join targelist is available (see
* grouping_planner).
+ *
+ * Note on aggregation push-down: parallel paths are not supported so far.
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON &&
+ !grouped)
generate_gather_paths(root, rel, false);
/*
@@ -552,10 +632,24 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* add_path(), or delete or modify paths added by the core code.
*/
if (set_rel_pathlist_hook)
- (*set_rel_pathlist_hook) (root, rel, rti, rte);
+ (*set_rel_pathlist_hook) (root, rel, rti, rte, rel_grouped,
+ agg_info);
+
+ /*
+ * The grouped relation is not guaranteed to have any paths. If the
+ * pathlist is empty, there's no point in calling set_cheapest().
+ */
+ if (rel_grouped && rel_grouped->pathlist == NIL)
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
/* Now find the cheapest of the paths for this rel */
- set_cheapest(rel);
+ if (!grouped)
+ set_cheapest(rel);
+ else
+ set_cheapest(rel_grouped);
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -565,9 +659,13 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_size
* Set size estimates for a plain relation (no subquery, no inheritance)
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
-set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
+ RelAggInfo *agg_info)
{
/*
* Test any partial indexes of rel for applicability. We must do this
@@ -576,7 +674,7 @@ set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
check_index_predicates(root, rel);
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, agg_info);
}
/*
@@ -757,11 +855,19 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_pathlist
* Build access paths for a plain relation (no subquery, no inheritance)
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
-set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
Relids required_outer;
+ Path *seq_path;
+ bool grouped = rel_grouped != NULL;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
@@ -770,18 +876,38 @@ set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
*/
required_outer = rel->lateral_relids;
- /* Consider sequential scan */
- add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
+ /* Consider sequential scan. */
+ seq_path = create_seqscan_path(root, rel, required_outer, 0, rel_grouped,
+ agg_info);
+
+ if (!grouped)
+ add_path(rel, seq_path);
+
+ /*
+ * It's probably not good idea to repeat hashed aggregation with different
+ * parameters, so check if there are no parameters.
+ */
+ else if (required_outer == NULL)
+ {
+ /*
+ * Only AGG_HASHED is suitable here as it does not expect the input
+ * set to be sorted.
+ */
+ add_grouped_path(root, rel_grouped, seq_path, AGG_HASHED, agg_info);
+ }
/* If appropriate, consider parallel sequential scan */
- if (rel->consider_parallel && required_outer == NULL)
+ if (rel->consider_parallel && required_outer == NULL && !grouped)
create_plain_partial_paths(root, rel);
- /* Consider index scans */
- create_index_paths(root, rel);
+ /*
+ * Consider index scans, possibly including the grouped paths.
+ */
+ create_index_paths(root, rel, rel_grouped, agg_info);
/* Consider TID scans */
- create_tidscan_paths(root, rel);
+ if (!grouped)
+ create_tidscan_paths(root, rel);
}
/*
@@ -793,15 +919,55 @@ create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
- parallel_workers = compute_parallel_worker(rel, rel->pages, -1,
- max_parallel_workers_per_gather);
+ parallel_workers = compute_parallel_worker(rel, rel->pages, -1, max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
- add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
+ add_partial_path(rel, create_seqscan_path(root, rel, NULL,
+ parallel_workers, NULL, NULL));
+}
+
+/*
+ * Apply aggregation to a subpath and add the AggPath to the pathlist.
+ *
+ * The return value tells whether the path was added to the pathlist.
+ *
+ * XXX Pass the plain rel and fetch the grouped from rel->grouped? How about
+ * subroutines then?
+ */
+bool
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+ /*
+ * Repeated creation of hash table does not sound like a good idea. Caller
+ * should avoid asking us to do so.
+ */
+ Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ {
+ add_path(rel, (Path *) agg_path);
+
+ return true;
+ }
+
+ return false;
}
/*
@@ -841,7 +1007,7 @@ set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
rel->tuples = tuples;
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -1130,7 +1296,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's size.
*/
- set_rel_size(root, childrel, childRTindex, childRTE);
+ set_rel_size(root, childrel, childRTindex, childRTE, NULL);
/*
* It is possible that constraint exclusion detected a contradiction
@@ -1279,7 +1445,7 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's access paths.
*/
- set_rel_pathlist(root, childrel, childRTindex, childRTE);
+ set_rel_pathlist(root, childrel, childRTindex, childRTE, NULL, NULL);
/*
* If child is dummy, ignore it.
@@ -2569,7 +2735,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
-static RelOptInfo *
+static RelOptInfoSet *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
@@ -2595,13 +2761,35 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
- RelOptInfo *thisrel;
+ RelOptInfoSet *thisrel;
if (IsA(jlnode, RangeTblRef))
{
- int varno = ((RangeTblRef *) jlnode)->rtindex;
+ RangeTblRef *rtref = castNode(RangeTblRef, jlnode);
+ int varno = rtref->rtindex;
+ RangeTblEntry *rte = root->simple_rte_array[varno];
+ RelOptInfo *rel_grouped;
+
+ thisrel = makeNode(RelOptInfoSet);
+ thisrel->rel_plain = find_base_rel(root, varno);
- thisrel = find_base_rel(root, varno);
+ /*
+ * Create the grouped counterpart of rel_plain if possible.
+ */
+ build_simple_grouped_rel(root, thisrel);
+ rel_grouped = thisrel->rel_grouped;
+
+ /*
+ * If succeeded, do what should already have been done for the
+ * plain relation.
+ */
+ if (rel_grouped)
+ {
+ set_rel_size(root, rel_grouped, varno, rte,
+ thisrel->agg_info);
+ set_rel_pathlist(root, thisrel->rel_plain, varno, rte,
+ rel_grouped, thisrel->agg_info);
+ }
}
else if (IsA(jlnode, List))
{
@@ -2623,7 +2811,7 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
/*
* Single joinlist node, so we're done.
*/
- return (RelOptInfo *) linitial(initial_rels);
+ return (RelOptInfoSet *) linitial(initial_rels);
}
else
{
@@ -2637,11 +2825,19 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
root->initial_rels = initial_rels;
if (join_search_hook)
- return (*join_search_hook) (root, levels_needed, initial_rels);
+ return (*join_search_hook) (root, levels_needed,
+ initial_rels);
else if (enable_geqo && levels_needed >= geqo_threshold)
+ {
+ /*
+ * TODO Teach GEQO about grouped relations. Don't forget that
+ * pathlist can be NIL before set_cheapest() gets called.
+ */
return geqo(root, levels_needed, initial_rels);
+ }
else
- return standard_join_search(root, levels_needed, initial_rels);
+ return standard_join_search(root, levels_needed,
+ initial_rels);
}
}
@@ -2674,11 +2870,11 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
-RelOptInfo *
+RelOptInfoSet *
standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
{
int lev;
- RelOptInfo *rel;
+ RelOptInfoSet *result;
/*
* This function cannot be invoked recursively within any one planning
@@ -2720,13 +2916,20 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
+ *
+ * Neither partitionwise join nor parallel processing is supported for
+ * grouped relations so far.
*/
foreach(lc, root->join_rel_level[lev])
{
- rel = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *rel_set;
+ RelOptInfo *rel_plain;
+
+ rel_set = (RelOptInfoSet *) lfirst(lc);
+ rel_plain = rel_set->rel_plain;
/* Create paths for partitionwise joins. */
- generate_partitionwise_join_paths(root, rel);
+ generate_partitionwise_join_paths(root, rel_plain);
/*
* Except for the topmost scan/join rel, consider gathering
@@ -2734,10 +2937,33 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* once we know the final targetlist (see grouping_planner).
*/
if (lev < levels_needed)
- generate_gather_paths(root, rel, false);
+ generate_gather_paths(root, rel_plain, false);
/* Find and save the cheapest paths for this rel */
- set_cheapest(rel);
+ set_cheapest(rel_plain);
+
+ if (rel_set->rel_grouped)
+ {
+ RelOptInfo *rel_grouped;
+
+ rel_grouped = rel_set->rel_grouped;
+
+ /* Partial paths not supported yet. */
+ Assert(rel_grouped->partial_pathlist == NIL);
+
+ if (rel_grouped->pathlist != NIL)
+ set_cheapest(rel_grouped);
+ else
+ {
+ /*
+ * When checking whether the grouped relation can supply
+ * any input paths to join, test for existence of the
+ * relation will be easier than a test of pathlist.
+ */
+ pfree(rel_grouped);
+ rel_set->rel_grouped = NULL;
+ }
+ }
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -2752,11 +2978,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
Assert(list_length(root->join_rel_level[levels_needed]) == 1);
- rel = (RelOptInfo *) linitial(root->join_rel_level[levels_needed]);
+ result = (RelOptInfoSet *) linitial(root->join_rel_level[levels_needed]);
root->join_rel_level = NULL;
- return rel;
+ return result;
}
/*****************************************************************************
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index f1baa36886..5c8a009d09 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2329,13 +2329,23 @@ cost_group(Path *path, PlannerInfo *root,
/*
* estimate_join_rows
* Set rows of a join path according to its parent relation or according
- * to parameters.
+ * to parameters. If agg_info is passed, the join path is grouped.
*/
static void
-estimate_join_rows(PlannerInfo *root, Path *path)
+estimate_join_rows(PlannerInfo *root, Path *path, RelAggInfo *agg_info)
{
if (path->param_info)
+ {
path->rows = path->param_info->ppi_rows;
+ if (agg_info)
+ {
+ double nrows;
+
+ nrows = estimate_num_groups(root, agg_info->group_exprs,
+ path->rows, NULL);
+ path->rows = clamp_row_est(nrows);
+ }
+ }
else
path->rows = path->parent->rows;
}
@@ -2461,7 +2471,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2904,7 +2914,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3331,7 +3341,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -4336,27 +4346,59 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
* restriction clauses).
* width: the estimated average output tuple width in bytes.
* baserestrictcost: estimated cost of evaluating baserestrictinfo clauses.
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
void
-set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
+set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info)
{
double nrows;
+ bool grouped = agg_info != NULL;
/* Should only be applied to base relations */
Assert(rel->relid > 0);
- nrows = rel->tuples *
- clauselist_selectivity(root,
- rel->baserestrictinfo,
- 0,
- JOIN_INNER,
- NULL);
+ if (!grouped)
+ {
+ nrows = rel->tuples *
+ clauselist_selectivity(root,
+ rel->baserestrictinfo,
+ 0,
+ JOIN_INNER,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
- rel->rows = clamp_row_est(nrows);
+ /*
+ * Only set the estimate for grouped base rel if aggregation can take
+ * place. (Aggregation is the only way to build grouped base relation.)
+ */
+ else if (!bms_equal(rel->relids, root->all_baserels))
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ nrows = estimate_num_groups(root,
+ agg_info->group_exprs,
+ agg_info->input_rows,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
- set_rel_width(root, rel);
+ /*
+ * The grouped target should have the cost and width set immediately on
+ * creation, see create_rel_agg_info().
+ */
+ if (!grouped)
+ set_rel_width(root, rel);
+#ifdef USE_ASSERT_CHECKING
+ else
+ Assert(rel->reltarget->width > 0);
+#endif
}
/*
@@ -4920,7 +4962,7 @@ set_subquery_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4958,7 +5000,7 @@ set_function_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4980,7 +5022,7 @@ set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 100;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5011,7 +5053,7 @@ set_values_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = list_length(rte->values_lists);
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5049,7 +5091,7 @@ set_cte_size_estimates(PlannerInfo *root, RelOptInfo *rel, double cte_rows)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5082,7 +5124,7 @@ set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 1000;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5105,7 +5147,7 @@ set_result_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 1;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5329,11 +5371,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5364,6 +5406,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 3454f12912..0452d415fb 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -65,7 +65,6 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
-
/*
* process_equivalence
* The given clause has a mergejoinable operator and can be applied without
@@ -2511,3 +2510,137 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 7e1a3908f1..c4080b0587 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -77,13 +77,14 @@ typedef struct
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
-
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths);
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -92,7 +93,9 @@ static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -100,19 +103,24 @@ static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static bool eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
List *indexjoinclauses);
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths);
+ List **bitindexpaths, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop);
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -216,6 +224,10 @@ static Const *string_to_const(const char *str, Oid datatype);
*
* 'rel' is the relation for which we want to generate index paths
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Note: check_index_predicates() must have been run previously for this rel.
*
* Note: in cases involving LATERAL references in the relation's tlist, it's
@@ -228,7 +240,8 @@ static Const *string_to_const(const char *str, Oid datatype);
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
-create_index_paths(PlannerInfo *root, RelOptInfo *rel)
+create_index_paths(PlannerInfo *root, RelOptInfo *rel, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
List *bitindexpaths;
@@ -238,6 +251,7 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
IndexClauseSet jclauseset;
IndexClauseSet eclauseset;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/* Skip the whole mess if no indexes */
if (rel->indexlist == NIL)
@@ -273,8 +287,8 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
- get_index_paths(root, rel, index, &rclauseset,
- &bitindexpaths);
+ get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
+ rel_grouped, agg_info);
/*
* Identify the join clauses that can match the index. For the moment
@@ -303,15 +317,24 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
&rclauseset,
&jclauseset,
&eclauseset,
- &bitjoinpaths);
+ &bitjoinpaths,
+ rel_grouped,
+ agg_info);
}
/*
+ * It does not seem too efficient to aggregate the individual paths and
+ * then AND them together.
+ */
+ if (grouped)
+ return;
+
+ /*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
- indexpaths = generate_bitmap_or_paths(root, rel,
- rel->baserestrictinfo, NIL);
+ indexpaths = generate_bitmap_or_paths(root, rel, rel->baserestrictinfo,
+ NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
@@ -433,6 +456,10 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* 'jclauseset' is the collection of indexable simple join clauses
* 'eclauseset' is the collection of indexable clauses from EquivalenceClasses
* '*bitindexpaths' is the list to add bitmap paths to
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
@@ -440,7 +467,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
int considered_clauses = 0;
List *considered_relids = NIL;
@@ -476,7 +505,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
jclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
/* Consider each applicable eclass join clause */
considered_clauses += list_length(eclauseset->indexclauses[indexcol]);
consider_index_join_outer_rels(root, rel, index,
@@ -484,7 +515,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
eclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -499,6 +532,10 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
* 'indexjoinclauses' is a list of RestrictInfos for join clauses
* 'considered_clauses' is the total number of clauses considered (so far)
* '*considered_relids' is a list of all relids sets already considered
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
@@ -509,7 +546,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
ListCell *lc;
@@ -576,7 +615,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
bms_union(clause_relids, oldrelids),
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
/* Also try this set of relids by itself */
@@ -584,7 +625,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
clause_relids,
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -600,6 +643,10 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
* 'bitindexpaths', 'considered_relids' as above
* 'relids' is the current set of relids to consider (the target rel plus
* one or more outer rels)
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -609,7 +656,9 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexClauseSet clauseset;
int indexcol;
@@ -666,7 +715,8 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
- get_index_paths(root, rel, index, &clauseset, bitindexpaths);
+ get_index_paths(root, rel, index, &clauseset, bitindexpaths,
+ rel_grouped, agg_info);
/*
* Remember we considered paths for this set of relids. We use lcons not
@@ -716,7 +766,6 @@ bms_equal_any(Relids relids, List *relids_list)
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
@@ -731,16 +780,23 @@ bms_equal_any(Relids relids, List *relids_list)
* paths, and then make a separate attempt to include them in bitmap paths.
* Furthermore, we should consider excluding lower-order ScalarArrayOpExpr
* quals so as to create ordered paths.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
bool skip_nonnative_saop = false;
bool skip_lower_saop = false;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/*
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
@@ -753,7 +809,40 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- &skip_lower_saop);
+ &skip_lower_saop,
+ rel_grouped,
+ agg_info);
+
+ /*
+ * If the relation is grouped, apply the partial aggregation.
+ *
+ * Only AGG_SORTED strategy is used, so we ignore bitmap paths, as well as
+ * indexes that can only produce them.
+ */
+ if (grouped && index->amhasgettuple)
+ {
+ foreach(lc, indexpaths)
+ {
+ IndexPath *ipath = (IndexPath *) lfirst(lc);
+ Path *subpath = &ipath->path;
+
+ if (subpath->pathkeys != NIL)
+ {
+ AggPath *agg_path;
+
+ agg_path = create_agg_sorted_path(root,
+ subpath,
+ agg_info);
+ if (agg_path)
+ add_path(rel_grouped, (Path *) agg_path);
+ }
+ }
+
+ /*
+ * Nothing more to do for grouped relation.
+ */
+ return;
+ }
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
@@ -768,7 +857,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- NULL));
+ NULL,
+ rel_grouped,
+ agg_info));
}
/*
@@ -808,7 +899,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
false,
ST_BITMAPSCAN,
NULL,
- NULL);
+ NULL,
+ rel_grouped,
+ agg_info);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
@@ -852,7 +945,15 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
- * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
+ * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
+ * XXX When enabling the aggregate push-down for partial paths, we'll need the
+ * function to return the aggregation input paths in a separate list instead
+ * of calling add_partial_path() on them immediately.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -860,7 +961,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop)
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *result = NIL;
IndexPath *ipath;
@@ -877,6 +980,12 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool include_partial;
+
+ /*
+ * Grouped partial paths are not supported yet.
+ */
+ include_partial = rel_grouped == NULL;
/*
* Check that index supports the desired scan type(s)
@@ -1046,14 +1155,16 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
*/
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1069,7 +1180,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth using
@@ -1103,11 +1216,13 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1121,7 +1236,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth
@@ -1243,6 +1360,8 @@ build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
useful_predicate,
ST_BITMAPSCAN,
NULL,
+ NULL,
+ NULL,
NULL);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d8ff4bf432..f99d6b98a9 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -120,7 +120,9 @@ add_paths_to_joinrel(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
@@ -143,6 +145,8 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.agg_info = agg_info;
+ extra.rel_agg_input = rel_agg_input;
/*
* See if the inner relation is provably unique for this outer rel.
@@ -376,6 +380,8 @@ try_nestloop_path(PlannerInfo *root,
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Paths are parameterized by top-level parents, so run parameterization
@@ -422,10 +428,37 @@ try_nestloop_path(PlannerInfo *root,
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * If the join output should be aggregated, the precheck is skipped
+ * because it makes little sense to compare the new join path to existing,
+ * already aggregated paths. Since we don't have row count estimate yet,
+ * it's hard to involve AggPath in the precheck.
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ /*
+ * If the join output is subject to aggregation, the path must
+ * generate aggregation input.
+ */
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
@@ -447,21 +480,57 @@ try_nestloop_path(PlannerInfo *root,
}
}
- add_path(joinrel, (Path *)
- create_nestloop_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer));
+ path = (Path *) create_nestloop_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /*
+ * Non-grouped rel had to be passed to create_nestloop_path() so
+ * that row estimate reflects the aggregation input data, however
+ * even the aggregation input path should eventually be owned by
+ * the grouped joinrel.
+ */
+ path->parent = joinrel;
+
+ /*
+ * Try both AGG_HASHED and AGG_SORTED aggregation.
+ *
+ * AGG_HASHED should not be parameterized because we don't want to
+ * create the hashtable again for each set of parameters.
+ */
+ if (required_outer == NULL)
+ success = add_grouped_path(root, joinrel, path,
+ AGG_HASHED, extra->agg_info);
+
+ /*
+ * Don't try AGG_SORTED if add_grouped_path() would reject it
+ * anyway.
+ */
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root, joinrel, path, AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
- /* Waste no memory when we reject a path here */
+ /* Waste no memory when we reject path(s) here */
bms_free(required_outer);
}
}
@@ -538,6 +607,7 @@ try_partial_nestloop_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_nestloop_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -568,8 +638,11 @@ try_mergejoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
- if (is_partial)
+ if (!grouped && is_partial)
{
try_partial_mergejoin_path(root,
joinrel,
@@ -617,26 +690,73 @@ try_mergejoin_path(PlannerInfo *root,
outersortkeys, innersortkeys,
extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_mergejoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer,
- mergeclauses,
- outersortkeys,
- innersortkeys));
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_mergejoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer,
+ mergeclauses,
+ outersortkeys,
+ innersortkeys);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ if (required_outer == NULL)
+ success = add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_HASHED,
+ extra->agg_info);
+
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -700,6 +820,7 @@ try_partial_mergejoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_mergejoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -729,6 +850,9 @@ try_hashjoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Check to see if proposed path is still parameterized, and reject if the
@@ -745,30 +869,79 @@ try_hashjoin_path(PlannerInfo *root,
}
/*
+ * Parameterized execution of grouped path would mean repeated hashing of
+ * the output of the hashjoin output, so forget about AGG_HASHED if there
+ * are any parameters. And AGG_SORTED makes no sense because the hash join
+ * output is not sorted.
+ */
+ if (required_outer && grouped)
+ return;
+
+ /*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- NIL, required_outer))
+
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ NIL, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_hashjoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- false, /* parallel_hash */
- extra->restrictlist,
- required_outer,
- hashclauses));
+ PathTarget *target;
+ Path *path = NULL;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_hashjoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ false, /* parallel_hash */
+ extra->restrictlist,
+ required_outer,
+ hashclauses);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ /*
+ * As the hashjoin path is not sorted, only try AGG_HASHED.
+ */
+ if (add_grouped_path(root, joinrel, path, AGG_HASHED,
+ extra->agg_info))
+ success = true;
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -824,6 +997,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_hashjoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -892,6 +1066,7 @@ sort_inner_and_outer(PlannerInfo *root,
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped = extra->agg_info != NULL;
/*
* We only consider the cheapest-total-cost input paths, since we are
@@ -1051,7 +1226,7 @@ sort_inner_and_outer(PlannerInfo *root,
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
- if (cheapest_partial_outer && cheapest_safe_inner)
+ if (!grouped && cheapest_partial_outer && cheapest_safe_inner)
try_partial_mergejoin_path(root,
joinrel,
cheapest_partial_outer,
@@ -1341,6 +1516,7 @@ match_unsorted_outer(PlannerInfo *root,
Path *inner_cheapest_total = innerrel->cheapest_total_path;
Path *matpath = NULL;
ListCell *lc1;
+ bool grouped = extra->agg_info != NULL;
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
@@ -1516,7 +1692,8 @@ match_unsorted_outer(PlannerInfo *root,
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
- if (joinrel->consider_parallel &&
+ if (!grouped &&
+ joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index fc9eb95f5a..8014f10ac3 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -24,24 +24,33 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
static void make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
+static void set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
-static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input);
+static void make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist);
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
@@ -54,7 +63,6 @@ static SpecialJoinInfo *build_child_join_sjinfo(PlannerInfo *root,
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
-
/*
* join_search_one_level
* Consider ways to produce join relations containing exactly 'level'
@@ -89,7 +97,15 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel;
+
+ /*
+ * Both rel_plain and rel_grouped should have the same clauses, so it
+ * does not matter which one we use here. The only difference is that
+ * rel_grouped is not guaranteed to exist, so use rel_plain.
+ */
+ old_rel = old_relset->rel_plain;
if (old_rel->joininfo != NIL || old_rel->has_eclass_joins ||
has_join_restriction(root, old_rel))
@@ -115,7 +131,7 @@ join_search_one_level(PlannerInfo *root, int level)
other_rels = list_head(joinrels[1]);
make_rels_by_clause_joins(root,
- old_rel,
+ old_relset,
other_rels);
}
else
@@ -133,7 +149,7 @@ join_search_one_level(PlannerInfo *root, int level)
* avoid the duplicated effort.
*/
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
}
@@ -159,7 +175,8 @@ join_search_one_level(PlannerInfo *root, int level)
foreach(r, joinrels[k])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel = old_relset->rel_plain;
ListCell *other_rels;
ListCell *r2;
@@ -179,7 +196,8 @@ join_search_one_level(PlannerInfo *root, int level)
for_each_cell(r2, other_rels)
{
- RelOptInfo *new_rel = (RelOptInfo *) lfirst(r2);
+ RelOptInfoSet *new_relset = (RelOptInfoSet *) lfirst(r2);
+ RelOptInfo *new_rel = new_relset->rel_plain;
if (!bms_overlap(old_rel->relids, new_rel->relids))
{
@@ -191,7 +209,7 @@ join_search_one_level(PlannerInfo *root, int level)
if (have_relevant_joinclause(root, old_rel, new_rel) ||
have_join_order_restriction(root, old_rel, new_rel))
{
- (void) make_join_rel(root, old_rel, new_rel);
+ (void) make_join_rel(root, old_relset, new_relset);
}
}
}
@@ -225,10 +243,10 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
@@ -274,25 +292,31 @@ join_search_one_level(PlannerInfo *root, int level)
* 'other_rels': the first cell in a linked list containing the other
* rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it
* will work for joining to joinrels too.
*/
static void
make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(old_rel->relids, other_rel->relids) &&
(have_relevant_joinclause(root, old_rel, other_rel) ||
have_join_order_restriction(root, old_rel, other_rel)))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
@@ -308,27 +332,62 @@ make_rels_by_clause_joins(PlannerInfo *root,
* 'other_rels': the first cell of a linked list containing the
* other rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it would
* work for joining to joinrels too.
*/
static void
make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(other_rel->relids, old_rel->relids))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
+/*
+ * Set joinrel's reltarget according to agg_info and estimate the number of
+ * rows.
+ */
+static void
+set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info)
+{
+ Assert(agg_info != NULL);
+
+ /*
+ * build_join_rel() does not create the target for grouped relation.
+ */
+ Assert(joinrel->reltarget == NULL);
+
+ joinrel->reltarget = agg_info->target;
+
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied to
+ * one of the input rels. Is this worth extra effort, e.g. maintaining a
+ * separate RelOptInfo for each case (one difficulty that would introduce
+ * is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+}
/*
* join_is_legal
@@ -661,9 +720,18 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation
+ * should not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -671,10 +739,15 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
SpecialJoinInfo sjinfo_data;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool do_aggregate = rel_agg_input != NULL;
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
+ /* do_aggregate implies the output to be grouped. */
+ Assert(!do_aggregate || grouped);
+
/* Construct Relids set that identifies the joinrel. */
joinrelids = bms_union(rel1->relids, rel2->relids);
@@ -724,7 +797,18 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
+
+ if (grouped)
+ {
+ /*
+ * Make sure the grouped joinrel has reltarget initialized. Caller
+ * should supply the target for grouped relation, so build_join_rel()
+ * should have omitted its creation.
+ */
+ if (joinrel->reltarget == NULL)
+ set_grouped_joinrel_target(root, joinrel, agg_info);
+ }
/*
* If we've already proven this join is empty, we needn't consider any
@@ -738,13 +822,72 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
bms_free(joinrelids);
return joinrel;
}
+static void
+make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped = NULL;
+ RelOptInfo *rel2_grouped = NULL;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /*
+ * Retrieve the grouped relations.
+ *
+ * Dummy rel may indicates a join relation that is able to generate
+ * grouped paths as such (i.e. it has valid agg_info), but for which the
+ * path actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ if (relset1->rel_grouped)
+ rel1_grouped = relset1->rel_grouped;
+ if (relset2->rel_grouped)
+ rel2_grouped = relset2->rel_grouped;
+
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /*
+ * Nothing to do?
+ */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, relset2->rel_plain, agg_info,
+ NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, relset1->rel_plain, rel2_grouped, agg_info,
+ NULL);
+}
+
/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
@@ -753,14 +896,101 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but there's no useful grouped input
+ * relation.
+ *
+ * Only the plain relation is returned.
+ *
+ * TODO geqo is the only caller interested in the result. We'll need to return
+ * RelOptInfoSet if geqo adopts the aggregate pushdown technique.
*/
RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info;
+ RelOptInfoSet *joinrelset;
+ RelOptInfo *joinrel_plain;
+ RelOptInfo *rel1 = relset1->rel_plain;
+ RelOptInfo *rel2 = relset2->rel_plain;
+
+ /* 1) form the plain join. */
+ joinrel_plain = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ /*
+ * If the same grouped joinrel was already formed, just with the base rels
+ * divided between rel1 and rel2 in a different way, we should already
+ * have the matching agg_info.
+ */
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrelset = find_join_rel(root, joinrelids);
+
+ /*
+ * At the moment we know that non-grouped join exists, so the containing
+ * joinrelset should have been fetched.
+ */
+ Assert(joinrelset != NULL);
+
+ if (joinrelset->rel_grouped != NULL)
+ {
+ /*
+ * The same agg_info should be used for all the rels consisting of
+ * exactly joinrelids.
+ */
+ agg_info = joinrelset->agg_info;
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ if (agg_info != NULL)
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * Cannot we build grouped join?
+ */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the final one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_common_grouped(root, relset1, relset2, agg_info);
+
+ return joinrel_plain;
}
/*
@@ -773,8 +1003,11 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist)
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
+ bool grouped = agg_info != NULL;
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -804,10 +1037,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
@@ -821,10 +1054,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
@@ -835,10 +1068,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
/*
* If there are join quals that aren't mergeable or hashable, we
@@ -871,7 +1104,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
/*
@@ -894,10 +1127,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
break;
case JOIN_ANTI:
@@ -912,7 +1145,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
default:
/* other values not expected here */
@@ -920,8 +1153,16 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
}
- /* Apply partitionwise join technique, if possible. */
- try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
+ /*
+ * The aggregate push-down feature currently does not support
+ * partition-wise aggregation.
+ */
+ if (grouped)
+ return;
+
+ /* Apply partition-wise join technique, if possible. */
+ try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
}
@@ -1121,7 +1362,8 @@ has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel)
foreach(lc, root->initial_rels)
{
- RelOptInfo *rel2 = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *relset2 = lfirst_node(RelOptInfoSet, lc);
+ RelOptInfo *rel2 = relset2->rel_plain;
/* ignore rels that are already in "rel" */
if (bms_overlap(rel->relids, rel2->relids))
@@ -1452,7 +1694,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
- child_restrictlist);
+ child_restrictlist, NULL, NULL);
}
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 1b4f7db649..d22de6a909 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -901,6 +901,18 @@ use_physical_tlist(PlannerInfo *root, Path *path, int flags)
}
}
+ /*
+ * If aggregate was pushed down, the target can contain aggregates. The
+ * original target must be preserved then.
+ */
+ foreach(lc, path->pathtarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, Aggref))
+ return false;
+ }
+
return true;
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index d6ffa7869d..d7008b33b9 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -47,6 +47,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -97,10 +99,9 @@ static void check_hashjoinable(RestrictInfo *restrictinfo);
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
- * every non-join RTE that is used in the query. Therefore, this routine
- * is the only place that should call build_simple_rel with reloptkind
- * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
- * "other rel" RelOptInfos for the members of any appendrels we find here.)
+ * every non-grouped non-join RTE that is used in the query. (Note:
+ * build_simple_rel recurses internally to build "other rel" RelOptInfos for
+ * the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
@@ -242,6 +243,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 86617099df..1f2f5b0832 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -349,6 +349,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
List *tlist;
NullTest *ntest;
SortGroupClause *sortcl;
+ RelOptInfoSet *final_relset;
RelOptInfo *final_rel;
Path *sorted_path;
Cost path_cost;
@@ -442,7 +443,8 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_relset = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_rel = final_relset->rel_plain;
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 3cedd01c98..ad5b18eb61 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -53,13 +53,14 @@
* qp_callback once we have completed merging the query's equivalence classes.
* (We cannot construct canonical pathkeys until that's done.)
*/
-RelOptInfo *
+RelOptInfoSet *
query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra)
{
Query *parse = root->parse;
List *joinlist;
RelOptInfo *final_rel;
+ RelOptInfoSet *final_relset;
/*
* Init planner lists to empty.
@@ -77,6 +78,7 @@ query_planner(PlannerInfo *root, List *tlist,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -147,7 +149,10 @@ query_planner(PlannerInfo *root, List *tlist,
*/
(*qp_callback) (root, qp_extra);
- return final_rel;
+ final_relset = makeNode(RelOptInfoSet);
+ final_relset->rel_plain = final_rel;
+
+ return final_relset;
}
}
}
@@ -260,14 +265,25 @@ query_planner(PlannerInfo *root, List *tlist,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
+ /*
* Ready to do the primary planning.
*/
- final_rel = make_one_rel(root, joinlist);
+ final_relset = make_one_rel(root, joinlist);
+ final_rel = final_relset->rel_plain;
/* Check that we got at least one usable path */
if (!final_rel || !final_rel->cheapest_total_path ||
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
- return final_rel;
+ return final_relset;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 0614f31be4..fcfac56564 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -149,7 +149,7 @@ static double get_number_of_groups(PlannerInfo *root,
grouping_sets_data *gd,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -162,7 +162,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -630,6 +630,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1683,6 +1684,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ RelOptInfoSet *current_relset;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
ListCell *lc;
@@ -1902,8 +1904,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, tlist,
- standard_qp_callback, &qp_extra);
+ current_relset = query_planner(root, tlist,
+ standard_qp_callback, &qp_extra);
+ current_rel = current_relset->rel_plain;
/*
* Convert the query's result tlist into PathTarget format.
@@ -2043,7 +2046,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
if (have_grouping)
{
current_rel = create_grouping_paths(root,
- current_rel,
+ current_relset,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3674,13 +3677,14 @@ get_number_of_groups(PlannerInfo *root,
*/
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd)
{
Query *parse = root->parse;
+ RelOptInfo *input_rel = input_relset->rel_plain;
RelOptInfo *grouped_rel;
RelOptInfo *partially_grouped_rel;
@@ -3765,7 +3769,7 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_relset, grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -3921,13 +3925,14 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
* function creates, or to NULL if it doesn't create one.
*/
static void
-create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
GroupPathExtraData *extra,
RelOptInfo **partially_grouped_rel_p)
{
+ RelOptInfo *input_rel = input_relset->rel_plain;
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *partially_grouped_rel = NULL;
double dNumGroups;
@@ -3971,13 +3976,25 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ RelOptInfo *input_rel_grouped = input_relset->rel_grouped;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ input_rel_grouped->pathlist != NIL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -3986,6 +4003,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4010,10 +4044,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -6092,7 +6130,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
comparisonCost = 2.0 * (indexExprCost.startup + indexExprCost.per_tuple);
/* Estimate the cost of seq scan + sort */
- seqScanPath = create_seqscan_path(root, rel, NULL, 0);
+ seqScanPath = create_seqscan_path(root, rel, NULL, 0, NULL, NULL);
cost_sort(&seqScanAndSortPath, root, NIL,
seqScanPath->total_cost, rel->tuples, rel->reltarget->width,
comparisonCost, maintenance_work_mem, -1.0);
@@ -6101,7 +6139,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
indexScanPath = create_index_path(root, indexInfo,
NIL, NIL, NIL, NIL, NIL,
ForwardScanDirection, false,
- NULL, 1.0, false);
+ NULL, 1.0, false, NULL, NULL);
return (seqScanAndSortPath.total_cost < indexScanPath->path.total_cost);
}
@@ -7114,6 +7152,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
for (cnt_parts = 0; cnt_parts < nparts; cnt_parts++)
{
RelOptInfo *child_input_rel = input_rel->part_rels[cnt_parts];
+ RelOptInfoSet *child_input_relset;
PathTarget *child_target = copy_pathtarget(target);
AppendRelInfo **appinfos;
int nappinfos;
@@ -7172,12 +7211,17 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
continue;
}
+ child_input_relset = makeNode(RelOptInfoSet);
+ child_input_relset->rel_plain = child_input_rel;
+
/* Create grouping paths for this child relation. */
- create_ordinary_grouping_paths(root, child_input_rel,
+ create_ordinary_grouping_paths(root, child_input_relset,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ pfree(child_input_relset);
+
if (child_partially_grouped_rel)
{
partially_grouped_live_children =
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0213a37670..d618007929 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2300,6 +2300,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index aebe162713..9886c18bbd 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -891,6 +891,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index b57de6b4c6..50bdf69b2e 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -17,6 +17,8 @@
#include <math.h>
#include "miscadmin.h"
+#include "access/sysattr.h"
+#include "catalog/pg_constraint.h"
#include "foreign/fdwapi.h"
#include "nodes/extensible.h"
#include "nodes/nodeFuncs.h"
@@ -58,7 +60,6 @@ static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
-
/*****************************************************************************
* MISC. PATH UTILITIES
*****************************************************************************/
@@ -953,13 +954,15 @@ add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
*/
Path *
create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers)
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info)
{
Path *pathnode = makeNode(Path);
pathnode->pathtype = T_SeqScan;
- pathnode->parent = rel;
- pathnode->pathtarget = rel->reltarget;
+ pathnode->parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
@@ -1019,6 +1022,10 @@ create_samplescan_path(PlannerInfo *root, RelOptInfo *rel, Relids required_outer
* estimates of caching behavior.
* 'partial_path' is true if constructing a parallel index scan path.
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Returns the new path node.
*/
IndexPath *
@@ -1033,7 +1040,9 @@ create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path)
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexPath *pathnode = makeNode(IndexPath);
RelOptInfo *rel = index->rel;
@@ -1041,8 +1050,9 @@ create_index_path(PlannerInfo *root,
*indexqualcols;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
- pathnode->path.parent = rel;
- pathnode->path.pathtarget = rel->reltarget;
+ pathnode->path.parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->path.pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
@@ -1186,8 +1196,8 @@ create_bitmap_or_path(PlannerInfo *root,
* Creates a path corresponding to a scan by TID, returning the pathnode.
*/
TidPath *
-create_tidscan_path(PlannerInfo *root, RelOptInfo *rel, List *tidquals,
- Relids required_outer)
+create_tidscan_path(PlannerInfo *root, RelOptInfo *rel,
+ List *tidquals, Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
@@ -1342,7 +1352,8 @@ append_startup_cost_compare(const void *a, const void *b)
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
- * pathnode.
+ * pathnode. target can be supplied by caller. If NULL is passed, the field
+ * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
@@ -1533,7 +1544,9 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
MemoryContext oldcontext;
int numCols;
- /* Caller made a mistake if subpath isn't cheapest_total ... */
+ /*
+ * Caller made a mistake if subpath isn't cheapest_total.
+ */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
@@ -2180,6 +2193,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
* relations.
*
* 'joinrel' is the join relation.
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_nestloop
* 'extra' contains various information about the join
@@ -2194,6 +2208,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
NestPath *
create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2234,7 +2249,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
- pathnode->path.pathtarget = joinrel->reltarget;
+ pathnode->path.pathtarget = target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2266,6 +2281,7 @@ create_nestloop_path(PlannerInfo *root,
* two relations
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_mergejoin
* 'extra' contains various information about the join
@@ -2282,6 +2298,7 @@ create_nestloop_path(PlannerInfo *root,
MergePath *
create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2298,7 +2315,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2334,6 +2351,7 @@ create_mergejoin_path(PlannerInfo *root,
* Creates a pathnode corresponding to a hash join between two relations.
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_hashjoin
* 'extra' contains various information about the join
@@ -2348,6 +2366,7 @@ create_mergejoin_path(PlannerInfo *root,
HashPath *
create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2362,7 +2381,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2444,8 +2463,8 @@ create_projection_path(PlannerInfo *root,
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
- if (is_projection_capable_path(subpath) ||
- equal(oldtarget->exprs, target->exprs))
+ if ((is_projection_capable_path(subpath) ||
+ equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
@@ -2830,8 +2849,7 @@ create_agg_path(PlannerInfo *root,
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
@@ -2864,6 +2882,152 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
@@ -3548,7 +3712,8 @@ reparameterize_path(PlannerInfo *root, Path *path,
switch (path->pathtype)
{
case T_SeqScan:
- return create_seqscan_path(root, rel, required_outer, 0);
+ return create_seqscan_path(root, rel, required_outer, 0, NULL,
+ NULL);
case T_SampleScan:
return (Path *) create_samplescan_path(root, rel, required_outer);
case T_IndexScan:
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index f04c6b76f4..c54964ce64 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,16 +17,21 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
@@ -34,7 +39,7 @@
typedef struct JoinHashEntry
{
Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
+ RelOptInfoSet *join_relset;
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
@@ -54,7 +59,7 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
-static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
+static void add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
@@ -63,6 +68,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -323,6 +331,97 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation.
+ */
+void
+build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *relset)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ rel_plain = relset->rel_plain;
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return;;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, relset->rel_plain);
+ if (agg_info == NULL)
+ return;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * The aggregation paths will get their input target from agg_info, so
+ * store it too.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ relset->rel_grouped = rel_grouped;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+ relset->agg_info = agg_info;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -371,16 +470,16 @@ build_join_rel_hash(PlannerInfo *root)
/* Insert all the already-existing joinrels */
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *relset = (RelOptInfoSet *) lfirst(l);
JoinHashEntry *hentry;
bool found;
hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
+ &(relset->rel_plain->relids),
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->join_relset = relset;
}
root->join_rel_hash = hashtab;
@@ -391,7 +490,7 @@ build_join_rel_hash(PlannerInfo *root)
* Returns relation entry corresponding to 'relids' (a set of RT indexes),
* or NULL if none exists. This is for join relations.
*/
-RelOptInfo *
+RelOptInfoSet *
find_join_rel(PlannerInfo *root, Relids relids)
{
/*
@@ -419,7 +518,13 @@ find_join_rel(PlannerInfo *root, Relids relids)
HASH_FIND,
NULL);
if (hentry)
- return hentry->join_rel;
+ {
+ RelOptInfoSet *result = hentry->join_relset;;
+
+ /* The plain relation should always be there. */
+ Assert(result->rel_plain != NULL);
+ return result;
+ }
}
else
{
@@ -427,10 +532,10 @@ find_join_rel(PlannerInfo *root, Relids relids)
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *result = (RelOptInfoSet *) lfirst(l);
- if (bms_equal(rel->relids, relids))
- return rel;
+ if (bms_equal(result->rel_plain->relids, relids))
+ return result;
}
}
@@ -493,23 +598,26 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset)
{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
+ /*
+ * GEQO requires us to append the new joinrel to the end of the list!
+ */
+ root->join_rel_list = lappend(root->join_rel_list, joinrelset);
/* store it into the auxiliary hashtable if there is one. */
if (root->join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
+ Relids relids = joinrelset->rel_plain->relids;
hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
+ &relids,
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = joinrel;
+ hentry->join_relset = joinrelset;
}
}
@@ -525,6 +633,8 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' contains information needed for the join to form grouped
+ * paths. If NULL, the join is not grouped.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -535,10 +645,20 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
+ RelOptInfoSet *joinrelset;
+ bool new_set = false;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool create_target;
+
+ /*
+ * Target for grouped relation will be supplied by caller.
+ */
+ create_target = !grouped;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -546,7 +666,19 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrelset = find_join_rel(root, joinrelids);
+ if (joinrelset == NULL)
+ {
+ /*
+ * The plain joinrel should be the first one to be added to the set.
+ */
+ Assert(!grouped);
+
+ joinrelset = makeNode(RelOptInfoSet);
+ new_set = true;
+ }
+
+ joinrel = !grouped ? joinrelset->rel_plain : joinrelset->rel_grouped;
if (joinrel)
{
@@ -573,7 +705,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
- joinrel->reltarget = create_empty_pathtarget();
+ joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
@@ -638,9 +770,13 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (create_target)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -676,45 +812,68 @@ build_join_rel(PlannerInfo *root,
sjinfo->jointype);
/*
- * Set estimates of the joinrel's size.
+ * Assign the joinrel to the set.
*/
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
+ if (!grouped)
+ joinrelset->rel_plain = joinrel;
+ else
+ {
+ joinrelset->rel_grouped = joinrel;
+ joinrelset->agg_info = agg_info;
+ }
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (new_set)
+ {
+ /* Add the joinrelset to the PlannerInfo. */
+ add_join_rel(root, joinrelset);
- /* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ /*
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only
+ * assert <=.
+ */
+ if (root->join_rel_level)
+ {
+ Assert(root->join_cur_level > 0);
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
+ root->join_rel_level[root->join_cur_level] =
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrelset);
+ }
+ }
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Set estimates of the joinrel's size.
+ *
+ * XXX set_joinrel_size_estimates() claims to need reltarget but it does
+ * not seem to actually use it. Should we call it unconditionally so that
+ * callers of build_join_rel() do not have to care?
*/
- if (root->join_rel_level)
+ if (create_target)
{
- Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
- root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
}
return joinrel;
@@ -741,6 +900,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
JoinType jointype)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
+ RelOptInfoSet *joinrelset = makeNode(RelOptInfoSet);
AppendRelInfo **appinfos;
int nappinfos;
@@ -852,7 +1012,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
Assert(!find_join_rel(root, joinrel->relids));
/* Add the relation to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ joinrelset->rel_plain = joinrel;
+ add_join_rel(root, joinrelset);
return joinrel;
}
@@ -1749,3 +1910,626 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->target = target;
+ result->input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /*
+ * The "input_rows" field should be set by caller.
+ */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 14d1c67a94..bf820fe66c 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -427,7 +427,6 @@ get_sortgrouplist_exprs(List *sgClauses, List *targetList)
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
@@ -802,6 +801,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 67e268cc7a..0cc56f8b66 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -4884,8 +4884,12 @@ examine_variable(PlannerInfo *root, Node *node, int varRelid,
case BMS_MULTIPLE:
if (varRelid == 0)
{
+ RelOptInfoSet *relset;
+
/* treat it as a variable of a join relation */
- vardata->rel = find_join_rel(root, varnos);
+ relset = find_join_rel(root, varnos);
+ if (relset)
+ vardata->rel = relset->rel_plain;
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
@@ -5743,7 +5747,13 @@ find_join_input_rel(PlannerInfo *root, Relids relids)
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
- rel = find_join_rel(root, relids);
+ {
+ RelOptInfoSet *relset;
+
+ relset = find_join_rel(root, relids);
+ if (relset)
+ rel = relset->rel_plain;
+ }
break;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 8681ada33a..52b1204345 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1007,6 +1007,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index e215ad4978..96fa16c2d3 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -223,6 +223,8 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
+ T_RelOptInfoSet,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -266,6 +268,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index d3c477a542..60b059ad05 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -284,6 +284,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -296,7 +298,7 @@ struct PlannerInfo
List *part_schemes; /* Canonicalised partition schemes used in the
* query. */
- List *initial_rels; /* RelOptInfos we are now trying to join */
+ List *initial_rels; /* RelOptInfoSets we are now trying to join */
/* Use fetch_upper_rel() to get any particular upper rel */
List *upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
@@ -310,6 +312,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -734,6 +742,82 @@ typedef struct RelOptInfo
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
/*
+ * RelAggInfo
+ *
+ * RelOptInfo needs information contained here if its paths should be
+ * aggregated.
+ *
+ * "target" will be used as pathtarget for aggregation if "explicit
+ * aggregation" is applied to base relation or join. The same target will also
+ * --- if the relation is a join --- be used to joinin grouped path to a
+ * non-grouped one.
+ *
+ * These targets contain plain-Var grouping expressions and Aggrefs which.
+ * Once Aggref is evaluated, its value is passed to the upper paths w/o being
+ * evaluated again.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ struct PathTarget *target; /* Target for grouped paths.. */
+
+ struct PathTarget *input; /* pathtarget of paths that generate input for
+ * aggregation paths. */
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+} RelAggInfo;
+
+/*
+ * RelOptInfoSet
+ *
+ * Structure to use where RelOptInfo for other UpperRelationKind than
+ * UPPERREL_FINAL may be needed. Currently we only need UPPERREL_FINAL
+ * and UPPERREL_PARTIAL_GROUP_AGG.
+ *
+ * rel_plain should always be initialized. Code paths that do not
+ * distinguish between plain and grouped relation use rel_plain,
+ * e.g. has_legal_joinclause().
+ *
+ * agg_info is initialized iff rel_grouped is.
+ *
+ * XXX Is RelOptInfoPair more appropriate name?
+ */
+typedef struct RelOptInfoSet
+{
+ NodeTag type;
+
+ RelOptInfo *rel_plain; /* UPPERREL_FINAL */
+ RelOptInfo *rel_grouped; /* UPPERREL_PARTIAL_GROUP_AGG */
+
+ RelAggInfo *agg_info; /* Information needed to create rel_grouped
+ * and its paths. It seems just convenient to
+ * store it here. */
+} RelOptInfoSet;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
@@ -2214,6 +2298,26 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
@@ -2320,6 +2424,11 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * agg_info passes information necessary for joins to produce partially
+ * grouped data.
+ * rel_agg_input describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
typedef struct JoinPathExtraData
{
@@ -2329,6 +2438,8 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ RelAggInfo *agg_info;
+ RelOptInfo *rel_agg_input;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 23073c0402..25ab90e7f2 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -56,4 +56,6 @@ extern void CommuteRowCompareExpr(RowCompareExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index ac6de0f6be..b9757c31fd 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -64,6 +64,7 @@ extern PGDLLIMPORT bool enable_partitionwise_aggregate;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
extern PGDLLIMPORT bool enable_partition_pruning;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int constraint_exclusion;
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
@@ -171,7 +172,8 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
SpecialJoinInfo *sjinfo,
List *restrictlist,
SemiAntiJoinFactors *semifactors);
-extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
+extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
List *param_clauses);
diff --git a/src/include/optimizer/geqo.h b/src/include/optimizer/geqo.h
index 5b3327665e..8a4054b59a 100644
--- a/src/include/optimizer/geqo.h
+++ b/src/include/optimizer/geqo.h
@@ -78,7 +78,7 @@ typedef struct
/* routines in geqo_main.c */
-extern RelOptInfo *geqo(PlannerInfo *root,
+extern RelOptInfoSet *geqo(PlannerInfo *root,
int number_of_rels, List *initial_rels);
/* routines in geqo_eval.c */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index d0c8f99d0a..88b460ee58 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -35,7 +35,8 @@ extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers);
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern Path *create_samplescan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer);
extern IndexPath *create_index_path(PlannerInfo *root,
@@ -49,7 +50,9 @@ extern IndexPath *create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path);
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern BitmapHeapPath *create_bitmap_heap_path(PlannerInfo *root,
RelOptInfo *rel,
Path *bitmapqual,
@@ -127,6 +130,7 @@ extern Relids calc_non_nestloop_required_outer(Path *outer_path, Path *inner_pat
extern NestPath *create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -138,6 +142,7 @@ extern NestPath *create_nestloop_path(PlannerInfo *root,
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -152,6 +157,7 @@ extern MergePath *create_mergejoin_path(PlannerInfo *root,
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -200,6 +206,12 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -267,14 +279,16 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern void build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *rel);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
-extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern RelOptInfoSet *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -300,5 +314,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 1b02b3b889..ad9439439d 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -29,7 +30,9 @@ extern PGDLLIMPORT int min_parallel_index_scan_size;
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
RelOptInfo *rel,
Index rti,
- RangeTblEntry *rte);
+ RangeTblEntry *rte,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
/* Hook for plugins to get control in add_paths_to_joinrel() */
@@ -42,19 +45,24 @@ typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
-typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
- int levels_needed,
- List *initial_rels);
+typedef RelOptInfoSet *(*join_search_hook_type) (PlannerInfo *root,
+ int levels_needed,
+ List *initial_rels);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
-extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
+extern RelOptInfoSet *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
-extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
+extern RelOptInfoSet *standard_join_search(PlannerInfo *root,
+ int levels_needed,
List *initial_rels);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+
+extern bool add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -70,7 +78,8 @@ extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
* indxpath.c
* routines to generate index paths
*/
-extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
+extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
@@ -101,7 +110,9 @@ extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
/*
* joinrels.c
@@ -109,7 +120,7 @@ extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
*/
extern void join_search_one_level(PlannerInfo *root, int level);
extern RelOptInfo *make_join_rel(PlannerInfo *root,
- RelOptInfo *rel1, RelOptInfo *rel2);
+ RelOptInfoSet *relset1, RelOptInfoSet *relset2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_dangerous_phv(PlannerInfo *root,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..75e48185a3 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -27,7 +27,7 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
/*
* prototypes for plan/planmain.c
*/
-extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
+extern RelOptInfoSet *query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra);
/*
@@ -68,6 +68,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 58db79203b..fa927cd6f4 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -49,6 +49,13 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4051a4ad4e..22058b1f91 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -104,6 +104,9 @@ test: rules psql_crosstab amutils
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index ac1ea622d6..bc1ed68a9e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -136,6 +136,7 @@ test: rules
test: psql_crosstab
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..b3a97f86d6
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,217 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-02-21 20:18 Tom Lane <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Tom Lane @ 2019-02-21 20:18 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Michael Paquier <[email protected]>; pgsql-hackers
Antonin Houska <[email protected]> writes:
> Michael Paquier <[email protected]> wrote:
>> Latest patch set fails to apply, so moved to next CF, waiting on
>> author.
> Rebased.
This is in need of rebasing again :-(. I went ahead and pushed the 001
part, since that seemed fairly uncontroversial. (Note that I changed
estimate_hashagg_tablesize's result type to double on the way; you
probably want to make corresponding adjustments in your patch.)
I did not spend a whole lot of time looking at the patch today, but
I'm still pretty distressed at the data structures you've chosen.
I remain of the opinion that a grouped relation and a base relation
are, er, unrelated, even if they happen to share the same relid set.
So I do not see the value of the RelOptInfoSet struct you propose here,
and I definitely don't think there's any value in having, eg,
create_seqscan_path or create_index_path dealing with this stuff.
I also don't like changing create_nestloop_path et al to take a PathTarget
rather than using the RelOptInfo's pathtarget; IMO, it's flat out wrong
for a path to generate a tlist different from what its parent RelOptInfo
says that the relation produces.
I think basically the way this ought to work is that we generate baserel
paths pretty much the same as today, and then we run through the baserels
and see which ones are potentially worth doing partial aggregation on,
and for each one that is, we create a separate "upper relation" RelOptInfo
describing that. The paths for this RelOptInfo would be
partial-aggregation paths using paths from the corresponding baserel as
input. Then we'd run a join search that only considers joining grouped
rels with plain rels (I concur that joining two grouped rels is not worth
coping with, at least for now).
regards, tom lane
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-02-28 16:02 Antonin Houska <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2019-02-28 16:02 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Michael Paquier <[email protected]>; pgsql-hackers
Tom Lane <[email protected]> wrote:
> Antonin Houska <[email protected]> writes:
> > Michael Paquier <[email protected]> wrote:
> >> Latest patch set fails to apply, so moved to next CF, waiting on
> >> author.
>
> > Rebased.
>
> This is in need of rebasing again :-(. I went ahead and pushed the 001
> part, since that seemed fairly uncontroversial.
ok, thanks.
> I did not spend a whole lot of time looking at the patch today, but
> I'm still pretty distressed at the data structures you've chosen.
> I remain of the opinion that a grouped relation and a base relation
> are, er, unrelated, even if they happen to share the same relid set.
> So I do not see the value of the RelOptInfoSet struct you propose here,
ok. As you suggested upthread, I try now to reuse the join_rel_list /
join_rel_hash structures, see v11-001-Introduce_RelInfoList.patch.
> and I definitely don't think there's any value in having, eg,
> create_seqscan_path or create_index_path dealing with this stuff.
Originally I tried to aggregate any path that ever gets passed to agg_path(),
but that's probably not worth the code complexity. Now the partial aggregation
is only applied to paths that survived agg_path() on the plain relation.
> I also don't like changing create_nestloop_path et al to take a PathTarget
> rather than using the RelOptInfo's pathtarget; IMO, it's flat out wrong
> for a path to generate a tlist different from what its parent RelOptInfo
> says that the relation produces.
Likewise, the current patch version is less invasive, so create_nestloop_path
et al are not touched at all.
> I think basically the way this ought to work is that we generate baserel
> paths pretty much the same as today, and then we run through the baserels
> and see which ones are potentially worth doing partial aggregation on,
> and for each one that is, we create a separate "upper relation" RelOptInfo
> describing that. The paths for this RelOptInfo would be
> partial-aggregation paths using paths from the corresponding baserel as
> input. Then we'd run a join search that only considers joining grouped
> rels with plain rels (I concur that joining two grouped rels is not worth
> coping with, at least for now).
make_join_rel() is the core of my implementation: besides joining two plain
relations it tries to join plain relation to grouped one, and also to
aggregate the join of the two plain relations. I consider this approach less
invasive and more efficient than running the whole standard_join_search again
for the grouped rels.
The problem of numeric-like data types (i.e. types for wich equality of two
values of the grouping key does not justify putting them into the same group
because information like scale would be discarded this way) remains open. My
last idea was to add a boolean flag to operator class which tells that
equality implies "bitwise equality", and to disallow aggregate push-down if
SortGroupClause.eqop is in an opclass which has this field FALSE. I'd like to
hear your opinion before I do any coding.
--
Antonin Houska
https://www.cybertec-postgresql.com
Attachments:
[text/x-diff] v11-001-Introduce_RelInfoList.patch (11.6K, ../../23468.1551369774@localhost/2-v11-001-Introduce_RelInfoList.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 6b96e7de0a..4484fb4fbc 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -4906,7 +4906,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 65302fe65b..8b333f069a 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2272,6 +2272,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
}
static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
+static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
WRITE_NODE_TYPE("INDEXOPTINFO");
@@ -4031,6 +4039,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index e07bab831e..82b4f5c56a 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 3cedd01c98..52d9943192 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -67,8 +67,7 @@ query_planner(PlannerInfo *root, List *tlist,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 4130514952..bcf2ffefb4 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -31,11 +31,11 @@
#include "utils/hsearch.h"
-typedef struct JoinHashEntry
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -346,11 +346,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -359,47 +359,50 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
+
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -409,28 +412,32 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids;
- if (bms_equal(rel->relids, relids))
- return rel;
+ Assert(IsA(item, RelOptInfo));
+ item_relids = ((RelOptInfo *) item)->relids;
+
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
@@ -438,6 +445,58 @@ find_join_rel(PlannerInfo *root, Relids relids)
}
/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
+/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
* tables (or joins) belonging to the same server and assigned to the same
@@ -488,32 +547,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
-/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
* creating a new relation entry if none already exists.
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index f9389257c6..8e34496764 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -220,6 +220,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a008ae07da..88a2c20fbe 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -234,15 +234,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -734,6 +728,24 @@ typedef struct RelOptInfo
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
/*
+ * RelInfoList
+ * A list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
[text/x-diff] v11-002-Introduce_make_join_rel_common.patch (2.5K, ../../23468.1551369774@localhost/3-v11-002-Introduce_make_join_rel_common.patch)
download | inline diff:
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index dfbbfdac6d..45255e6ee0 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -38,6 +38,8 @@ static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -657,21 +659,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
- */
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ */
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -754,6 +747,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
+/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
[text/x-diff] v11-003-Agg_pushdown_basic.patch (107.5K, ../../23468.1551369774@localhost/4-v11-003-Agg_pushdown_basic.patch)
download | inline diff:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e15724bb0e..5e2f4388a9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2198,8 +2198,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2340,6 +2340,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5109,6 +5123,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8b333f069a..9e86efe784 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2183,6 +2183,8 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(grouped_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2196,6 +2198,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2203,6 +2206,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2422,6 +2426,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
}
static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
+static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
@@ -2521,6 +2539,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4063,6 +4093,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4081,6 +4114,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..4633fb8768 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 8586b8f3b8..859c0d6397 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -59,6 +59,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -121,6 +122,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -1709,6 +1713,7 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
}
}
+
/*
* generate_mergeappend_paths
* Generate MergeAppend paths for an append relation
@@ -2566,6 +2571,80 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
}
/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
+/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
*
@@ -2605,6 +2684,34 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+
+ /*
+ * Create a grouped relation to facilitate aggregate push-down.
+ * This makes no sense if thisrel is the only relation of the
+ * query.
+ */
+ if (bms_nonempty_difference(root->all_baserels, thisrel->relids))
+ {
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /*
+ * Build grouped relation if thisrel is suitable for partial
+ * aggregation.
+ */
+ rel_grouped = build_simple_grouped_rel(root, varno, &agg_info);
+
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+
+ /* Add the aggregation paths to it. */
+ generate_grouping_paths(root, rel_grouped, thisrel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
+ }
}
else if (IsA(jlnode, List))
{
@@ -2706,6 +2813,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -2742,6 +2850,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 4b9be13f08..47a90c3537 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4355,7 +4355,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5333,11 +5332,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5368,6 +5367,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 61b5b119b0..52d1ddc1a9 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2521,6 +2521,140 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
}
/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
+
+/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
* list. Here, for convenience, we test both simple identity and
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 45255e6ee0..a75fc8b8c6 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -24,6 +24,7 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -39,7 +40,9 @@ static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
- RelOptInfo *rel2);
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -662,9 +665,17 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
- */
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
+ */
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -725,7 +736,7 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -738,8 +749,26 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
@@ -747,6 +776,62 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
+/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
* the two given rels, and add to it path information for paths
@@ -754,14 +839,83 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
* NB: will return NULL if attempted join is not valid. This can happen when
* working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
*/
RelOptInfo *
make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
}
/*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 2afc3f1dfe..c3033b76c5 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -47,6 +47,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -242,6 +244,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 86617099df..474db04c97 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -442,7 +442,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 52d9943192..b344fd75ad 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -47,6 +47,9 @@
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
*
+ * If final_rel_grouped_p is valid, relation containing grouped paths may be
+ * saved to *final_rel_grouped_p.
+ *
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
* plan. This value is *not* available at call time, but is computed by
@@ -55,7 +58,8 @@
*/
RelOptInfo *
query_planner(PlannerInfo *root, List *tlist,
- query_pathkeys_callback qp_callback, void *qp_extra)
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p)
{
Query *parse = root->parse;
List *joinlist;
@@ -68,6 +72,8 @@ query_planner(PlannerInfo *root, List *tlist,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->grouped_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -76,6 +82,7 @@ query_planner(PlannerInfo *root, List *tlist,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -259,6 +266,16 @@ query_planner(PlannerInfo *root, List *tlist,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
+ /*
* Ready to do the primary planning.
*/
final_rel = make_one_rel(root, joinlist);
@@ -268,5 +285,9 @@ query_planner(PlannerInfo *root, List *tlist,
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
+ if (final_rel_grouped_p)
+ *final_rel_grouped_p = find_grouped_rel(root, final_rel->relids,
+ NULL);
+
return final_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index bc81535905..2933bf7560 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -150,6 +150,7 @@ static double get_number_of_groups(PlannerInfo *root,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -163,6 +164,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -630,6 +632,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1820,6 +1823,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *current_rel_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
@@ -1927,7 +1931,8 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* of the query's sort clause, distinct clause, etc.
*/
current_rel = query_planner(root, tlist,
- standard_qp_callback, &qp_extra);
+ standard_qp_callback, &qp_extra,
+ ¤t_rel_grouped);
/*
* Convert the query's result tlist into PathTarget format.
@@ -2070,6 +2075,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
{
current_rel = create_grouping_paths(root,
current_rel,
+ current_rel_grouped,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3701,6 +3707,7 @@ get_number_of_groups(PlannerInfo *root,
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -3791,7 +3798,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel, input_rel_grouped,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -3948,6 +3956,7 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
*/
static void
create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -3997,13 +4006,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4012,6 +4031,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4036,10 +4072,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -7200,6 +7240,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
/* Create grouping paths for this child relation. */
create_ordinary_grouping_paths(root, child_input_rel,
+ NULL,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0213a37670..d618007929 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2300,6 +2300,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index aebe162713..9886c18bbd 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -891,6 +891,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 169e51e792..7248fa0262 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2949,6 +2949,146 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index bcf2ffefb4..190513fb7d 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,18 +17,24 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
typedef struct RelInfoEntry
@@ -63,6 +69,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -323,6 +332,102 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -433,8 +538,12 @@ find_rel_info(RelInfoList *list, Relids relids)
void *item = lfirst(l);
Relids item_relids;
- Assert(IsA(item, RelOptInfo));
- item_relids = ((RelOptInfo *) item)->relids;
+ Assert(IsA(item, RelOptInfo) ||IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
@@ -463,7 +572,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) ||IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -475,7 +584,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -497,6 +610,57 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
}
/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(root->grouped_rel_list, rel);
+ add_rel_info(root->agg_info_list, rel);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(root->grouped_rel_list, relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
+/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
* tables (or joins) belonging to the same server and assigned to the same
@@ -558,6 +722,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -568,10 +733,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -579,7 +746,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -671,9 +839,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -705,49 +885,73 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
-
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
+ }
+
+ /* Set estimates of the joinrel's size. */
+ if (!grouped)
+ {
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
}
return joinrel;
@@ -1791,3 +1995,625 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 14d1c67a94..3b81b39c8e 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -802,6 +802,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 62b34f1b8e..d4e63a54f4 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1008,6 +1008,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 8e34496764..8c15181e78 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -224,6 +224,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -268,6 +269,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 88a2c20fbe..24c5bfafae 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -239,6 +239,23 @@ struct PlannerInfo
struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
+ * grouped_rel_list is a list of RelOptInfos that represent grouped
+ * relations, both base relations and joins. Unlike join_rel_list, base
+ * relations are accepted because grouped base relation is of rather
+ * limited scope, i.e. it's only needed during join search. Thus it does
+ * not deserve separate storage like simple_rel_array.
+ */
+ struct RelInfoList *grouped_rel_list; /* list of grouped relation
+ * RelOptInfos */
+
+ /*
+ * agg_info_list contains one instance of RelAggInfo per an item of
+ * grouped_rel_list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
+
+ /*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
* join_cur_level is the current level. New join-relation RelOptInfos are
@@ -278,6 +295,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -304,6 +323,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -746,6 +771,60 @@ typedef struct RelInfoList
} RelInfoList;
/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
@@ -2259,6 +2338,26 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 5e10fb1d50..db838a3581 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index ac6de0f6be..0871e8a516 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -64,6 +64,7 @@ extern PGDLLIMPORT bool enable_partitionwise_aggregate;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
extern PGDLLIMPORT bool enable_partition_pruning;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int constraint_exclusion;
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 574bb85b50..973ac5f7d5 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -212,6 +212,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -279,14 +287,21 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -312,5 +327,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 040335a7c5..90d2d52660 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -55,6 +56,11 @@ extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..1aa8570d0d 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -28,7 +28,8 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
- query_pathkeys_callback qp_callback, void *qp_extra);
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p);
/*
* prototypes for plan/planagg.c
@@ -68,6 +69,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 58db79203b..fa927cd6f4 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -49,6 +49,13 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..786f4f3cff
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,214 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(9 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Hash
+ -> Seq Scan on agg_pushdown_parent p
+(10 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+(11 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(9 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = x)
+(10 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index a1c90eb905..55cfd3fff7 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -89,7 +90,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(17 rows)
+(18 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4051a4ad4e..22058b1f91 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -104,6 +104,9 @@ test: rules psql_crosstab amutils
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index ac1ea622d6..bc1ed68a9e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -136,6 +136,7 @@ test: rules
test: psql_crosstab
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-07-03 06:26 Richard Guo <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Richard Guo @ 2019-07-03 06:26 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
On Fri, Mar 1, 2019 at 12:01 AM Antonin Houska <[email protected]> wrote:
> Tom Lane <[email protected]> wrote:
>
> > Antonin Houska <[email protected]> writes:
> > > Michael Paquier <[email protected]> wrote:
> > >> Latest patch set fails to apply, so moved to next CF, waiting on
> > >> author.
> >
> > > Rebased.
> >
> > This is in need of rebasing again :-(. I went ahead and pushed the 001
> > part, since that seemed fairly uncontroversial.
>
> ok, thanks.
>
Another rebase is needed for the patches.
Thanks
Richard
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-07-09 13:47 Antonin Houska <[email protected]>
parent: Richard Guo <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2019-07-09 13:47 UTC (permalink / raw)
To: Richard Guo <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Richard Guo <[email protected]> wrote:
> Another rebase is needed for the patches.
Done.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-07-11 10:27 Richard Guo <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Richard Guo @ 2019-07-11 10:27 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
On Tue, Jul 9, 2019 at 9:47 PM Antonin Houska <[email protected]> wrote:
> Richard Guo <[email protected]> wrote:
>
> > Another rebase is needed for the patches.
>
> Done.
>
I didn't fully follow the whole thread and mainly looked into the latest
patch set. So what are the considerations for abandoning the aggmultifn
concept? In my opinion, aggmultifn would enable us to do a lot more
types of transformation. For example, consider the query below:
select sum(foo.c) from foo join bar on foo.b = bar.b group by foo.a, bar.a;
With the latest patch, the plan looks like:
Finalize HashAggregate <------ sum(psum)
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (bar.b = foo.b)
-> Seq Scan on bar
-> Hash
-> Partial HashAggregate <------ sum(foo.c) as psum
Group Key: foo.a, foo.b
-> Seq Scan on foo
If we have aggmultifn, we can perform the query this way:
Finalize HashAggregate <------ sum(foo.c)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on bar
And this way:
Finalize HashAggregate <------ sum(psum)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Partial HashAggregate <------ sum(foo.c) as psum
Group Key: foo.a, foo.b
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on bar
My another question is in function add_grouped_path(), when creating
sorted aggregation path on top of subpath. If the subpath is not sorted,
then the sorted aggregation path would not be generated. Why not in this
case we create a sort path on top of subpath first and then create group
aggregation path on top of the sort path?
Core dump when running one query in agg_pushdown.sql
EXPLAIN ANALYZE
SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
#0 0x00000000006def98 in CheckVarSlotCompatibility (slot=0x0, attnum=1,
vartype=23) at execExprInterp.c:1850
#1 0x00000000006def5d in CheckExprStillValid (state=0x2b63a28,
econtext=0x2ba4958) at execExprInterp.c:1814
#2 0x00000000006dee38 in ExecInterpExprStillValid (state=0x2b63a28,
econtext=0x2ba4958, isNull=0x7fff7cd16a37) at execExprInterp.c:1763
#3 0x00000000007144dd in ExecEvalExpr (state=0x2b63a28,
econtext=0x2ba4958, isNull=0x7fff7cd16a37)
at ../../../src/include/executor/executor.h:288
#4 0x0000000000715475 in ExecIndexEvalRuntimeKeys (econtext=0x2ba4958,
runtimeKeys=0x2b63910, numRuntimeKeys=1) at nodeIndexscan.c:630
#5 0x000000000071533b in ExecReScanIndexScan (node=0x2b62bf8) at
nodeIndexscan.c:568
#6 0x00000000006d4ce6 in ExecReScan (node=0x2b62bf8) at execAmi.c:182
#7 0x00000000007152a0 in ExecIndexScan (pstate=0x2b62bf8) at
nodeIndexscan.c:530
This is really a cool feature. Thank you for working on this.
Thanks
Richard
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-07-12 08:41 Antonin Houska <[email protected]>
parent: Richard Guo <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2019-07-12 08:41 UTC (permalink / raw)
To: Richard Guo <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Richard Guo <[email protected]> wrote:
> I didn't fully follow the whole thread and mainly looked into the
> latest
> patch set. So what are the considerations for abandoning the
> aggmultifn
> concept?
Originally the function was there to support join where both relations are
grouped. My doubts about usefulness of such join started at [1]. (See the
thread referenced from [2].)
> In my opinion, aggmultifn would enable us to do a lot more
> types of transformation. For example, consider the query below:
>
> select sum(foo.c) from foo join bar on foo.b = bar.b group by foo.a,
> bar.a;
>
> With the latest patch, the plan looks like:
>
> Finalize HashAggregate <------ sum(psum)
> Group Key: foo.a, bar.a
> -> Hash Join
> Hash Cond: (bar.b = foo.b)
> -> Seq Scan on bar
> -> Hash
> -> Partial HashAggregate <------ sum(foo.c) as
> psum
> Group Key: foo.a, foo.b
> -> Seq Scan on foo
>
>
> If we have aggmultifn, we can perform the query this way:
>
> Finalize HashAggregate <------ sum(foo.c)*cnt
> Group Key: foo.a, bar.a
> -> Hash Join
> Hash Cond: (foo.b = bar.b)
> -> Seq Scan on foo
> -> Hash
> -> Partial HashAggregate <------ count(*) as cnt
> Group Key: bar.a, bar.b
> -> Seq Scan on bar
Perhaps, but it that would require changes to nodeAgg.c, which I want to avoid
for now.
> And this way:
>
> Finalize HashAggregate <------ sum(psum)*cnt
> Group Key: foo.a, bar.a
> -> Hash Join
> Hash Cond: (foo.b = bar.b)
> -> Partial HashAggregate <------ sum(foo.c) as
> psum
> Group Key: foo.a, foo.b
> -> Seq Scan on foo
> -> Hash
> -> Partial HashAggregate <------ count(*) as cnt
> Group Key: bar.a, bar.b
> -> Seq Scan on bar
This looks like my idea presented in [1], for which there seems to be no
common use case.
> My another question is in function add_grouped_path(), when creating
> sorted aggregation path on top of subpath. If the subpath is not
> sorted,
> then the sorted aggregation path would not be generated. Why not in
> this
> case we create a sort path on top of subpath first and then create
> group
> aggregation path on top of the sort path?
I see no reason not to do it. (If you want to try, just go ahead.) However the
current patch version brings only the basic functionality and I'm not going to
add new functionality (such as parallel aggregation, partitioned tables or
postgres_fdw) until the open design questions are resolved. Otherwise there's
a risk that the additional work will be wasted due to major rework of the core
functionality.
> Core dump when running one query in agg_pushdown.sql
Thanks for the report! Fixed in the new version.
> This is really a cool feature. Thank you for working on this.
I appreciate to hear that :-) Let's see which postgres release will adopt it.
[1] https://www.postgresql.org/message-id/cc823e89-3fbc-f94e-b9d4-9c713b044b5d%402ndquadrant.com
[2] https://www.postgresql.org/message-id/flat/9666.1491295317%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-07-15 10:28 Richard Guo <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Richard Guo @ 2019-07-15 10:28 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
On Fri, Jul 12, 2019 at 4:42 PM Antonin Houska <[email protected]> wrote:
> Richard Guo <[email protected]> wrote:
>
> > I didn't fully follow the whole thread and mainly looked into the
> > latest
> > patch set. So what are the considerations for abandoning the
> > aggmultifn
> > concept?
>
> Originally the function was there to support join where both relations are
> grouped. My doubts about usefulness of such join started at [1]. (See the
> thread referenced from [2].)
>
> > In my opinion, aggmultifn would enable us to do a lot more
> > types of transformation. For example, consider the query below:
> >
> > select sum(foo.c) from foo join bar on foo.b = bar.b group by foo.a,
> > bar.a;
> >
> > With the latest patch, the plan looks like:
> >
> > Finalize HashAggregate <------ sum(psum)
> > Group Key: foo.a, bar.a
> > -> Hash Join
> > Hash Cond: (bar.b = foo.b)
> > -> Seq Scan on bar
> > -> Hash
> > -> Partial HashAggregate <------ sum(foo.c) as
> > psum
> > Group Key: foo.a, foo.b
> > -> Seq Scan on foo
> >
> >
> > If we have aggmultifn, we can perform the query this way:
> >
> > Finalize HashAggregate <------ sum(foo.c)*cnt
> > Group Key: foo.a, bar.a
> > -> Hash Join
> > Hash Cond: (foo.b = bar.b)
> > -> Seq Scan on foo
> > -> Hash
> > -> Partial HashAggregate <------ count(*) as cnt
> > Group Key: bar.a, bar.b
> > -> Seq Scan on bar
>
> Perhaps, but it that would require changes to nodeAgg.c, which I want to
> avoid
> for now.
>
> > And this way:
> >
> > Finalize HashAggregate <------ sum(psum)*cnt
> > Group Key: foo.a, bar.a
> > -> Hash Join
> > Hash Cond: (foo.b = bar.b)
> > -> Partial HashAggregate <------ sum(foo.c) as
> > psum
> > Group Key: foo.a, foo.b
> > -> Seq Scan on foo
> > -> Hash
> > -> Partial HashAggregate <------ count(*) as cnt
> > Group Key: bar.a, bar.b
> > -> Seq Scan on bar
>
> This looks like my idea presented in [1], for which there seems to be no
> common use case.
>
> > My another question is in function add_grouped_path(), when creating
> > sorted aggregation path on top of subpath. If the subpath is not
> > sorted,
> > then the sorted aggregation path would not be generated. Why not in
> > this
> > case we create a sort path on top of subpath first and then create
> > group
> > aggregation path on top of the sort path?
>
> I see no reason not to do it. (If you want to try, just go ahead.) However
> the
> current patch version brings only the basic functionality and I'm not
> going to
> add new functionality (such as parallel aggregation, partitioned tables or
> postgres_fdw) until the open design questions are resolved. Otherwise
> there's
> a risk that the additional work will be wasted due to major rework of the
> core
> functionality.
>
> > Core dump when running one query in agg_pushdown.sql
>
> Thanks for the report! Fixed in the new version.
>
Another core dump for query below:
select sum(t1.s1) from t1, t2, t3, t4 where t1.j1 = t2.j2 group by t1.g1,
t2.g2;
This is due to a small mistake:
diff --git a/src/backend/optimizer/util/relnode.c
b/src/backend/optimizer/util/relnode.c
index 10becc0..9c2deed 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -648,7 +648,7 @@ void
add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
{
add_rel_info(root->grouped_rel_list, rel);
- add_rel_info(root->agg_info_list, rel);
+ add_rel_info(root->agg_info_list, agg_info);
}
Thanks
Richard
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-07-17 14:39 Antonin Houska <[email protected]>
parent: Richard Guo <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2019-07-17 14:39 UTC (permalink / raw)
To: Richard Guo <[email protected]>; +Cc: Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Richard Guo <[email protected]> wrote:
> Another core dump for query below:
>
> select sum(t1.s1) from t1, t2, t3, t4 where t1.j1 = t2.j2 group by t1.g1, t2.g2;
>
> This is due to a small mistake:
>
> diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
> index 10becc0..9c2deed 100644
> --- a/src/backend/optimizer/util/relnode.c
> +++ b/src/backend/optimizer/util/relnode.c
> @@ -648,7 +648,7 @@ void
> add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
> {
> add_rel_info(root->grouped_rel_list, rel);
> - add_rel_info(root->agg_info_list, rel);
> + add_rel_info(root->agg_info_list, agg_info);
> }
>
Thanks! Yes, this is what I had to fix.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-09-04 20:23 Alvaro Herrera <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Alvaro Herrera @ 2019-09-04 20:23 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Richard Guo <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
This stuff seems very useful. How come it sits unreviewed for so long?
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2019-09-05 06:10 Antonin Houska <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 2 replies; 59+ messages in thread
From: Antonin Houska @ 2019-09-05 06:10 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Richard Guo <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Alvaro Herrera <[email protected]> wrote:
> This stuff seems very useful. How come it sits unreviewed for so long?
I think the review is hard for people who are not interested in the planner
very much. And as for further development, there are a few design decisions
that can hardly be resolved without Tom Lane's comments. Right now I recall
two problems: 1) is the way I currently store RelOptInfo for the grouped
relations correct?, 2) how should we handle types for which logical equality
does not imply physical (byte-wise) equality?
Fortunately it seems now that I'm not the only one who cares about 2), so this
problem might be resolved soon:
https://www.postgresql.org/message-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ%40mail.gma...
But 1) still remains.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2020-01-11 01:11 Tomas Vondra <[email protected]>
parent: Antonin Houska <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: Tomas Vondra @ 2020-01-11 01:11 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Richard Guo <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Hi,
I've been looking at the last version (v14) of this patch series,
submitted way back in July and unfortunately quiet since then. Antonin
is probably right one of the reasons for the lack of reviews is that it
requires interest/experience with planner.
Initially it was also a bit hard to understand what are the benefits
(and the patch shifted a bit), which is now mostly addressed by the
README in the last patch. The trouble is that's hidden in the patch and
so not immediately obvious to people considering reviewing it :-( Tom
posted a nice summary in November 2018, but it was perhaps focused on
the internals.
So my first suggestion it to write a short summary with example how it
affects practical queries (plan change, speedup) etc.
My second suggestion is to have meaningful commit messages for each part
of the patch series, explaining why we need that particular part. It
might have been explained somewhere in the thread, but as a reviewer I
really don't want to reverse-engineer the whole thread.
Now, regarding individual parts of the patch:
1) v14-0001-Introduce-RelInfoList-structure.patch
-------------------------------------------------
- I'm not entirely sure why we need this change. We had the list+hash
before, so I assume we do this because we need the output functions?
- The RelInfoList naming is pretty confusing, because it's actually not
a list but a combination of list+hash. And the embedded list is called
items, to make it yet a bit more confusing. I suggest we call this
just RelInfo or RelInfoLookup or something else not including List.
- RelInfoEntry seems severely under-documented, particularly the data
part (which is void* making it even harder to understand what's it for
without having to read the functions). AFAICS it's just simply a link
to the embedded list, but maybe I'm wrong.
- I suggest we move find_join_rel/add_rel_info/add_join_rel in relnode.c
right before build_join_rel. This makes diff clearer/smaller and
visual diffs nicer.
2) v14-0002-Introduce-make_join_rel_common-function.patch
---------------------------------------------------------
I see no point in keeping this change in a separate patch, as it prety
much just renames make_join_rel to make_join_rel_common and then adds
make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
return make_join_rel_common(root, rel1, rel2);
}
which seems rather trivial and useless on it's own. I'd just merge it
into 0003 where we use the _common function more extensively.
3) v14-0003-Aggregate-push-down-basic-functionality.patch
---------------------------------------------------------
I haven't done much review on this yet, but I've done some testing and
benchmarking so let me share at least that. The queries I used were of
the type I mentioned earlier in this thread - essentially a star schema,
i.e. fact table referencing dimensions, with aggregation per columns in
the dimension. So something like
SELECT d.c, sum(f) FROM fact JOIN dimension d ON (d.id = f.d_id)
GROUP BY d.c;
where "fact" table is much much larger than the dimension.
Attached is a script I used for testing with a bunch of queries and a
number of parameters (parallelism, size of dimension, size of fact, ...)
and a spreadsheed summarizing the results.
Overall, the results seem pretty good - in many cases the queries get
about 2x faster and I haven't seen any regressions. That's pretty nice.
One annoying thing is that this only works for non-parallel queries.
That is, I saw no improvements with parallel queries. It was still
faster than the non-parallel query with aggregate pushdown, but the
parallel query did not improve.
An example of timing looks like this:
non-parallel (pushdown = off): 918 ms
non-parallel (pushdown = on): 561 ms
parallel (pushdown = off): 313 ms
parallel (pushdown = on): 313 ms
The non-parallel query gets faster because after enabling push-down the
plan changes (and we get partial aggregate below the join). With
parallel query that does not happen, the plans stay the same.
I'm not claiming this is a bug - we end up picking the fastest execution
plan (i.e. we don't make a mistake by e.g. switching to the non-parallel
one with pushdown).
My understanding is that the aggregate pushdown can't (currently?) be
used with queries that use partial aggregate because of parallelism.
That is we can either end up with a plan like this:
Finalize Aggregate
-> Join
-> Partial Aggregate
-> ...
or a parallel plan like this:
Finalize Aggregate
-> Gather
-> Partial Aggregate
-> Join
-> ...
-> ...
but we currently don't support a combination of both
Finalize Aggregate
-> Gather
-> Join
-> Partial Aggregate
-> ...
I wonder if that's actually even possible and what would it take to make
it work?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
[application/vnd.oasis.opendocument.spreadsheet] pushdown-agg.ods (18.1K, ../../20200111011124.efacqhdsptc36pm2@development/2-pushdown-agg.ods)
download
[application/x-sh] pushdown.sh (2.7K, ../../20200111011124.efacqhdsptc36pm2@development/3-pushdown.sh)
download
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2020-01-16 15:25 Antonin Houska <[email protected]>
parent: Tomas Vondra <[email protected]>
0 siblings, 2 replies; 59+ messages in thread
From: Antonin Houska @ 2020-01-16 15:25 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Richard Guo <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Tomas Vondra <[email protected]> wrote:
> Hi,
>
> I've been looking at the last version (v14) of this patch series,
> submitted way back in July and unfortunately quiet since then. Antonin
> is probably right one of the reasons for the lack of reviews is that it
> requires interest/experience with planner.
>
> Initially it was also a bit hard to understand what are the benefits
> (and the patch shifted a bit), which is now mostly addressed by the
> README in the last patch. The trouble is that's hidden in the patch and
> so not immediately obvious to people considering reviewing it :-( Tom
> posted a nice summary in November 2018, but it was perhaps focused on
> the internals.
>
> So my first suggestion it to write a short summary with example how it
> affects practical queries (plan change, speedup) etc.
I think README plus regression test output should be enough for someone who is
about to review patch as complex as this.
> My second suggestion is to have meaningful commit messages for each part
> of the patch series, explaining why we need that particular part. It
> might have been explained somewhere in the thread, but as a reviewer I
> really don't want to reverse-engineer the whole thread.
ok, done.
> Now, regarding individual parts of the patch:
>
>
> 1) v14-0001-Introduce-RelInfoList-structure.patch
> -------------------------------------------------
>
> - I'm not entirely sure why we need this change. We had the list+hash
> before, so I assume we do this because we need the output functions?
I believe that this is what Tom proposed in [1], see "Maybe an appropriate
preliminary patch is to refactor the support code ..." near the end of that
message. The point is that now we need two lists: one for "plain" relations
and one for grouped ones.
> - The RelInfoList naming is pretty confusing, because it's actually not
> a list but a combination of list+hash. And the embedded list is called
> items, to make it yet a bit more confusing. I suggest we call this
> just RelInfo or RelInfoLookup or something else not including List.
I think it can be considered a list by the caller of add_join_rel() etc. The
hash table is hidden from user and is empty until the list becomes long
enough. The word "list" in the structure name may just indicate that new
elements can be added to the end, which shouldn't be expected if the structure
was an array.
I searched a bit in tools/pgindent/typedefs.list and found at least a few
structures that also do have "list" in the name but are not lists internally:
CatCList, FuncCandidateList, MCVList.
Actually I'm not opposed to renaming the structure, but don't have better idea
right now. As for *Lookup: following is the only use of such a structure name
in PG code and it's not something to store data in:
typedef enum
{
IDENTIFIER_LOOKUP_NORMAL, /* normal processing of var names */
IDENTIFIER_LOOKUP_DECLARE, /* In DECLARE --- don't look up names */
IDENTIFIER_LOOKUP_EXPR /* In SQL expression --- special case */
} IdentifierLookup;
> - RelInfoEntry seems severely under-documented, particularly the data
> part (which is void* making it even harder to understand what's it for
> without having to read the functions). AFAICS it's just simply a link
> to the embedded list, but maybe I'm wrong.
This is just JoinHashEntry (which was also undocumented) renamed. I've added a
short comment now.
> - I suggest we move find_join_rel/add_rel_info/add_join_rel in relnode.c
> right before build_join_rel. This makes diff clearer/smaller and
> visual diffs nicer.
Hm, it might improve readability of the diff, but this API is exactly where I
still need feedback from Tom. I'm not eager to optimize the diff as long as
there's a risk that these functions will be removed or renamed.
> 2) v14-0002-Introduce-make_join_rel_common-function.patch
> ---------------------------------------------------------
>
> I see no point in keeping this change in a separate patch, as it prety
> much just renames make_join_rel to make_join_rel_common and then adds
>
> make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
> {
> return make_join_rel_common(root, rel1, rel2);
> }
>
> which seems rather trivial and useless on it's own. I'd just merge it
> into 0003 where we use the _common function more extensively.
ok. I thought that this improves readability of the diffs, but it doesn't look
that bad if this is included in 0003.
>
> 3) v14-0003-Aggregate-push-down-basic-functionality.patch
> ---------------------------------------------------------
>
> I haven't done much review on this yet, but I've done some testing and
> benchmarking so let me share at least that. The queries I used were of
> the type I mentioned earlier in this thread - essentially a star schema,
> i.e. fact table referencing dimensions, with aggregation per columns in
> the dimension. So something like
>
> SELECT d.c, sum(f) FROM fact JOIN dimension d ON (d.id = f.d_id)
> GROUP BY d.c;
>
> where "fact" table is much much larger than the dimension.
>
> Attached is a script I used for testing with a bunch of queries and a
> number of parameters (parallelism, size of dimension, size of fact, ...)
> and a spreadsheed summarizing the results.
>
> Overall, the results seem pretty good - in many cases the queries get
> about 2x faster and I haven't seen any regressions. That's pretty nice.
Thanks for such an elaborate testing! The ultimate goal of this patch is to
improve sharding (using postgres_fdw), but it's nice to see significant
improvement even for local queries.
> One annoying thing is that this only works for non-parallel queries.
> That is, I saw no improvements with parallel queries. It was still
> faster than the non-parallel query with aggregate pushdown, but the
> parallel query did not improve.
> An example of timing looks like this:
>
> non-parallel (pushdown = off): 918 ms
> non-parallel (pushdown = on): 561 ms
> parallel (pushdown = off): 313 ms
> parallel (pushdown = on): 313 ms
>
> The non-parallel query gets faster because after enabling push-down the
> plan changes (and we get partial aggregate below the join). With
> parallel query that does not happen, the plans stay the same.
>
> I'm not claiming this is a bug - we end up picking the fastest execution
> plan (i.e. we don't make a mistake by e.g. switching to the non-parallel
> one with pushdown).
>
> My understanding is that the aggregate pushdown can't (currently?) be
> used with queries that use partial aggregate because of parallelism.
> That is we can either end up with a plan like this:
>
> Finalize Aggregate
> -> Join
> -> Partial Aggregate
> -> ...
>
> or a parallel plan like this:
>
> Finalize Aggregate
> -> Gather
> -> Partial Aggregate
> -> Join
> -> ...
> -> ...
>
> but we currently don't support a combination of both
>
> Finalize Aggregate
> -> Gather
> -> Join
> -> Partial Aggregate
> -> ...
>
> I wonder if that's actually even possible and what would it take to make
> it work?
I had a prototype of the feature that does affect parallel queries (as well as
partitioned tables and postgres_fdw), but didn't include these parts in the
recent patch versions. The point is that the API to store RelOptInfos can
still change and in such a case I'd have to rebase too much code.
v15 is attached. Actually there are no significant changes relative to v14,
but v14 does not apply to the current master:
error: patch failed: src/test/regress/serial_schedule:140
error: src/test/regress/serial_schedule: patch does not apply
This is interesting because no problem is currently reported at
http://commitfest.cputube.org/
[1] https://www.postgresql.org/message-id/[email protected]
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-01-31 21:59 legrand legrand <[email protected]>
parent: Antonin Houska <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: legrand legrand @ 2020-01-31 21:59 UTC (permalink / raw)
To: pgsql-hackers
Hello,
Thank you for this great feature !
I hope this will be reviewed/validated soon ;o)
Just a comment:
set enable_agg_pushdown to true;
isn't displayed in EXPLAIN (SETTINGS) syntax.
The following modification seems to fix that:
src/backend/utils/misc/guc.c
{"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables aggregation push-down."),
NULL,
GUC_EXPLAIN <<<<--- line Added --->>>>
},
&enable_agg_pushdown,
false,
NULL, NULL, NULL
},
then
postgres=# set enable_agg_pushdown = true;
SET
postgres=# explain (settings) select 1;
QUERY PLAN
------------------------------------------
Result (cost=0.00..0.01 rows=1 width=4)
Settings: enable_agg_pushdown = 'on'
(2 rows)
Regards
PAscal
--
Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-02-03 07:46 Antonin Houska <[email protected]>
parent: legrand legrand <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: Antonin Houska @ 2020-02-03 07:46 UTC (permalink / raw)
To: legrand legrand <[email protected]>; +Cc: pgsql-hackers
legrand legrand <[email protected]> wrote:
> set enable_agg_pushdown to true;
> isn't displayed in EXPLAIN (SETTINGS) syntax.
>
>
> The following modification seems to fix that:
>
> src/backend/utils/misc/guc.c
>
> {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
> gettext_noop("Enables aggregation push-down."),
> NULL,
> GUC_EXPLAIN <<<<--- line Added --->>>>
> },
> &enable_agg_pushdown,
> false,
> NULL, NULL, NULL
> },
Thanks. I'll include this change in the next version.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2020-02-06 08:30 Richard Guo <[email protected]>
parent: Antonin Houska <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: Richard Guo @ 2020-02-06 08:30 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Alvaro Herrera <[email protected]>; Richard Guo <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Hi,
I've been looking at the 'make_join_rel()' part of the patch, and I'm
aware that, if we are joining A and B, a 'grouped join rel (AB)' would
be created besides the 'plain join rel (AB)', and paths are added by 1)
applying partial aggregate to the paths of the 'plain join rel (AB)', or
2) joining grouped A to plain B or joining plain A to grouped B.
This is a smart idea. One issue I can see is that some logic would have
to be repeated several times. For example, the validity check for the
same proposed join would be performed at most three times by
join_is_legal().
I'm thinking of another idea that, instead of using a separate
RelOptInfo for the grouped rel, we add in RelOptInfo a
'grouped_pathlist' for the grouped paths, just like how we implement
parallel query via adding 'partial_pathlist'.
For base rel, if we decide it can produce grouped paths, we create the
grouped paths by applying partial aggregation to the paths in 'pathlist'
and add them into 'grouped_pathlist'.
For join rel (AB), we can create the grouped paths for it by:
1) joining a grouped path from 'A->grouped_pathlist' to a plain path
from 'B->pathlist', or vice versa;
2) applying partial aggregation to the paths in '(AB)->pathlist'.
This is basically the same idea, just moves the grouped paths from the
grouped rel to a 'grouped_pathlist'. With it we should not need to make
any code changes to 'make_join_rel()'. Most code changes would happen in
'add_paths_to_joinrel()'.
Will this idea work? Is it better or worse?
Thanks
Richard
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: [HACKERS] WIP: Aggregation push-down
@ 2020-02-07 09:05 Antonin Houska <[email protected]>
parent: Richard Guo <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: Antonin Houska @ 2020-02-07 09:05 UTC (permalink / raw)
To: Richard Guo <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Alvaro Herrera <[email protected]>; Richard Guo <[email protected]>; Tom Lane <[email protected]>; Michael Paquier <[email protected]>; pgsql-hackers
Richard Guo <[email protected]> wrote:
> Hi,
>
> I've been looking at the 'make_join_rel()' part of the patch, and I'm
> aware that, if we are joining A and B, a 'grouped join rel (AB)' would
> be created besides the 'plain join rel (AB)', and paths are added by 1)
> applying partial aggregate to the paths of the 'plain join rel (AB)', or
> 2) joining grouped A to plain B or joining plain A to grouped B.
>
> This is a smart idea. One issue I can see is that some logic would have
> to be repeated several times. For example, the validity check for the
> same proposed join would be performed at most three times by
> join_is_legal().
>
> I'm thinking of another idea that, instead of using a separate
> RelOptInfo for the grouped rel, we add in RelOptInfo a
> 'grouped_pathlist' for the grouped paths, just like how we implement
> parallel query via adding 'partial_pathlist'.
>
> For base rel, if we decide it can produce grouped paths, we create the
> grouped paths by applying partial aggregation to the paths in 'pathlist'
> and add them into 'grouped_pathlist'.
>
> For join rel (AB), we can create the grouped paths for it by:
> 1) joining a grouped path from 'A->grouped_pathlist' to a plain path
> from 'B->pathlist', or vice versa;
> 2) applying partial aggregation to the paths in '(AB)->pathlist'.
>
> This is basically the same idea, just moves the grouped paths from the
> grouped rel to a 'grouped_pathlist'. With it we should not need to make
> any code changes to 'make_join_rel()'. Most code changes would happen in
> 'add_paths_to_joinrel()'.
>
> Will this idea work? Is it better or worse?
I tried such approach in an earlier version of the patch [1], and I think the
reason also was to avoid repeated calls of functions like
join_is_legal(). However there were objections against such approach,
e.g. [2], and I admit that it was more invasive than what the current patch
version does.
Perhaps we can cache the result of join_is_legal() that we get for the plain
relation, and use it for the group relation. I'll consider that. Thanks.
[1] https://www.postgresql.org/message-id/18007.1513957437%40localhost
[2] https://www.postgresql.org/message-id/CA%2BTgmob8og%2B9HzMg1vM%2B3LwDm2f_bHUi9%2Bg1bqLDTjqpw5s%2BnQ%...
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-02-26 21:10 legrand legrand <[email protected]>
parent: Antonin Houska <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: legrand legrand @ 2020-02-26 21:10 UTC (permalink / raw)
To: pgsql-hackers
Antonin Houska-2 wrote
> Alvaro Herrera <
> alvherre@
> > wrote:
>
>> This stuff seems very useful. How come it sits unreviewed for so long?
>
> I think the review is hard for people who are not interested in the
> planner
> very much. And as for further development, there are a few design
> decisions
> that can hardly be resolved without Tom Lane's comments. Right now I
> recall
> two problems: 1) is the way I currently store RelOptInfo for the grouped
> relations correct?, 2) how should we handle types for which logical
> equality
> does not imply physical (byte-wise) equality?
>
> Fortunately it seems now that I'm not the only one who cares about 2), so
> this
> problem might be resolved soon:
>
> https://www.postgresql.org/message-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ%40mail.gma...
>
> But 1) still remains.
>
> --
> Antonin Houska
> Web: https://www.cybertec-postgresql.com
Hello
would "pgsql: Add equalimage B-Tree support functions."
https://www.postgresql.org/message-id/[email protected]
help for 2) ?
Regards
PAscal
--
Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-02-27 08:51 Antonin Houska <[email protected]>
parent: legrand legrand <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2020-02-27 08:51 UTC (permalink / raw)
To: legrand legrand <[email protected]>; +Cc: pgsql-hackers
legrand legrand <[email protected]> wrote:
> Antonin Houska-2 wrote
> > Right now I recall two problems: 1) is the way I currently store
> > RelOptInfo for the grouped relations correct?, 2) how should we handle
> > types for which logical equality does not imply physical (byte-wise)
> > equality?
> >
> > Fortunately it seems now that I'm not the only one who cares about 2), so
> > this
> > problem might be resolved soon:
> >
> > https://www.postgresql.org/message-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ%40mail.gma...
> >
> > But 1) still remains.
> >
>
> Hello
> would "pgsql: Add equalimage B-Tree support functions."
> https://www.postgresql.org/message-id/[email protected]
Yes, it seems so. I'll adapt the patch soon, hopefully next week. Thanks for
reminder.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-04-21 08:37 Andy Fan <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 2 replies; 59+ messages in thread
From: Andy Fan @ 2020-04-21 08:37 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: legrand legrand <[email protected]>; pgsql-hackers
On Thu, Feb 27, 2020 at 4:50 PM Antonin Houska <[email protected]> wrote:
> legrand legrand <[email protected]> wrote:
>
> > Antonin Houska-2 wrote
>
> > > Right now I recall two problems: 1) is the way I currently store
> > > RelOptInfo for the grouped relations correct?, 2) how should we handle
> > > types for which logical equality does not imply physical (byte-wise)
> > > equality?
> > >
> > > Fortunately it seems now that I'm not the only one who cares about 2),
> so
> > > this
> > > problem might be resolved soon:
> > >
> > >
> https://www.postgresql.org/message-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ%40mail.gma...
> > >
> > > But 1) still remains.
> > >
> >
> > Hello
> > would "pgsql: Add equalimage B-Tree support functions."
> >
> https://www.postgresql.org/message-id/[email protected]
>
> Yes, it seems so. I'll adapt the patch soon, hopefully next week. Thanks
> for
> reminder.
>
> --
> Antonin Houska
> Web: https://www.cybertec-postgresql.com
>
>
Hi Antonin:
The more tests on your patch, the more powerful I feel it is! At the same
time,
I think the most difficult part to understand your design is you can accept
any number of generic join clauses, so I guess more explanation on this
part
may be helpful.
At the code level, I did some slight changes on init_grouping_targets which
may
make the code easier to read. You are free to to use/not use it.
Hope your patch get more attention soon!
Best Regards
Andy Fan
Attachments:
[application/octet-stream] v2-0001-tiny-changes-for-init_grouping_targets.patch (11.3K, ../../CAKU4AWqK_X4L5WDsjnsFpm_Gt8Mx0+H4frC-6Pom1ro=aq44mQ@mail.gmail.com/3-v2-0001-tiny-changes-for-init_grouping_targets.patch)
download | inline diff:
From 7a2a460f80c61909b3f6ab7b39f3ecf2c2fba2b0 Mon Sep 17 00:00:00 2001
From: Andy Fan <[email protected]>
Date: Tue, 14 Apr 2020 11:27:41 +0800
Subject: [PATCH v2] tiny changes for init_grouping_targets
---
src/backend/optimizer/util/relnode.c | 274 +++++++++++----------------
1 file changed, 107 insertions(+), 167 deletions(-)
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 3ee38bf1f6..68a347e2d9 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -73,6 +73,7 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool aggref_used_var(PlannerInfo *root, Var *var);
static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
PathTarget *target, PathTarget *agg_input,
List *gvis, List **group_exprs_extra_p);
@@ -2066,8 +2067,6 @@ create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
*/
Assert(root->grouped_var_list != NIL);
- result = makeNode(RelAggInfo);
-
/*
* The current implementation of aggregation push-down cannot handle
* PlaceHolderVar (PHV).
@@ -2334,6 +2333,8 @@ create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
* SortGroupClauses.
*/
i = 0;
+
+ result = makeNode(RelAggInfo);
foreach(lc, target->exprs)
{
Index sortgroupref = 0;
@@ -2429,10 +2430,15 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
PathTarget *target, PathTarget *agg_input,
List *gvis, List **group_exprs_extra_p)
{
- ListCell *lc1,
- *lc2;
- List *unresolved = NIL;
- List *unresolved_sortgrouprefs = NIL;
+ ListCell *lc1;
+ List *grouping_columns = NIL;
+
+ foreach(lc1, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvar = lfirst_node(GroupedVarInfo, lc1);
+ if (IsA(gvar->gvexpr, Var))
+ grouping_columns = lappend(grouping_columns, gvar->gvexpr);
+ }
foreach(lc1, rel->reltarget->exprs)
{
@@ -2440,7 +2446,8 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
bool is_grouping;
Index sortgroupref = 0;
bool derived = false;
- bool needed_by_aggregate;
+ RangeTblEntry *rte;
+ List *deps = NIL;
/*
* Given that PlaceHolderVar currently prevents us from doing
@@ -2452,6 +2459,8 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
&sortgroupref, &derived);
+ rte = root->simple_rte_array[tvar->varno];
+
/*
* Derived grouping expressions should not be referenced by the query
* targetlist, so let them fall into vars_unresolved. It'll be checked
@@ -2475,190 +2484,121 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
* sortgroupref in addition.
*/
add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
-
- /* Process the next expression. */
- continue;
}
-
- /*
- * Is this Var needed in the query targetlist for anything else than
- * aggregate input?
- */
- needed_by_aggregate = false;
- foreach(lc2, root->grouped_var_list)
+ else if (aggref_used_var(root, tvar) &&
+ tlist_member((Expr*) tvar, root->processed_tlist) == NULL)
{
- GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
- ListCell *lc3;
- List *vars;
-
- if (!IsA(gvi->gvexpr, Aggref))
- continue;
-
- if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
- continue;
-
- /*
- * XXX Consider some sort of caching.
- */
- vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
- foreach(lc3, vars)
- {
- Var *var = lfirst_node(Var, lc3);
-
- if (equal(var, tvar))
- {
- needed_by_aggregate = true;
- break;
- }
- }
- list_free(vars);
- if (needed_by_aggregate)
- break;
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
}
-
- if (needed_by_aggregate)
- {
- bool found = false;
-
- foreach(lc2, root->processed_tlist)
- {
- TargetEntry *te = lfirst_node(TargetEntry, lc2);
-
- if (IsA(te->expr, Aggref))
- continue;
-
- if (equal(te->expr, tvar))
- {
- found = true;
- break;
- }
- }
-
- /*
- * If it's only Aggref input, add it to the aggregation input
- * target and that's it.
- */
- if (!found)
- {
- add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
- continue;
- }
- }
-
- /*
- * Further investigation involves dependency check, for which we need
- * to have all the (plain-var) grouping expressions gathered.
- */
- unresolved = lappend(unresolved, tvar);
- unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
- sortgroupref);
- }
-
- /*
- * Check for other possible reasons for the var to be in the plain target.
- */
- forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
- {
- Var *var = lfirst_node(Var, lc1);
- Index sortgroupref = lfirst_int(lc2);
- RangeTblEntry *rte;
- List *deps = NIL;
- Relids relids_subtract;
- int ndx;
- RelOptInfo *baserel;
-
- rte = root->simple_rte_array[var->varno];
-
- /*
- * Check if the Var can be in the grouping key even though it's not
- * mentioned by the GROUP BY clause (and could not be derived using
- * ECs).
- */
- if (sortgroupref == 0 &&
- check_functional_grouping(rte->relid, var->varno,
- var->varlevelsup,
- target->exprs, &deps))
+ else if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, tvar->varno,
+ tvar->varlevelsup, grouping_columns, &deps))
{
/*
* The var shouldn't be actually used as a grouping key (instead,
* the one this depends on will be), so sortgroupref should not be
* important.
*/
- add_new_column_to_pathtarget(target, (Expr *) var);
- add_new_column_to_pathtarget(agg_input, (Expr *) var);
-
- /*
- * The var may or may not be present in generic grouping
- * expression(s) in addition, but this is handled elsewhere.
- */
- continue;
+ add_new_column_to_pathtarget(target, (Expr *) tvar);
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
}
-
- /*
- * Isn't the expression needed by joins above the current rel?
- *
- * The relids we're not interested in do include 0, which is the
- * top-level targetlist. The only reason for relids to contain 0
- * should be that arg_var is referenced either by aggregate or by
- * grouping expression, but right now we're interested in the *other*
- * reasons. (As soon aggregation is pushed down, the aggregates in the
- * query targetlist no longer need direct reference to arg_var
- * anyway.)
- */
- relids_subtract = bms_copy(rel->relids);
- bms_add_member(relids_subtract, 0);
-
- baserel = find_base_rel(root, var->varno);
- ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx],
- relids_subtract))
+ else
{
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
/*
- * The variable is needed by a join involving this relation. That
- * case includes variable that is referenced by a generic grouping
- * expression.
+ * Isn't the expression needed by joins above the current rel?
*
- * The only way to bring this var to the aggregation output is to
- * add it to the grouping expressions too.
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
*/
- if (sortgroupref > 0)
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, tvar->varno);
+ ndx = tvar->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
{
/*
- * The var could be recognized as a potentially useful
- * grouping expression at the top of the loop, so we can add
- * it to the grouping target, as well as to the agg_input.
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
*/
- add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
- add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, tvar);
+ }
}
else
{
/*
- * Since root->parse->groupClause is not supposed to contain
- * this expression, we need to construct special
- * SortGroupClause. Its tleSortGroupRef needs to be unique
- * within target_agg, so postpone creation of the
- * SortGroupRefs until we're done with the iteration of
- * rel->reltarget->exprs.
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
*/
- *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ return false;
}
}
- else
- {
- /*
- * As long as the query is semantically correct, arriving here
- * means that the var is referenced by a generic grouping
- * expression but not referenced by any join.
- *
- * create_rel_agg_info() should add this variable to "agg_input"
- * target and also add the whole generic expression to "target",
- * but that's subject to future enhancement of the aggregate
- * push-down feature.
- */
- return false;
- }
}
-
return true;
}
+
+static bool
+aggref_used_var(PlannerInfo *root, Var *var)
+{
+
+ ListCell *lc;
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(var->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ if (list_member(vars, var))
+ return true;
+ }
+
+ return false;
+}
--
2.21.0
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-04-22 03:39 Andy Fan <[email protected]>
parent: Andy Fan <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: Andy Fan @ 2020-04-22 03:39 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: legrand legrand <[email protected]>; pgsql-hackers
>
> > 1) v14-0001-Introduce-RelInfoList-structure.patch
> > -------------------------------------------------
> >
> > - I'm not entirely sure why we need this change. We had the list+hash
> > before, so I assume we do this because we need the output functions?
>
> I believe that this is what Tom proposed in [1], see "Maybe an appropriate
> preliminary patch is to refactor the support code ..." near the end of that
> message. The point is that now we need two lists: one for "plain" relations
> and one for grouped ones.
>
>
I guess what Toms suggested[1] is to store the the grouped ones into
root->upper_rels rather than a separated member, see fetch_upper_rel
or UpperRelationKind. If we do need the list+hash method for long list
lookup,
we can merge it into upper_rels. If we want this benefits at other place
rather
than root->upper_rels, we can store a generic type in RelInfoList, looks
currently
it is bounded to RelAggInfo besides RelOptInfo. But overall, personally I
think we can
ignore such optimization at the first stage to save the attention of the
core reviewers
since they are too precious :) Just FYI
[1] https://www.postgresql.org/message-id/[email protected]
Best Regards
Andy Fan
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-04-24 12:11 Antonin Houska <[email protected]>
parent: Andy Fan <[email protected]>
1 sibling, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2020-04-24 12:11 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: legrand legrand <[email protected]>; pgsql-hackers
Andy Fan <[email protected]> wrote:
> The more tests on your patch, the more powerful I feel it is!
Thanks for the appreciation. Given the poor progress it's increasingly hard
for me to find motivation to work on it. I'll try to be more professional :-)
> At the same time, I think the most difficult part to understand your design
> is you can accept any number of generic join clauses, so I guess more
> explanation on this part may be helpful.
ok, I'll consider adding some comments, although the concept is mentioned in
optimizer/README
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
...
> At the code level, I did some slight changes on init_grouping_targets which may
> make the code easier to read. You are free to to use/not use it.
I'm going to accept your change of create_rel_agg_info(), but I hesitate about
the changes to init_grouping_targets().
First, is it worth to spend CPU cycles on construction of an extra list
grouping_columns? Is there a corner case in which we cannot simply pass
grouping_columns=target->exprs to check_functional_grouping()?
Second, it's obvious that you prefer the style
foreach ()
{
if ()
...
else if ()
...
else
...
}
over this
foreach ()
{
if ()
{
...
continue;
}
if ()
{
...
continue;
}
...
}
I often prefer the latter and I see that the existing planner code uses this
style quite often too. I think the reason is that it allows for more complex
tests, while the "else-if style" requires all tests to take place inside the
"if ()" expression. However, if several (not necessarily tightly related)
tests become "compressed" this way, it's less obvious how where to put
comments. Indeed it seems that some comments got lost due to your changes.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-04-24 13:01 Antonin Houska <[email protected]>
parent: Andy Fan <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: Antonin Houska @ 2020-04-24 13:01 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: legrand legrand <[email protected]>; pgsql-hackers
Andy Fan <[email protected]> wrote:
> > 1) v14-0001-Introduce-RelInfoList-structure.patch
> > -------------------------------------------------
> >
> > - I'm not entirely sure why we need this change. We had the list+hash
> > before, so I assume we do this because we need the output functions?
>
> I believe that this is what Tom proposed in [1], see "Maybe an appropriate
> preliminary patch is to refactor the support code ..." near the end of that
> message. The point is that now we need two lists: one for "plain" relations
> and one for grouped ones.
>
>
> I guess what Toms suggested[1] is to store the the grouped ones into
> root->upper_rels rather than a separated member, see fetch_upper_rel
> or UpperRelationKind. If we do need the list+hash method for long list lookup,
> we can merge it into upper_rels. If we want this benefits at other place rather
> than root->upper_rels, we can store a generic type in RelInfoList, looks currently
> it is bounded to RelAggInfo besides RelOptInfo. But overall, personally I think we can
> ignore such optimization at the first stage to save the attention of the core reviewers
> since they are too precious :) Just FYI
>
> [1] https://www.postgresql.org/message-id/[email protected]
Hm, you seem to be right, not sure why I missed the point. I thought that the
reason Tom doesn't like storing the grouped relations in simple_rel_array is
that we only need the grouped base relations inside query_planner(), but
simple_rel_array is used higher in the stack. So I introduced a new field and
used it only in query_planner() and subroutines.
Yes, it's better to use root->upper_rels than to introduce the new field. I'll
adjust the patch. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-04-26 08:12 Andy Fan <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Andy Fan @ 2020-04-26 08:12 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: legrand legrand <[email protected]>; pgsql-hackers
On Fri, Apr 24, 2020 at 8:10 PM Antonin Houska <[email protected]> wrote:
> Andy Fan <[email protected]> wrote:
>
> > The more tests on your patch, the more powerful I feel it is!
>
> Thanks for the appreciation. Given the poor progress it's increasingly hard
> for me to find motivation to work on it. I'll try to be more professional
> :-)
>
>
Let's not give up:) I see your patch can push down the aggregation in 3
cases
at least:
1). The group by clause exists in the join eq clause.
2). The group by clause doesn't exist in join eq clause, but we have
pk on on side of join eq clause.
3). The group by clause doesn't exist in join eq clause, and we don't have
the pk as well.
Tom well explained the correctness of the first 2 cases [1] and probably
the case
3) is correct as well, but it is a bit of hard to understand. If this is a
case for others
as well, that probably make the review harder.
So my little suggestion is can we split the patch into some smaller commit
to handle each case? like: commit 1 & 2 handles case 1 & 2，commit 3 handles
join relation as well. commit 4 handles the case 3. Commit 5 can avoid
the
two-phase aggregation for case 2. Commit 6 introduces the aggmultifn. and
commit 7 to handle some outer join case Ashutosh raised at [2]. However not
all the cases need to be handled at one time.
Actually when I read the code, I want such separation by my own to verify my
understanding is correct, but I don't have such time at least now. It
maybe much
easier for you since you are the author. I will be pretty pleasure to help
in any case
after I close my current working item (Hopefully in 2 months).
> At the code level, I did some slight changes on init_grouping_targets
> which may
> > make the code easier to read. You are free to to use/not use it.
>
> I'm going to accept your change of create_rel_agg_info(), but I hesitate
> about
> the changes to init_grouping_targets().
>
> First, is it worth to spend CPU cycles on construction of an extra list
> grouping_columns? Is there a corner case in which we cannot simply pass
> grouping_columns=target->exprs to check_functional_grouping()?
>
> Second, it's obvious that you prefer the style
>
> foreach ()
> {
> if ()
> ...
> else if ()
> ...
> else
> ...
> }
>
> over this
>
> foreach ()
> {
> if ()
> {
> ...
> continue;
> }
>
> if ()
> {
> ...
> continue;
> }
>
> ...
> }
>
> I often prefer the latter and I see that the existing planner code uses
> this
> style quite often too. I think the reason is that it allows for more
> complex
> tests, while the "else-if style" requires all tests to take place inside
> the
> "if ()" expression. However, if several (not necessarily tightly related)
> tests become "compressed" this way, it's less obvious how where to put
> comments. Indeed it seems that some comments got lost due to your changes.
Your explanation looks reasonable, thanks for that. the changes also used
some builtin function to avoid the one more level for-loop.
like tlist_member.
As for the high level design, based on my current knowledge, probably no
need
to change.
[1] https://www.postgresql.org/message-id/9726.1542577439%40sss.pgh.pa.us
[2]
https://www.postgresql.org/message-id/flat/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3%2BcoG%3D8ki7s8Zqjw%4...
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-05-19 08:17 Antonin Houska <[email protected]>
parent: Andy Fan <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2020-05-19 08:17 UTC (permalink / raw)
To: Andy Fan <[email protected]>; +Cc: legrand legrand <[email protected]>; pgsql-hackers
Andy Fan <[email protected]> wrote:
> On Fri, Apr 24, 2020 at 8:10 PM Antonin Houska <[email protected]> wrote:
>
> Andy Fan <[email protected]> wrote:
>
> Let's not give up:) I see your patch can push down the aggregation in 3 cases
> at least:
>
> 1). The group by clause exists in the join eq clause.
> 2). The group by clause doesn't exist in join eq clause, but we have
> pk on on side of join eq clause.
> 3). The group by clause doesn't exist in join eq clause, and we don't have
> the pk as well.
>
> Tom well explained the correctness of the first 2 cases [1]
I admit that I don't quite understand his statement
We can't simply aggregate during the scan of "a", because some values of
"x" might not appear at all in the input of the naively-computed aggregate
(if their rows have no join partner in "b"), or might appear multiple
times (if their rows have multiple join partners).
Consider the following tables
======= ======
A B
======= ======
x | y z
------- ------
3 | 4 1
1 | 4 2
4 | 0 3
1 | 0 4
and this query
SELECT sum(a.x), a.y FROM a, b WHERE a.y > b.z GROUP BY a.y;
Without aggregate push-down, the aggregation input is
a.x | a.y | b.z
----------------
3 | 4 | 1
3 | 4 | 2
3 | 4 | 3
1 | 4 | 1
1 | 4 | 2
1 | 4 | 3
and the query result is
sum(a.x) | a.y
-----------------
12 | 4
With the push-down, "A" aggregated looks like
sum(a.x) | a.y
-----------------
4 | 4
5 | 0
and the join with "B" produces this:
sum(a.x) | a.y | b.z
---------------------
4 | 4 | 1
4 | 4 | 2
4 | 4 | 3
After that, the final aggregation yields the same result that we got w/o the
aggregate push-down:
sum(a.x) | a.y
-----------------
12 | 4
So if a row of "A" has no / multiple join partners in "B", the same is true
for the group produced by the pushed-down aggregation. Of course it's assumed
that the input rows belonging to particular group can be distributed among the
PartialAgg nodes in arbitrary way, for example:
Agg
<value 1>
<value 1>
<value 2>
<value 2>
needs to be equivalent to
FinalAgg
PartialAgg
<value 1>
<value 2>
PartialAgg
<value 1>
<value 2>
This should be o.k. because the partial aggregation performed currently by
parallel workers relies o the same "distributive property" of aggregation.
> So my little suggestion is can we split the patch into some smaller commit
> to handle each case? like: commit 1 & 2 handles case 1 & 2，commit 3 handles
> join relation as well. commit 4 handles the case 3.
To separate the patch this way effectively means to rework the feature almost
from scratch. I'm still not convinced that such a rework is necesary.
> Commit 5 can avoid the two-phase aggregation for case 2.
The two-phase aggregation can be avoided in more generic way as soon as the
"unique key" feature (I think you do participated in the related discussion)
is committed.
> Commit 6 introduces the aggmultifnf.
I used this function in the initial version of the patch [3], to join two
grouped relations. The current version does not try to join two grouped
relations and it's hard to imagine how to determine the "multiplication
factor". I suppose this is not really trivial. Since the current version of
the feature hasn't find it's way into PG core for several years, I'm not
willing to make it even trickier soon, if at all.
> and commit 7 to handle some outer join case Ashutosh raised at [2].
That was too high-level design, as he admits in [4].
> However not all the cases need to be handled at one time.
Sure.
>
> > At the code level, I did some slight changes on init_grouping_targets which may
> > make the code easier to read. You are free to to use/not use it.
>
> I'm going to accept your change of create_rel_agg_info(), but I hesitate about
> the changes to init_grouping_targets().
>
> First, is it worth to spend CPU cycles on construction of an extra list
> grouping_columns? Is there a corner case in which we cannot simply pass
> grouping_columns=target->exprs to check_functional_grouping()?
>
> Second, it's obvious that you prefer the style
>
> foreach ()
> {
> if ()
> ...
> else if ()
> ...
> else
> ...
> }
>
> over this
>
> foreach ()
> {
> if ()
> {
> ...
> continue;
> }
>
> if ()
> {
> ...
> continue;
> }
>
> ...
> }
>
> I often prefer the latter and I see that the existing planner code uses this
> style quite often too. I think the reason is that it allows for more complex
> tests, while the "else-if style" requires all tests to take place inside the
> "if ()" expression. However, if several (not necessarily tightly related)
> tests become "compressed" this way, it's less obvious how where to put
> comments. Indeed it seems that some comments got lost due to your changes.
>
> Your explanation looks reasonable, thanks for that. the changes also used
> some builtin function to avoid the one more level for-loop. like tlist_member.
Eventually I've reworked init_grouping_targets(), although not exactly as you
proposed. I hope it's easier to understand now.
> [1] https://www.postgresql.org/message-id/9726.1542577439%40sss.pgh.pa.us
> [2] https://www.postgresql.org/message-id/flat/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3%2BcoG%3D8ki7s8Zqjw%4...
[3] https://www.postgresql.org/message-id/29111.1483984605@localhost
[4] https://www.postgresql.org/message-id/CAFjFpRfUKq3Asgtki1XctPkCN6YC4oA2vNWh66OgBo-H26ePWA%40mail.gma...
The next version is attached.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-07-02 12:39 Daniel Gustafsson <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Daniel Gustafsson @ 2020-07-02 12:39 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: Andy Fan <[email protected]>; legrand legrand <[email protected]>; pgsql-hackers
> On 19 May 2020, at 10:17, Antonin Houska <[email protected]> wrote:
> The next version is attached.
This version now fails to apply to HEAD, with what looks like like a trivial
error in the expected test output. Can you please submit a rebased version so
we can see it run in the patch tester CI? I'm marking the entry as Waiting on
Author in the meantime.
cheers ./daniel
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2020-07-03 10:07 Laurenz Albe <[email protected]>
parent: Daniel Gustafsson <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Laurenz Albe @ 2020-07-03 10:07 UTC (permalink / raw)
To: Daniel Gustafsson <[email protected]>; Antonin Houska <[email protected]>; +Cc: Andy Fan <[email protected]>; legrand legrand <[email protected]>; pgsql-hackers
On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
> This version now fails to apply to HEAD, with what looks like like a trivial
> error in the expected test output. Can you please submit a rebased version so
> we can see it run in the patch tester CI? I'm marking the entry as Waiting on
> Author in the meantime.
I have rebased the patch against current HEAD, it passes "make installcheck".
Yours,
Laurenz Albe
Attachments:
[text/x-patch] v17-0001-Introduce-RelInfoList-structure.patch (14.2K, ../../[email protected]/2-v17-0001-Introduce-RelInfoList-structure.patch)
download | inline diff:
From 67fa4b73d1f8483d561b7a16907e9edbf00215c9 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <[email protected]>
Date: Fri, 3 Jul 2020 10:20:07 +0200
Subject: [PATCH 1/3] Introduce RelInfoList structure.
This patch puts join_rel_list and join_rel_hash fields of PlannerInfo
structure into a new structure RelInfoList. It also adjusts add_join_rel() and
find_join_rel() functions so they only call add_rel_info() and find_rel_info()
respectively.
fetch_upper_rel() now uses the new API and the hash table as well because the
list stored in root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG] will contain many
relations as soon as the aggregate push-down feature is added.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/nodes/outfuncs.c | 11 ++
src/backend/optimizer/geqo/geqo_eval.c | 12 +-
src/backend/optimizer/plan/planmain.c | 3 +-
src/backend/optimizer/util/relnode.c | 182 +++++++++++++++----------
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 30 ++--
7 files changed, 149 insertions(+), 93 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 9fc53cad68..a46834a377 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5251,7 +5251,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515d..dbd36408c3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2315,6 +2315,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(partitioned_child_rels);
}
+static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
@@ -4112,6 +4120,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index ff33acc7b6..6b04ab3c2a 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 62dfc6d44a..5fa33ec200 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -65,8 +65,7 @@ query_planner(PlannerInfo *root,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a203e6f1ff..a95e6364ae 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -32,11 +32,15 @@
#include "utils/lsyscache.h"
-typedef struct JoinHashEntry
+/*
+ * An entry of a hash table that we use to make lookup for RelOptInfo
+ * structures more efficient.
+ */
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -390,11 +394,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -403,47 +407,53 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
+
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
+ if (list == NULL)
+ return NULL;
+
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -453,34 +463,90 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids = NULL;
+
+ Assert(IsA(item, RelOptInfo));
- if (bms_equal(rel->relids, relids))
- return rel;
+ item_relids = ((RelOptInfo *) item)->relids;
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
return NULL;
}
+/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -531,32 +597,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
}
-/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
@@ -1191,22 +1231,14 @@ subbuild_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *
fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
{
+ RelInfoList *list = &root->upper_rels[kind];
RelOptInfo *upperrel;
- ListCell *lc;
-
- /*
- * For the moment, our indexing data structure is just a List for each
- * relation kind. If we ever get so many of one kind that this stops
- * working well, we can improve it. No code outside this function should
- * assume anything about how to find a particular upperrel.
- */
/* If we already made this upperrel for the query, return it */
- foreach(lc, root->upper_rels[kind])
+ if (list)
{
- upperrel = (RelOptInfo *) lfirst(lc);
-
- if (bms_equal(upperrel->relids, relids))
+ upperrel = find_rel_info(list, relids);
+ if (upperrel)
return upperrel;
}
@@ -1225,7 +1257,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->cheapest_unique_path = NULL;
upperrel->cheapest_parameterized_paths = NIL;
- root->upper_rels[kind] = lappend(root->upper_rels[kind], upperrel);
+ add_rel_info(&root->upper_rels[kind], upperrel);
return upperrel;
}
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4..be5ab273f0 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -222,6 +222,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 485d1b06c9..44374de796 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -93,6 +93,23 @@ typedef enum InheritanceKind
INHKIND_PARTITIONED
} InheritanceKind;
+/*
+ * Hashed list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
/*----------
* PlannerGlobal
* Global information for planning/optimization
@@ -236,15 +253,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -308,7 +319,8 @@ struct PlannerInfo
List *initial_rels; /* RelOptInfos we are now trying to join */
/* Use fetch_upper_rel() to get any particular upper rel */
- List *upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+ RelInfoList upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+
/* Result tlists chosen by grouping_planner for upper-stage processing */
struct PathTarget *upper_targets[UPPERREL_FINAL + 1];
--
2.21.3
[text/x-patch] v17-0002-Aggregate-push-down-basic-functionality.patch (107.6K, ../../[email protected]/3-v17-0002-Aggregate-push-down-basic-functionality.patch)
download | inline diff:
From 437b3c2e9295ab27c6b61ea21ac631fd135477b1 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <[email protected]>
Date: Fri, 3 Jul 2020 10:21:25 +0200
Subject: [PATCH 2/3] Aggregate push-down - basic functionality.
With this patch, partial aggregation can be applied to a base relation or to a
join, and the resulting "grouped" relations can be joined to other "plain"
relations. Once all tables are joined, the aggregation is finalized. See
README for more information.
The next patches will enable the aggregate push-down feature for parallel
query processing, for partitioned tables and for foreign tables.
---
src/backend/nodes/copyfuncs.c | 20 +-
src/backend/nodes/outfuncs.c | 34 +
src/backend/optimizer/README | 69 ++
src/backend/optimizer/path/allpaths.c | 147 ++++
src/backend/optimizer/path/costsize.c | 17 +-
src/backend/optimizer/path/equivclass.c | 130 +++
src/backend/optimizer/path/joinrels.c | 195 ++++-
src/backend/optimizer/plan/initsplan.c | 290 +++++++
src/backend/optimizer/plan/planmain.c | 12 +
src/backend/optimizer/plan/planner.c | 47 +-
src/backend/optimizer/plan/setrefs.c | 33 +
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/util/pathnode.c | 143 +++-
src/backend/optimizer/util/relnode.c | 923 ++++++++++++++++++++-
src/backend/optimizer/util/tlist.c | 31 +
src/backend/utils/misc/guc.c | 10 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/pathnodes.h | 89 ++
src/include/optimizer/clauses.h | 2 +
src/include/optimizer/pathnode.h | 19 +-
src/include/optimizer/paths.h | 6 +
src/include/optimizer/planmain.h | 1 +
src/include/optimizer/tlist.h | 4 +-
src/test/regress/expected/agg_pushdown.out | 215 +++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/agg_pushdown.sql | 114 +++
27 files changed, 2477 insertions(+), 81 deletions(-)
create mode 100644 src/test/regress/expected/agg_pushdown.out
create mode 100644 src/test/regress/sql/agg_pushdown.sql
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d0..c5ec1e7e9c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2251,8 +2251,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2395,6 +2395,19 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5167,6 +5180,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index dbd36408c3..c02bcac698 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2223,6 +2223,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2237,6 +2238,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2244,6 +2246,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2465,6 +2468,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2566,6 +2583,17 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
WRITE_INT_FIELD(ph_width);
}
+static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+}
+
static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
@@ -4144,6 +4172,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4162,6 +4193,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index d174b8cb73..a73a46ae31 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1154,3 +1154,72 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resulting paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.y)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Also note that in this case the partial aggregate uses the "b.j" as grouping
+column although the column does not appear in the query target list. The point
+is that "b.j" is needed to evaluate the join condition, and there's no other
+way for the partial aggregate to emit its values.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.y + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index d984da25d7..84a918dc58 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -60,6 +60,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -73,6 +74,7 @@ join_search_hook_type join_search_hook = NULL;
static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
+static void setup_base_grouped_rels(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte);
@@ -124,6 +126,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -184,6 +189,13 @@ make_one_rel(PlannerInfo *root, List *joinlist)
*/
set_base_rel_sizes(root);
+ /*
+ * Now that the sizes are known, we can estimate the sizes of the grouped
+ * relations.
+ */
+ if (root->grouped_var_list)
+ setup_base_grouped_rels(root);
+
/*
* We should now have size estimates for every actual table involved in
* the query, and we also know which if any have been deleted from the
@@ -324,6 +336,48 @@ set_base_rel_sizes(PlannerInfo *root)
}
}
+/*
+ * setup_based_grouped_rels
+ * For each "plain" relation build a grouped relation if aggregate pushdown
+ * is possible and if this relation is suitable for partial aggregation.
+ */
+static void
+setup_base_grouped_rels(PlannerInfo *root)
+{
+ Index rti;
+
+ for (rti = 1; rti < root->simple_rel_array_size; rti++)
+ {
+ RelOptInfo *brel = root->simple_rel_array[rti];
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* there may be empty slots corresponding to non-baserel RTEs */
+ if (brel == NULL)
+ continue;
+
+ Assert(brel->relid == rti); /* sanity check on array */
+
+ /*
+ * The aggregate push-down feature only makes sense if there are
+ * multiple base rels in the query.
+ */
+ if (!bms_nonempty_difference(root->all_baserels, brel->relids))
+ continue;
+
+ /* ignore RTEs that are "other rels" */
+ if (brel->reloptkind != RELOPT_BASEREL)
+ continue;
+
+ rel_grouped = build_simple_grouped_rel(root, brel->relid, &agg_info);
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+ }
+ }
+}
+
/*
* set_base_rel_pathlists
* Finds all paths available for scanning each base-relation entry.
@@ -496,8 +550,21 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
}
else
{
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
/* Plain relation */
set_plain_rel_pathlist(root, rel, rte);
+
+ /* Add paths to the grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids,
+ &agg_info);
+ if (rel_grouped)
+ {
+ generate_grouping_paths(root, rel_grouped, rel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
}
break;
case RTE_SUBQUERY:
@@ -2943,6 +3010,80 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
}
}
+/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
@@ -3084,6 +3225,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -3120,6 +3262,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 4ff3c7a2fd..9e415fd881 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4637,7 +4637,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5608,11 +5607,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5643,6 +5642,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index b99cec00cb..81970fed15 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2841,6 +2841,136 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rel(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+
+ return result;
+}
+
/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 2d343cd293..1b0dea491a 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -21,6 +21,7 @@
#include "optimizer/paths.h"
#include "partitioning/partbounds.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -35,6 +36,10 @@ static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -669,21 +674,20 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -744,7 +748,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -757,14 +761,175 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
return joinrel;
}
+/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
+/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
+}
+
/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index e978b491f6..b10cdc8368 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/nbtree.h"
#include "catalog/pg_class.h"
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
@@ -33,6 +34,7 @@
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
+#include "utils/typcache.h"
/* These parameters are set by GUC */
int from_collapse_limit;
@@ -47,6 +49,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -272,6 +276,292 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars, but we only need Aggrefs.
+ */
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+ TypeCacheEntry *tce;
+ Oid equalimageproc;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ /*
+ * Aggregate push-down is only possible if equality of grouping keys
+ * per the equality operator implies bitwise equality. Otherwise, if
+ * we put keys of different byte images into the same group, we lose
+ * some information that may be needed to evaluate join clauses above
+ * the pushed-down aggregate node, or the WHERE clause.
+ *
+ * For example, the NUMERIC data type is not supported because values
+ * that fall into the same group according to the equality operator
+ * (e.g. 0 and 0.0) can have different scale.
+ */
+ tce = lookup_type_cache(exprType((Node *) te->expr),
+ TYPECACHE_BTREE_OPFAMILY);
+ if (!OidIsValid(tce->btree_opf) ||
+ !OidIsValid(tce->btree_opintype))
+ goto fail;
+
+ equalimageproc = get_opfamily_proc(tce->btree_opf,
+ tce->btree_opintype,
+ tce->btree_opintype,
+ BTEQUALIMAGE_PROC);
+ if (!OidIsValid(equalimageproc) ||
+ !DatumGetBool(OidFunctionCall1Coll(equalimageproc,
+ tce->typcollation,
+ ObjectIdGetDatum(tce->btree_opintype))))
+ goto fail;
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ return;
+
+fail:
+ root->grouped_var_list = NIL;
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 5fa33ec200..cb82d3b8de 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -66,6 +66,7 @@ query_planner(PlannerInfo *root,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -74,6 +75,7 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -252,6 +254,16 @@ query_planner(PlannerInfo *root,
*/
extract_restriction_or_clauses(root);
+ /*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
/*
* Now expand appendrels by adding "otherrels" for their children. We
* delay this to the end so that we have as much information as possible
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 4131019fc9..6fe12d08ce 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -625,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -3893,7 +3894,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -4100,11 +4102,11 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
bool force_rel_creation;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4137,10 +4139,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -6868,6 +6874,13 @@ create_partial_grouping_paths(PlannerInfo *root,
bool can_hash = (extra->flags & GROUPING_CAN_USE_HASH) != 0;
bool can_sort = (extra->flags & GROUPING_CAN_USE_SORT) != 0;
+ /*
+ * The output relation could have been already created due to aggregate
+ * push-down.
+ */
+ partially_grouped_rel = find_grouped_rel(root, input_rel->relids, NULL);
+ Assert(enable_agg_pushdown || partially_grouped_rel == NULL);
+
/*
* Consider whether we should generate partially aggregated non-partial
* paths. We can only do this if we have a non-partial path, and only if
@@ -6894,16 +6907,18 @@ create_partial_grouping_paths(PlannerInfo *root,
*/
if (cheapest_total_path == NULL &&
cheapest_partial_path == NULL &&
- !force_rel_creation)
+ !force_rel_creation &&
+ partially_grouped_rel == NULL)
return NULL;
/*
* Build a new upper relation to represent the result of partially
* aggregating the rows from the input relation.
*/
- partially_grouped_rel = fetch_upper_rel(root,
- UPPERREL_PARTIAL_GROUP_AGG,
- grouped_rel->relids);
+ if (partially_grouped_rel == NULL)
+ partially_grouped_rel = fetch_upper_rel(root,
+ UPPERREL_PARTIAL_GROUP_AGG,
+ grouped_rel->relids);
partially_grouped_rel->consider_parallel =
grouped_rel->consider_parallel;
partially_grouped_rel->reloptkind = grouped_rel->reloptkind;
@@ -6917,10 +6932,14 @@ create_partial_grouping_paths(PlannerInfo *root,
* emit the same tlist as regular aggregate paths, because (1) we must
* include Vars and Aggrefs needed in HAVING, which might not appear in
* the result tlist, and (2) the Aggrefs must be set in partial mode.
+ *
+ * If the target was already created for the sake of aggregate push-down,
+ * it should be compatible with what we'd create here.
*/
- partially_grouped_rel->reltarget =
- make_partial_grouping_target(root, grouped_rel->reltarget,
- extra->havingQual);
+ if (partially_grouped_rel->reltarget->exprs == NIL)
+ partially_grouped_rel->reltarget =
+ make_partial_grouping_target(root, grouped_rel->reltarget,
+ extra->havingQual);
if (!extra->partial_costs_set)
{
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index baefe0e946..026c098d11 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2457,6 +2457,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 14521728c6..d34065d61e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -923,6 +923,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e845a4b1ae..437d6d3f42 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2528,8 +2528,7 @@ create_projection_path(PlannerInfo *root,
pathnode->path.pathtype = T_Result;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
@@ -3020,6 +3019,146 @@ create_agg_path(PlannerInfo *root,
return pathnode;
}
+/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /* The agg path should require no fewer parameters than the plain one. */
+ result->path.param_info = subpath->param_info;
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ /* Do not try to create hash table for each parameter value. */
+ Assert(subpath->param_info == NULL);
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+ }
+ }
+
+ return result;
+}
+
/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a95e6364ae..7460794b46 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -18,17 +18,23 @@
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/inherit.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
#include "utils/lsyscache.h"
@@ -76,6 +82,11 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
+static bool is_var_in_aggref_only(PlannerInfo *root, Var *var);
+static bool is_var_needed_by_join(PlannerInfo *root, Var *var, RelOptInfo *rel);
/*
@@ -370,6 +381,109 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
return rel;
}
+/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ /*
+ * The number of output rows is supposedly different (lower) due to
+ * grouping.
+ */
+ rel_grouped->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
@@ -484,9 +598,13 @@ find_rel_info(RelInfoList *list, Relids relids)
void *item = lfirst(l);
Relids item_relids = NULL;
- Assert(IsA(item, RelOptInfo));
+ Assert(IsA(item, RelOptInfo) || IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
- item_relids = ((RelOptInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
}
@@ -514,7 +632,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) || IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -526,7 +644,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -547,6 +669,63 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
add_rel_info(root->join_rel_list, joinrel);
}
+/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG], rel);
+ add_rel_info(root->agg_info_list, agg_info);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ *
+ * The call fetch_upper_rel(root, UPPERREL_PARTIAL_GROUP_AGG, ...) should
+ * return the same relation if it exists, however the behavior is different if
+ * the relation is not there. find_grouped_rel() should be used in
+ * query_planner() and subroutines.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG],
+ relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -609,6 +788,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -619,10 +799,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -630,7 +812,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -725,9 +908,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -759,49 +954,75 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
+ if (!grouped)
+ {
+ /*
+ * Set estimates of the joinrel's size.
+ */
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+ }
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
}
return joinrel;
@@ -2059,3 +2280,621 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rel(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ foreach(lc, aggregates)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ result = makeNode(RelAggInfo);
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(target->exprs, lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc;
+ List *possibly_dependent = NIL;
+ Var *tvar;
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Index sortgroupref;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc);
+
+ sortgroupref = get_expression_sortgroupref((Expr *) tvar, gvis);
+ if (sortgroupref > 0)
+ {
+ /*
+ * If the target expression can be used as the grouping key, we
+ * don't have to worry whether it can be emitted by the AggPath
+ * pushed down to relation / join.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Another reason we might need this variable is that some
+ * aggregate pushed down to this relation references it. In such a
+ * case, add that var to agg_input, but not to "target". However,
+ * if the aggregate is not the only reason for the var to be in
+ * the target, some more checks need to be performed below.
+ */
+ if (is_var_in_aggref_only(root, tvar))
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ else if (is_var_needed_by_join(root, tvar, rel))
+ {
+ /*
+ * The variable is needed for a join, however it's neither in
+ * the GROUP BY clause nor can it be derived from it using EC.
+ * (Otherwise it would have to be added to the targets above.)
+ * We need to construct special SortGroupClause for that
+ * variable.
+ *
+ * Note that its tleSortGroupRef needs to be unique within
+ * agg_input, so we need to postpone creation of the
+ * SortGroupClause's until we're done with the iteration of
+ * rel->reltarget->exprs. Also it makes sense for the caller
+ * to do some more check before it starts to create those
+ * SortGroupClause's.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, tvar);
+ }
+ else
+ {
+ /*
+ * The Var can be functionally dependent on another expression
+ * of the target, but we cannot check until the other
+ * expressions are in the target.
+ */
+ possibly_dependent = lappend(possibly_dependent, tvar);
+ }
+ }
+ }
+
+ /*
+ * Now we can check whether the expression is functionally dependent on
+ * another one.
+ */
+ foreach(lc, possibly_dependent)
+ {
+ List *deps = NIL;
+ RangeTblEntry *rte;
+
+ tvar = lfirst_node(Var, lc);
+ rte = root->simple_rte_array[tvar->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (check_functional_grouping(rte->relid, tvar->varno,
+ tvar->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used for grouping key evaluation
+ * (instead, the one this depends on will be), so sortgroupref
+ * should not be important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) tvar);
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * If the aggregate push-down will support generic grouping
+ * expression sin the future, create_rel_agg_info() will have to
+ * add this variable to "agg_input" target and also add the whole
+ * generic expression to "target".
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/*
+ * Check whether given variable appears in Aggref(s) which we consider usable
+ * at relation / join level, and only in the Aggref(s).
+ */
+static bool
+is_var_in_aggref_only(PlannerInfo *root, Var *var)
+{
+ ListCell *lc;
+ bool found = false;
+
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ ListCell *lc2;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(var->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc2, vars)
+ {
+ Var *v = lfirst_node(Var, lc2);
+
+ if (equal(v, var))
+ {
+ found = true;
+ break;
+ }
+
+ }
+ list_free(vars);
+
+ if (found)
+ break;
+ }
+
+ /* No aggregate references the Var? */
+ if (!found)
+ return false;
+
+ /* Does the Var appear in the target outside aggregates? */
+ found = false;
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, var))
+ return false;
+
+ }
+
+ /* The Var is in aggregate(s) and only there. */
+ return true;
+}
+
+/*
+ * Check if given variable is needed by joins above the current rel?
+ *
+ * Consider pushing the aggregate avg(b.y) down to relation "b" for the
+ * following query:
+ *
+ * SELECT a.i, avg(b.y)
+ * FROM a JOIN b ON b.j = a.i
+ * GROUP BY a.i;
+ *
+ * If we aggregate the "b" relation alone, the column "b.j" needs to be used
+ * as the grouping key because otherwise it cannot find its way to the input
+ * of the join expression.
+ */
+static bool
+is_var_needed_by_join(PlannerInfo *root, Var *var, RelOptInfo *rel)
+{
+ Relids relids_no_top;
+ int ndx;
+ RelOptInfo *baserel;
+
+ /*
+ * The relids we're not interested in do include 0, which is the top-level
+ * targetlist. The only reason for relids to contain 0 should be that
+ * arg_var is referenced either by aggregate or by grouping expression,
+ * but right now we're interested in the *other* reasons. (As soon
+ * aggregation is pushed down, the aggregates in the query targetlist no
+ * longer need direct reference to arg_var anyway.)
+ */
+
+ relids_no_top = bms_copy(rel->relids);
+ bms_add_member(relids_no_top, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_no_top))
+ return true;
+
+ return false;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 02a3c6b165..5011afc033 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -825,6 +825,37 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
}
+/*
+ * Return sortgroupref if expr can be used as the grouping expression in an
+ * AggPath at relation or join level, or 0 if it can't.
+ *
+ * gvis a list of a list of GroupedVarInfo's available for the query,
+ * including those derived using equivalence classes.
+ */
+Index
+get_expression_sortgroupref(Expr *expr, List *gvis)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ return gvi->sortgroupref;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return 0;
+}
+
/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 75fc6f11d6..a06dd52a50 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1081,6 +1081,16 @@ static struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregate push-down."),
+ NULL,
+ GUC_EXPLAIN
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index be5ab273f0..62134356a0 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -226,6 +226,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -271,6 +272,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 44374de796..98be8b7de2 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -304,6 +304,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -321,6 +323,12 @@ struct PlannerInfo
/* Use fetch_upper_rel() to get any particular upper rel */
RelInfoList upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+ /*
+ * One instance of RelAggInfo per item of the
+ * upper_rels[UPPERREL_PARTIAL_GROUP_AGG] list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
/* Result tlists chosen by grouping_planner for upper-stage processing */
struct PathTarget *upper_targets[UPPERREL_FINAL + 1];
@@ -336,6 +344,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -787,6 +801,60 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
@@ -2323,6 +2391,27 @@ typedef struct PlaceHolderInfo
int32 ph_width; /* estimated attribute width */
} PlaceHolderInfo;
+/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ *
+ * TODO Rename, perhaps to GroupedTargetEntry? (Also rename the variables of
+ * this type.)
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+} GroupedVarInfo;
+
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index b7456e3e59..0464531e25 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rel(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 715a24ad29..47afa7ed1a 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -216,6 +216,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -287,14 +295,21 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -320,5 +335,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 10b6e81079..ee34442772 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -56,6 +57,11 @@ extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
extern void generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index f3cefe67b8..c2ab6b2115 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -72,6 +72,7 @@ extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 1d4c7da545..e2547732d6 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -50,8 +50,10 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location for this one. */
+extern Index get_expression_sortgroupref(Expr *expr, List *gvis);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
-
#endif /* TLIST_H */
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..413abc147e
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,215 @@
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 01b7786f01..5c15e2847d 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -90,7 +91,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(18 rows)
+(19 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 979d926119..71a1c31431 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -146,6 +146,7 @@ test: stats_ext
test: collate.linux.utf8
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: select_views
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6bbbc7a8a1
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,114 @@
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.21.3
[text/x-patch] v17-0003-Use-also-partial-paths-as-the-input-for-grouped-path.patch (17.1K, ../../[email protected]/4-v17-0003-Use-also-partial-paths-as-the-input-for-grouped-path.patch)
download | inline diff:
From cb03357f4413484e2c8a8bd96474c610484e81e3 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <[email protected]>
Date: Fri, 3 Jul 2020 10:22:14 +0200
Subject: [PATCH 3/3] Use also partial paths as the input for grouped paths.
---
src/backend/optimizer/path/allpaths.c | 46 +++++-
src/backend/optimizer/util/relnode.c | 46 +++---
src/test/regress/expected/agg_pushdown.out | 157 +++++++++++++++++++++
src/test/regress/sql/agg_pushdown.sql | 66 +++++++++
4 files changed, 285 insertions(+), 30 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 84a918dc58..e0b5418de4 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -128,7 +128,7 @@ static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath, AggStrategy aggstrategy,
- RelAggInfo *agg_info);
+ RelAggInfo *agg_info, bool partial);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -3021,6 +3021,7 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
RelOptInfo *rel_plain, RelAggInfo *agg_info)
{
ListCell *lc;
+ Path *path;
if (IS_DUMMY_REL(rel_plain))
{
@@ -3030,7 +3031,7 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
foreach(lc, rel_plain->pathlist)
{
- Path *path = (Path *) lfirst(lc);
+ path = (Path *) lfirst(lc);
/*
* Since the path originates from the non-grouped relation which is
@@ -3044,7 +3045,8 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
* add_grouped_path() will check whether the path has suitable
* pathkeys.
*/
- add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info,
+ false);
/*
* Repeated creation of hash table (for new parameter values) should
@@ -3052,12 +3054,38 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
* efficiency.
*/
if (path->param_info == NULL)
- add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info,
+ false);
}
/* Could not generate any grouped paths? */
if (rel_grouped->pathlist == NIL)
+ {
mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ /*
+ * Almost the same for partial paths.
+ *
+ * The difference is that parameterized paths are never created, see
+ * add_partial_path() for explanation.
+ */
+ foreach(lc, rel_plain->partial_pathlist)
+ {
+ path = (Path *) lfirst(lc);
+
+ if (path->param_info != NULL)
+ continue;
+
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info,
+ true);
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info,
+ true);
+ }
}
/*
@@ -3065,7 +3093,8 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
*/
static void
add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
- AggStrategy aggstrategy, RelAggInfo *agg_info)
+ AggStrategy aggstrategy, RelAggInfo *agg_info,
+ bool partial)
{
Path *agg_path;
@@ -3081,7 +3110,12 @@ add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
/* Add the grouped path to the list of grouped base paths. */
if (agg_path != NULL)
- add_path(rel, (Path *) agg_path);
+ {
+ if (!partial)
+ add_path(rel, (Path *) agg_path);
+ else
+ add_partial_path(rel, (Path *) agg_path);
+ }
}
/*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 7460794b46..4164f4b0e4 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -958,33 +958,12 @@ build_join_rel(PlannerInfo *root,
build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
restrictlist, sjinfo->jointype);
+ /*
+ * Set estimates of the joinrel's size.
+ */
if (!grouped)
- {
- /*
- * Set estimates of the joinrel's size.
- */
set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted
- * expressions in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they
- * could be divided between inner_rel and outer_rel in any arbitrary
- * way. We assume this doesn't matter, because we should hit all the
- * same baserels and joinclauses while building up to this joinrel no
- * matter which we take; therefore, we should make the same decision
- * here however we get here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
- }
else
{
/*
@@ -1000,6 +979,25 @@ build_join_rel(PlannerInfo *root,
agg_info->input_rows, NULL);
}
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted expressions
+ * in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they could
+ * be divided between inner_rel and outer_rel in any arbitrary way. We
+ * assume this doesn't matter, because we should hit all the same baserels
+ * and joinclauses while building up to this joinrel no matter which we
+ * take; therefore, we should make the same decision here however we get
+ * here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+
/* Add the joinrel to the PlannerInfo. */
if (!grouped)
add_join_rel(root, joinrel);
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
index 413abc147e..66d36d122e 100644
--- a/src/test/regress/expected/agg_pushdown.out
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -91,6 +91,7 @@ AS c1 ON c1.parent = p.i GROUP BY p.i;
-> Seq Scan on agg_pushdown_child1 c1
(12 rows)
+-- Restore the default values.
SET enable_nestloop TO on;
SET enable_hashjoin TO on;
-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
@@ -213,3 +214,159 @@ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
-> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
(13 rows)
+-- Most of the tests above with parallel query processing enforced.
+SET min_parallel_index_scan_size = 0;
+SET min_parallel_table_scan_size = 0;
+SET parallel_setup_cost = 0;
+SET parallel_tuple_cost = 0;
+-- Partially aggregate a single relation.
+--
+-- Nestloop join.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- Hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
+-- Merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(10 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(15 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Sort
+ Sort Key: p.i
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Parallel Hash
+ -> Parallel Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(17 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(15 rows)
+
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
index 6bbbc7a8a1..49ba6dd67c 100644
--- a/src/test/regress/sql/agg_pushdown.sql
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -61,6 +61,7 @@ EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
+-- Restore the default values.
SET enable_nestloop TO on;
SET enable_hashjoin TO on;
@@ -112,3 +113,68 @@ EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- Most of the tests above with parallel query processing enforced.
+SET min_parallel_index_scan_size = 0;
+SET min_parallel_table_scan_size = 0;
+SET parallel_setup_cost = 0;
+SET parallel_tuple_cost = 0;
+
+-- Partially aggregate a single relation.
+--
+-- Nestloop join.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- Hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- Merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.21.3
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2021-03-03 16:42 David Steele <[email protected]>
parent: Laurenz Albe <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: David Steele @ 2021-03-03 16:42 UTC (permalink / raw)
To: Laurenz Albe <[email protected]>; Antonin Houska <[email protected]>; Tom Lane <[email protected]>; Tomas Vondra <[email protected]>; +Cc: Andy Fan <[email protected]>; legrand legrand <[email protected]>; pgsql-hackers; Daniel Gustafsson <[email protected]>
On 7/3/20 6:07 AM, Laurenz Albe wrote:
> On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
>> This version now fails to apply to HEAD, with what looks like like a trivial
>> error in the expected test output. Can you please submit a rebased version so
>> we can see it run in the patch tester CI? I'm marking the entry as Waiting on
>> Author in the meantime.
>
> I have rebased the patch against current HEAD, it passes "make installcheck".
This needs another rebase so marked as Waiting on Author:
http://cfbot.cputube.org/patch_32_1247.log
As far as I can see reviewer concerns have been addressed as they have
come up, but as Tomas noted this is a bit of a niche feature as far as
who might commit it.
Tom, Tomas, would you care to share your remaining concerns, if any?
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2021-03-11 10:10 Antonin Houska <[email protected]>
parent: David Steele <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: Antonin Houska @ 2021-03-11 10:10 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: pgsql-hackers
David Steele <[email protected]> wrote:
> On 7/3/20 6:07 AM, Laurenz Albe wrote:
> > On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
> >> This version now fails to apply to HEAD, with what looks like like a trivial
> >> error in the expected test output. Can you please submit a rebased version so
> >> we can see it run in the patch tester CI? I'm marking the entry as Waiting on
> >> Author in the meantime.
> >
> > I have rebased the patch against current HEAD, it passes "make installcheck".
>
> This needs another rebase so marked as Waiting on Author:
> http://cfbot.cputube.org/patch_32_1247.log
Besides the fact that I don't have time to work on it now, I'm not able to
convince myself to put more effort into it. If there's almost no progress for
several years, I don't believe another rebase(s) is anything but wasted
effort.
So I've withdrawn the patch, also to save CFMs from examining the history
again and again uselessly. The code is free so anyone can continue if he
thinks it makes sense. If it finally finds its way into the PG core and if
meaningful part of my code remains, I'd appreciate it if I was mentioned in
the release notes. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 59+ messages in thread
* Re: WIP: Aggregation push-down
@ 2021-03-11 13:34 David Steele <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 59+ messages in thread
From: David Steele @ 2021-03-11 13:34 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: pgsql-hackers
On 3/11/21 5:10 AM, Antonin Houska wrote:
> David Steele <[email protected]> wrote:
>
>> On 7/3/20 6:07 AM, Laurenz Albe wrote:
>>> On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
>>>> This version now fails to apply to HEAD, with what looks like like a trivial
>>>> error in the expected test output. Can you please submit a rebased version so
>>>> we can see it run in the patch tester CI? I'm marking the entry as Waiting on
>>>> Author in the meantime.
>>>
>>> I have rebased the patch against current HEAD, it passes "make installcheck".
>>
>> This needs another rebase so marked as Waiting on Author:
>> http://cfbot.cputube.org/patch_32_1247.log
>
> Besides the fact that I don't have time to work on it now, I'm not able to
> convince myself to put more effort into it. If there's almost no progress for
> several years, I don't believe another rebase(s) is anything but wasted
> effort.
>
> So I've withdrawn the patch, also to save CFMs from examining the history
> again and again uselessly. The code is free so anyone can continue if he
> thinks it makes sense. If it finally finds its way into the PG core and if
> meaningful part of my code remains, I'd appreciate it if I was mentioned in
> the release notes. Thanks.
I'm sure you would receive credit if anyone picked this patch up.
Sorry it didn't work out, but it looks like withdrawing the patch was
the right way to go.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 59+ messages in thread
* RE: WIP: Aggregation push-down
@ 2022-03-21 21:09 [email protected] <[email protected]>
parent: David Steele <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: [email protected] @ 2022-03-21 21:09 UTC (permalink / raw)
To: pgsql-hackers; +Cc: [email protected] <[email protected]>
Hi everyone.
I develop postgresql's extension such as fdw in my work.
I'm interested in using postgresql for OLAP.
I think that this patch is realy useful when using OLAP queries.
Furthermore, I think it would be more useful if this patch works on a foreign table.
Actually, I changed this patch a little and confirmed that my idea is true.
The followings are things I found and differences of between my prototype and this patch.
1. Things I found
I execute a query which contain join of postgres_fdw's foreign table and a table and aggregation of the josin result.
In my setting, my prototype reduce this query's response by 93%.
2. Differences between my prototype and this patch
(1) Pushdown aggregation of foeign table if FDW pushdown partioal aggregation
(2) postgres_fdw pushdowns some partial aggregations
I attached my prototype source code and content of my experiment.
I want to resume development of this patch if there is some possibility of accept of this patch's function.
. I took a contact to Mr.Houska on resuming development of this patch.
As a result, Mr.Houska advised for me that I ask in pgsql-hackers whether any reviewers / committers are
interested to work on the patch.
Is anyone interested in my work?
Sincerely yours.
Yuuki Fujii
--
Yuuki Fujii
Information Technology R&D Center Mitsubishi Electric Corporation
1. Restrictions of my prototype
(1) aggregate functions are avg, sum, count, min, max
(2) argment or sum of avg is bigint var
2. My experiment settings
(1) hardware settings
cpu:Intel Corei5-9500 processor(3.0 GHz/9MB cache)(6 cores)
ram:DDR4 3200Mhz 128GB
storage:512GB SSD(M.2 NVMe)
(2) software
os: CentOS7
postgresql source code:14(in development) with commit-id 947456a823d6b0973b68c6b38c8623a0504054e7
# build by "make world"
patchs I applied:
v17-0001-Introduce-RelInfoList-structure.patch
v17-0002-Aggregate-push-down-basic-functionality.patch
v17-0003-Use-also-partial-paths-as-the-input-for-grouped-path.patch
(3) PostgreSQL settings
max_parallel_workers_per_gather=6
work_mem=1000000
(4) tables and data
execute the following commands using postgresql's client tool such as psql
--
\c tmp;
create table log as select 1::bigint as id, 1::bigint as size, 1::bigint as total from generate_series(1,1000000000) t;
create table master as select t as id, 'hoge' || mod(t, 1000) as name from generate_series(1,1000000) t;
create index master_idx on master(id);
create extension postgres_fdw;
create server local_server foreign data wrapper postgres_fdw OPTIONS (host 'localhost', port '5432', dbname 'tmp');
create user mapping for user server local_server options (user 'postgres', password 'postgres');
create foreign table f_log(id bigint, size bigint, total bigint) server local_server options (table_name 'log');
analyze;
set enable_agg_pushdown=true;
explain (verbose, analyze) select b.name, avg(a.total) from f_log a join master b on a.id = b.id group by b.name;
set enable_agg_pushdown=false;
explain (verbose, analyze) select b.name, avg(a.total) from f_log a join master b on a.id = b.id group by b.name;
--
Attachments:
[application/octet-stream] v17-0004-pushdown-aggregation-foreign-table.patch (26.9K, ../../OS3PR01MB66609239ED8E3F7C3892070895169@OS3PR01MB6660.jpnprd01.prod.outlook.com/2-v17-0004-pushdown-aggregation-foreign-table.patch)
download | inline diff:
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index ad37a74221..ef4d8ab120 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -100,8 +100,21 @@ typedef struct deparse_expr_cxt
* a base relation. */
StringInfo buf; /* output buffer to append to */
List **params_list; /* exprs that will become remote Params */
+ RelAggInfo *agg_info;
+ List **avg_attrs;
+ List **sum_attrs;
+ AggSplit aggsplit;
} deparse_expr_cxt;
+typedef enum {
+ AGGFUNC_SUM,
+ AGGFUNC_AVG,
+ AGGFUNC_COUNT,
+ AGGFUNC_MAX,
+ AGGFUNC_MIN,
+ AGGFUNC_OTHER
+} AggFuncType;
+
#define REL_ALIAS_PREFIX "r"
/* Handy macro to add relation name qualification */
#define ADD_REL_QUALIFIER(buf, varno) \
@@ -184,6 +197,7 @@ static void appendAggOrderBy(List *orderList, List *targetList,
static void appendFunctionName(Oid funcid, deparse_expr_cxt *context);
static Node *deparseSortGroupClause(Index ref, List *tlist, bool force_colno,
deparse_expr_cxt *context);
+static AggFuncType getAggFunc(Aggref* aggref);
/*
* Helper functions
@@ -693,6 +707,7 @@ foreign_expr_walker(Node *node,
case T_Aggref:
{
Aggref *agg = (Aggref *) node;
+ AggFuncType aggfunc_type = AGGFUNC_OTHER;
ListCell *lc;
/* Not safe to pushdown when not in grouping context */
@@ -700,8 +715,15 @@ foreign_expr_walker(Node *node,
return false;
/* Only non-split aggregates are pushable. */
- if (agg->aggsplit != AGGSPLIT_SIMPLE)
+ if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL) {
+ fpinfo->aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ aggfunc_type = getAggFunc(agg);
+ if (aggfunc_type == AGGFUNC_OTHER) {
+ return false;
+ }
+ } else if (agg->aggsplit != AGGSPLIT_SIMPLE) {
return false;
+ }
/* As usual, it must be shippable. */
if (!is_shippable(agg->aggfnoid, ProcedureRelationId, fpinfo))
@@ -723,6 +745,18 @@ foreign_expr_walker(Node *node,
n = (Node *) tle->expr;
}
+ if (fpinfo->aggsplit == AGGSPLIT_INITIAL_SERIAL) {
+ bool pushdown = false;
+ if (nodeTag(n) == T_Var) {
+ Var *var = (Var*)n;
+ if (var->vartype == INT8OID) {
+ pushdown = true;
+ }
+ }
+ if (pushdown == false) {
+ return false;
+ }
+ }
if (!foreign_expr_walker(n, glob_cxt, &inner_cxt))
return false;
@@ -995,7 +1029,8 @@ void
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
List *tlist, List *remote_conds, List *pathkeys,
bool has_final_sort, bool has_limit, bool is_subquery,
- List **retrieved_attrs, List **params_list)
+ List **retrieved_attrs, List **params_list, List **avg_attrs,
+ List **sum_attrs)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
@@ -1013,6 +1048,10 @@ deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
context.foreignrel = rel;
context.scanrel = IS_UPPER_REL(rel) ? fpinfo->outerrel : rel;
context.params_list = params_list;
+ context.agg_info = fpinfo->agg_info;
+ context.avg_attrs = avg_attrs;
+ context.sum_attrs = sum_attrs;
+ context.aggsplit = fpinfo->aggsplit;
/* Construct SELECT clause */
deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context);
@@ -1075,7 +1114,7 @@ deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
- deparse_expr_cxt *context)
+ deparse_expr_cxt *context)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
@@ -1391,6 +1430,142 @@ get_jointype_name(JoinType jointype)
return NULL;
}
+/* Get function name */
+static StringInfo getFuncName(Oid funcid) {
+ StringInfo buf;
+ HeapTuple proctup;
+ Form_pg_proc procform;
+ const char* proname;
+
+ buf = makeStringInfo();
+ initStringInfo(buf);
+
+ proctup = SearchSysCache1(PROCOID, ObjectIdGetDatum(funcid));
+ if (!HeapTupleIsValid(proctup))
+ elog(ERROR, "cache lookup failed for function %u", funcid);
+ procform = (Form_pg_proc)GETSTRUCT(proctup);
+
+ /* Print schema name only if it's not pg_catalog */
+ if (procform->pronamespace != PG_CATALOG_NAMESPACE)
+ {
+ const char* schemaname;
+
+ schemaname = get_namespace_name(procform->pronamespace);
+ appendStringInfo(buf, "%s.", quote_identifier(schemaname));
+ }
+
+ /* Always print the function name */
+ proname = NameStr(procform->proname);
+ appendStringInfoString(buf, quote_identifier(proname));
+
+ ReleaseSysCache(proctup);
+
+ return buf;
+}
+
+/* Get name of aggregation function */
+static AggFuncType getAggFunc(Aggref *aggref) {
+ AggFuncType type = AGGFUNC_OTHER;
+ StringInfo buf = getFuncName(aggref->aggfnoid);
+ if (strcmp(buf->data, "avg") == 0) {
+ type = AGGFUNC_AVG;
+ } else if (strcmp(buf->data, "sum") == 0) {
+ type = AGGFUNC_SUM;
+ } else if (strcmp(buf->data, "count") == 0) {
+ type = AGGFUNC_COUNT;
+ } else if (strcmp(buf->data, "min") == 0) {
+ type = AGGFUNC_MIN;
+ } else if (strcmp(buf->data, "max") == 0) {
+ type = AGGFUNC_MAX;
+ } else {
+ type = AGGFUNC_OTHER;
+ }
+ return type;
+}
+
+/* Deparse arg of aggref */
+static void deparseAggrefArg(Aggref * node, deparse_expr_cxt *context){
+ StringInfo buf = context->buf;
+ bool use_variadic;
+
+ use_variadic = node->aggvariadic;
+
+ appendStringInfoChar(buf, '(');
+
+ /* Add DISTINCT */
+ appendStringInfoString(buf, (node->aggdistinct != NIL) ? "DISTINCT " : "");
+
+ if (AGGKIND_IS_ORDERED_SET(node->aggkind))
+ {
+ /* Add WITHIN GROUP (ORDER BY ..) */
+ ListCell* arg;
+ bool first = true;
+
+ Assert(!node->aggvariadic);
+ Assert(node->aggorder != NIL);
+
+ foreach(arg, node->aggdirectargs)
+ {
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseExpr((Expr*)lfirst(arg), context);
+ }
+
+ appendStringInfoString(buf, ") WITHIN GROUP (ORDER BY ");
+ appendAggOrderBy(node->aggorder, node->args, context);
+ }
+ else
+ {
+ /* aggstar can be set only in zero-argument aggregates */
+ if (node->aggstar)
+ appendStringInfoChar(buf, '*');
+ else
+ {
+ ListCell* arg;
+ bool first = true;
+
+ /* Add all the arguments */
+ foreach(arg, node->args)
+ {
+ TargetEntry* tle = (TargetEntry*)lfirst(arg);
+ Node* n = (Node*)tle->expr;
+
+ if (tle->resjunk)
+ continue;
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ /* Add VARIADIC */
+ if (use_variadic && lnext(node->args, arg) == NULL)
+ appendStringInfoString(buf, "VARIADIC ");
+
+ deparseExpr((Expr*)n, context);
+ }
+ }
+
+ /* Add ORDER BY */
+ if (node->aggorder != NIL)
+ {
+ appendStringInfoString(buf, " ORDER BY ");
+ appendAggOrderBy(node->aggorder, node->args, context);
+ }
+ }
+
+ /* Add FILTER (WHERE ..) */
+ if (node->aggfilter != NULL)
+ {
+ appendStringInfoString(buf, ") FILTER (WHERE ");
+ deparseExpr((Expr*)node->aggfilter, context);
+ }
+
+ appendStringInfoChar(buf, ')');
+}
+
+
/*
* Deparse given targetlist and append it to context->buf.
*
@@ -1417,13 +1592,33 @@ deparseExplicitTargetList(List *tlist,
foreach(lc, tlist)
{
TargetEntry *tle = lfirst_node(TargetEntry, lc);
+ AggFuncType type;
if (i > 0)
appendStringInfoString(buf, ", ");
else if (is_returning)
appendStringInfoString(buf, " RETURNING ");
- deparseExpr((Expr *) tle->expr, context);
+ if ((nodeTag((Expr*)tle->expr) == T_Aggref)
+ && (context->aggsplit == AGGSPLIT_INITIAL_SERIAL)) {
+ type = getAggFunc((Aggref*)tle->expr);
+ if (type == AGGFUNC_AVG) {
+ *(context->avg_attrs) = lappend_int(*(context->avg_attrs), i + 1);
+ appendStringInfoString(buf, "count");
+ deparseAggrefArg((Aggref*)tle->expr, context);
+
+ appendStringInfoString(buf, ", ");
+ appendStringInfoString(buf, "sum");
+ deparseAggrefArg((Aggref*)tle->expr, context);
+ } else if (type == AGGFUNC_SUM) {
+ *(context->sum_attrs) = lappend_int(*(context->sum_attrs), i + 1);
+ deparseExpr((Expr*)tle->expr, context);
+ } else {
+ deparseExpr((Expr*)tle->expr, context);
+ }
+ } else {
+ deparseExpr((Expr*)tle->expr, context);
+ }
*retrieved_attrs = lappend_int(*retrieved_attrs, i + 1);
i++;
@@ -1646,6 +1841,8 @@ deparseRangeTblRef(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
if (make_subquery)
{
List *retrieved_attrs;
+ List *avg_attrs = NIL;
+ List *sum_attrs = NIL;
int ncols;
/*
@@ -1661,7 +1858,8 @@ deparseRangeTblRef(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
fpinfo->remote_conds, NIL,
false, false, true,
- &retrieved_attrs, params_list);
+ &retrieved_attrs, params_list, &avg_attrs,
+ &sum_attrs);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
@@ -3148,9 +3346,17 @@ appendGroupByClause(List *tlist, deparse_expr_cxt *context)
Query *query = context->root->parse;
ListCell *lc;
bool first = true;
+ List *groupClause;
+
+ if (context->agg_info) {
+ groupClause = context->agg_info->group_clauses;
+ }
+ else {
+ groupClause = query->groupClause;
+ }
/* Nothing to be done, if there's no GROUP BY clause in the query. */
- if (!query->groupClause)
+ if (!groupClause)
return;
appendStringInfoString(buf, " GROUP BY ");
@@ -3161,7 +3367,7 @@ appendGroupByClause(List *tlist, deparse_expr_cxt *context)
*/
Assert(!query->groupingSets);
- foreach(lc, query->groupClause)
+ foreach(lc, groupClause)
{
SortGroupClause *grp = (SortGroupClause *) lfirst(lc);
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index a46834a377..f99a0e2b7f 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -44,6 +44,8 @@
#include "utils/rel.h"
#include "utils/sampling.h"
#include "utils/selfuncs.h"
+#include "utils/numeric.h"
+#include "libpq/pqformat.h"
PG_MODULE_MAGIC;
@@ -72,6 +74,10 @@ enum FdwScanPrivateIndex
/* Integer representing the desired fetch_size */
FdwScanPrivateFetchSize,
+ FdwScanPrivateAvgAttrs,
+ FdwScanPrivateSumAttrs,
+ FdwScanPrivateAggSplit,
+
/*
* String describing join i.e. names of relations being joined and types
* of join, added when the scan is join
@@ -159,6 +165,10 @@ typedef struct PgFdwScanState
MemoryContext temp_cxt; /* context for per-tuple temporary data */
int fetch_size; /* number of tuples per fetch */
+
+ List *avg_attrs;
+ List *sum_attrs;
+ AggSplit aggsplit;
} PgFdwScanState;
/*
@@ -481,7 +491,7 @@ static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinPathExtraData *extra);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
- Node *havingQual);
+ Node *havingQual, RelAggInfo *agg_info);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
RelOptInfo *rel);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
@@ -1190,6 +1200,8 @@ postgresGetForeignPlan(PlannerInfo *root,
bool has_final_sort = false;
bool has_limit = false;
ListCell *lc;
+ List *avg_attrs = NIL;
+ List *sum_attrs = NIL;
/*
* Get FDW private data created by postgresGetForeignUpperPaths(), if any.
@@ -1351,7 +1363,8 @@ postgresGetForeignPlan(PlannerInfo *root,
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_exprs, best_path->path.pathkeys,
has_final_sort, has_limit, false,
- &retrieved_attrs, ¶ms_list);
+ &retrieved_attrs, ¶ms_list, &avg_attrs,
+ &sum_attrs);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
@@ -1360,9 +1373,12 @@ postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
- fdw_private = list_make3(makeString(sql.data),
+ fdw_private = list_make4(makeString(sql.data),
retrieved_attrs,
- makeInteger(fpinfo->fetch_size));
+ makeInteger(fpinfo->fetch_size),
+ avg_attrs);
+ fdw_private = lappend(fdw_private, sum_attrs);
+ fdw_private = lappend(fdw_private, makeInteger(fpinfo->aggsplit));
if (IS_JOIN_REL(foreignrel) || IS_UPPER_REL(foreignrel))
fdw_private = lappend(fdw_private,
makeString(fpinfo->relation_name));
@@ -1448,6 +1464,15 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
fsstate->fetch_size = intVal(list_nth(fsplan->fdw_private,
FdwScanPrivateFetchSize));
+ fsstate->avg_attrs = (List*)(list_nth(fsplan->fdw_private,
+ FdwScanPrivateAvgAttrs));
+
+ fsstate->sum_attrs = (List*)(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSumAttrs));
+
+ fsstate->aggsplit = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateAggSplit));
+
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
@@ -2710,6 +2735,8 @@ estimate_path_cost_size(PlannerInfo *root,
/* Required only to be passed to deparseSelectStmtForRel */
List *retrieved_attrs;
+ List *avg_attrs;
+ List *sum_attrs;
/*
* param_join_conds might contain both clauses that are safe to send
@@ -2743,7 +2770,8 @@ estimate_path_cost_size(PlannerInfo *root,
remote_conds, pathkeys,
fpextra ? fpextra->has_final_sort : false,
fpextra ? fpextra->has_limit : false,
- false, &retrieved_attrs, NULL);
+ false, &retrieved_attrs, NULL, &avg_attrs,
+ &sum_attrs);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
@@ -2920,6 +2948,7 @@ estimate_path_cost_size(PlannerInfo *root,
double input_rows;
int numGroupCols;
double numGroups = 1;
+ List *group_exprs;
/* The upper relation should have its outer relation set */
Assert(outerrel);
@@ -2957,9 +2986,16 @@ estimate_path_cost_size(PlannerInfo *root,
/* Get number of grouping columns and possible number of groups */
numGroupCols = list_length(root->parse->groupClause);
+ numGroups = 100;
+ if (fpinfo->agg_info) {
+ group_exprs = fpinfo->agg_info->group_exprs;
+ }
+ else {
+ group_exprs = get_sortgrouplist_exprs(root->parse->groupClause,
+ fpinfo->grouped_tlist);
+ }
numGroups = estimate_num_groups(root,
- get_sortgrouplist_exprs(root->parse->groupClause,
- fpinfo->grouped_tlist),
+ group_exprs,
input_rows, NULL);
/*
@@ -5568,7 +5604,7 @@ postgresGetForeignJoinPaths(PlannerInfo *root,
*/
static bool
foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
- Node *havingQual)
+ Node *havingQual, RelAggInfo *agg_info)
{
Query *query = root->parse;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
@@ -5615,9 +5651,10 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Expr *expr = (Expr *) lfirst(lc);
Index sgref = get_pathtarget_sortgroupref(grouping_target, i);
ListCell *l;
+ List *groupClause = (agg_info != NULL) ? agg_info->group_clauses : query->groupClause;
/* Check whether this expression is part of GROUP BY clause */
- if (sgref && get_sortgroupref_clause_noerr(sgref, query->groupClause))
+ if (sgref && get_sortgroupref_clause_noerr(sgref, groupClause))
{
TargetEntry *tle;
@@ -5888,6 +5925,7 @@ add_foreign_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
fpinfo->table = ifpinfo->table;
fpinfo->server = ifpinfo->server;
fpinfo->user = ifpinfo->user;
+ fpinfo->agg_info = extra->agg_info;
merge_fdw_options(fpinfo, ifpinfo, NULL);
/*
@@ -5896,7 +5934,8 @@ add_foreign_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
* Use HAVING qual from extra. In case of child partition, it will have
* translated Vars.
*/
- if (!foreign_grouping_ok(root, grouped_rel, extra->havingQual))
+ if (!foreign_grouping_ok(root, grouped_rel, extra->havingQual,
+ extra->agg_info))
return;
/*
@@ -6329,6 +6368,7 @@ make_tuple_from_result_row(PGresult *res,
MemoryContext oldcontext;
ListCell *lc;
int j;
+ PgFdwScanState *festate = (fsstate == NULL) ? NULL : (PgFdwScanState*)fsstate->fdw_state;
Assert(row < PQntuples(res));
@@ -6370,13 +6410,43 @@ make_tuple_from_result_row(PGresult *res,
foreach(lc, retrieved_attrs)
{
int i = lfirst_int(lc);
- char *valstr;
-
- /* fetch next column's textual value */
- if (PQgetisnull(res, row, j))
- valstr = NULL;
- else
- valstr = PQgetvalue(res, row, j);
+ char *valstr = NULL;
+ char *countstr = NULL;
+ char *sumstr = NULL;
+ bool split_serial = false;
+
+ if (festate == NULL) {
+ split_serial = false;
+ } else if (festate->aggsplit == AGGSPLIT_INITIAL_SERIAL) {
+ split_serial = true;
+ } else {
+ split_serial = false;
+ }
+ if (split_serial == true) {
+ if (list_member_int(festate->avg_attrs, i)) {
+ /* fetch next column's textual value */
+ if (PQgetisnull(res, row, j)) {
+ countstr = NULL;
+ sumstr = NULL;
+ } else {
+ countstr = PQgetvalue(res, row, j);
+ j++;
+ sumstr = PQgetvalue(res, row, j);
+ }
+ } else {
+ /* fetch next column's textual value */
+ if (PQgetisnull(res, row, j))
+ valstr = NULL;
+ else
+ valstr = PQgetvalue(res, row, j);
+ }
+ } else {
+ /* fetch next column's textual value */
+ if (PQgetisnull(res, row, j))
+ valstr = NULL;
+ else
+ valstr = PQgetvalue(res, row, j);
+ }
/*
* convert value to internal representation
@@ -6388,12 +6458,63 @@ make_tuple_from_result_row(PGresult *res,
{
/* ordinary column */
Assert(i <= tupdesc->natts);
- nulls[i - 1] = (valstr == NULL);
- /* Apply the input function even to nulls, to support domains */
- values[i - 1] = InputFunctionCall(&attinmeta->attinfuncs[i - 1],
- valstr,
- attinmeta->attioparams[i - 1],
- attinmeta->atttypmods[i - 1]);
+
+ if (split_serial == false) {
+ nulls[i - 1] = (valstr == NULL);
+ /* Apply the input function even to nulls, to support domains */
+ values[i - 1] = InputFunctionCall(&attinmeta->attinfuncs[i - 1],
+ valstr,
+ attinmeta->attioparams[i - 1],
+ attinmeta->atttypmods[i - 1]);
+ } else {
+ if (list_member_int(festate->avg_attrs, i)) {
+ StringInfoData buf;
+ int64 count;
+ Numeric sumN;
+ bytea* sumX;
+ Datum temp;
+ bytea* result;
+
+ nulls[i - 1] = (sumstr == NULL);
+
+ count = DatumGetInt32(DirectFunctionCall1(int8in, PointerGetDatum(countstr)));
+
+ sumN = DatumGetNumeric(DirectFunctionCall3(numeric_in, PointerGetDatum(sumstr), 0,
+ Int32GetDatum(attinmeta->atttypmods[i - 1])));
+ temp = DirectFunctionCall1(numeric_send, NumericGetDatum(sumN));
+ sumX = DatumGetByteaPP(temp);
+ pq_begintypsend(&buf);
+ pq_sendint64(&buf, count);
+ pq_sendbytes(&buf, VARDATA_ANY(sumX), VARSIZE_ANY_EXHDR(sumX));
+ result = pq_endtypsend(&buf);
+ values[i - 1] = PointerGetDatum(result);
+ } else if (list_member_int(festate->sum_attrs, i)) {
+ StringInfoData buf;
+ Numeric sumN;
+ bytea* sumX;
+ Datum temp;
+ bytea* result;
+
+ nulls[i - 1] = (valstr == NULL);
+
+ sumN = DatumGetNumeric(DirectFunctionCall3(numeric_in, PointerGetDatum(valstr), 0,
+ Int32GetDatum(attinmeta->atttypmods[i - 1])));
+ temp = DirectFunctionCall1(numeric_send, NumericGetDatum(sumN));
+ sumX = DatumGetByteaPP(temp);
+ pq_begintypsend(&buf);
+ pq_sendint64(&buf, 1);
+ pq_sendbytes(&buf, VARDATA_ANY(sumX), VARSIZE_ANY_EXHDR(sumX));
+ result = pq_endtypsend(&buf);
+ values[i - 1] = PointerGetDatum(result);
+ } else {
+ nulls[i - 1] = (valstr == NULL);
+ /* Apply the input function even to nulls, to support domains */
+ values[i - 1] = InputFunctionCall(&attinmeta->attinfuncs[i - 1],
+ valstr,
+ attinmeta->attioparams[i - 1],
+ attinmeta->atttypmods[i - 1]);
+ }
+ }
}
else if (i == SelfItemPointerAttributeNumber)
{
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index eef410db39..40c564abef 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -122,6 +122,8 @@ typedef struct PgFdwRelationInfo
* representing the relation.
*/
int relation_index;
+ RelAggInfo *agg_info;
+ AggSplit aggsplit;
} PgFdwRelationInfo;
/* in postgres_fdw.c */
@@ -200,8 +202,9 @@ extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
List *remote_conds, List *pathkeys,
bool has_final_sort, bool has_limit,
- bool is_subquery,
- List **retrieved_attrs, List **params_list);
+ bool is_subquery, List **retrieved_attrs,
+ List **params_list, List **avg_attrs,
+ List **sum_attrs);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index e0b5418de4..0fd00ef2bd 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -49,7 +49,6 @@
#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
-
/* results of subquery_is_pushdown_safe */
typedef struct pushdown_safety_info
{
@@ -540,8 +539,37 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
+ RelOptInfo* rel_grouped;
+ RelAggInfo* agg_info;
+ GroupPathExtraData extra;
+
/* Foreign table */
set_foreign_pathlist(root, rel, rte);
+
+ /* Add paths to the grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids,
+ &agg_info);
+ if (rel_grouped) {
+ extra.target_parallel_safe = false;
+ extra.targetList = root->parse->targetList;
+ extra.havingQual = root->parse->havingQual;
+ extra.partial_costs_set = false;
+ extra.agg_info = agg_info;
+
+ if (enable_partitionwise_aggregate && !root->parse->groupingSets)
+ extra.patype = PARTITIONWISE_AGGREGATE_FULL;
+ else
+ extra.patype = PARTITIONWISE_AGGREGATE_NONE;
+
+ rel_grouped->fdwroutine->GetForeignUpperPaths(root, UPPERREL_GROUP_AGG,
+ rel, rel_grouped, (void*)&extra);
+ if (rel_grouped->pathlist != NIL) {
+ set_cheapest(rel_grouped);
+ }
+ else {
+ del_grouped_rel(root, rel_grouped, agg_info);
+ }
+ }
}
else if (rte->tablesample != NULL)
{
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 6fe12d08ce..0f6b787a91 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -3882,6 +3882,7 @@ create_grouping_paths(PlannerInfo *root,
extra.havingQual = parse->havingQual;
extra.targetList = parse->targetList;
extra.partial_costs_set = false;
+ extra.agg_info = NULL;
/*
* Determine whether partitionwise aggregation is in theory possible.
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 4164f4b0e4..0854d426f9 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -412,9 +412,7 @@ build_simple_grouped_rel(PlannerInfo *root, int relid,
* TODO Consider relaxing some of these restrictions.
*/
rte = root->simple_rte_array[rel_plain->relid];
- if (rte->rtekind != RTE_RELATION ||
- rte->relkind == RELKIND_FOREIGN_TABLE ||
- rte->tablesample != NULL)
+ if (rte->rtekind != RTE_RELATION || rte->tablesample != NULL)
return NULL;
/*
@@ -443,6 +441,10 @@ build_simple_grouped_rel(PlannerInfo *root, int relid,
*/
rel_grouped = makeNode(RelOptInfo);
memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+ if (rte->relkind == RELKIND_FOREIGN_TABLE) {
+ rel_grouped->fdw_private = NULL;
+ rel_grouped->reloptkind = RELOPT_UPPER_REL;
+ }
/*
* Note on consider_startup: while the AGG_HASHED strategy needs the whole
@@ -658,6 +660,39 @@ add_rel_info(RelInfoList *list, void *data)
}
}
+/*
+ * del_rel_info
+ * Delete relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+del_rel_info(RelInfoList* list, void* data)
+{
+ Assert(IsA(data, RelOptInfo) || IsA(data, RelAggInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = list_delete(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ bool found;
+
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo*)data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo*)data)->relids;
+
+ hash_search(list->hash,
+ &relids,
+ HASH_REMOVE,
+ &found);
+ Assert(!found);
+ }
+}
+
+
/*
* add_join_rel
* Add given join relation to the list of join relations in the given
@@ -682,6 +717,19 @@ add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
add_rel_info(root->agg_info_list, agg_info);
}
+/*
+ * del_grouped_rel
+ * Del grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+del_grouped_rel(PlannerInfo* root, RelOptInfo* rel, RelAggInfo* agg_info)
+{
+ del_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG], rel);
+ del_rel_info(root->agg_info_list, agg_info);
+}
+
/*
* find_grouped_rel
* Returns grouped relation entry (base or join relation) corresponding to
diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c
index 1773fa292e..4cf757c676 100644
--- a/src/backend/utils/adt/numeric.c
+++ b/src/backend/utils/adt/numeric.c
@@ -4874,7 +4874,6 @@ int8_avg_serialize(PG_FUNCTION_ARGS)
{
Datum temp;
NumericVar num;
-
init_var(&num);
#ifdef HAVE_INT128
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 98be8b7de2..1b35d94035 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2593,6 +2593,7 @@ typedef struct
Node *havingQual;
List *targetList;
PartitionwiseAggregateType patype;
+ RelAggInfo* agg_info;
} GroupPathExtraData;
/*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 47afa7ed1a..e902f11da0 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -301,6 +301,8 @@ extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
RelAggInfo *agg_info);
+extern void del_grouped_rel(PlannerInfo* root, RelOptInfo* rel,
+ RelAggInfo* agg_info);
extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
[text/plain] readme.txt (1.8K, ../../OS3PR01MB66609239ED8E3F7C3892070895169@OS3PR01MB6660.jpnprd01.prod.outlook.com/3-readme.txt)
download | inline:
1. Restrictions of my prototype
(1) aggregate functions are avg, sum, count, min, max
(2) argment or sum of avg is bigint var
2. My experiment settings
(1) hardware settings
cpu:Intel Corei5-9500 processor(3.0 GHz/9MB cache)(6 cores)
ram:DDR4 3200Mhz 128GB
storage:512GB SSD(M.2 NVMe)
(2) software
os: CentOS7
postgresql source code:14(in development) with commit-id 947456a823d6b0973b68c6b38c8623a0504054e7
# build by "make world"
patchs I applied:
v17-0001-Introduce-RelInfoList-structure.patch
v17-0002-Aggregate-push-down-basic-functionality.patch
v17-0003-Use-also-partial-paths-as-the-input-for-grouped-path.patch
(3) PostgreSQL settings
max_parallel_workers_per_gather=6
work_mem=1000000
(4) tables and data
execute the following commands using postgresql's client tool such as psql
--
\c tmp;
create table log as select 1::bigint as id, 1::bigint as size, 1::bigint as total from generate_series(1,1000000000) t;
create table master as select t as id, 'hoge' || mod(t, 1000) as name from generate_series(1,1000000) t;
create index master_idx on master(id);
create extension postgres_fdw;
create server local_server foreign data wrapper postgres_fdw OPTIONS (host 'localhost', port '5432', dbname 'tmp');
create user mapping for user server local_server options (user 'postgres', password 'postgres');
create foreign table f_log(id bigint, size bigint, total bigint) server local_server options (table_name 'log');
analyze;
set enable_agg_pushdown=true;
explain (verbose, analyze) select b.name, avg(a.total) from f_log a join master b on a.id = b.id group by b.name;
set enable_agg_pushdown=false;
explain (verbose, analyze) select b.name, avg(a.total) from f_log a join master b on a.id = b.id group by b.name;
--
^ permalink raw reply [nested|flat] 59+ messages in thread
* [PATCH v23 4/8] Row pattern recognition patch (planner).
@ 2024-10-25 03:56 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 59+ messages in thread
From: Tatsuo Ishii @ 2024-10-25 03:56 UTC (permalink / raw)
---
src/backend/optimizer/plan/createplan.c | 24 +++++++++++++++-----
src/backend/optimizer/plan/planner.c | 3 +++
src/backend/optimizer/plan/setrefs.c | 27 ++++++++++++++++++++++-
src/backend/optimizer/prep/prepjointree.c | 9 ++++++++
src/include/nodes/plannodes.h | 19 ++++++++++++++++
5 files changed, 76 insertions(+), 6 deletions(-)
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index f2ed0d81f6..1490ded185 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -290,9 +290,11 @@ static WindowAgg *make_windowagg(List *tlist, Index winref,
int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations,
int frameOptions, Node *startOffset, Node *endOffset,
Oid startInRangeFunc, Oid endInRangeFunc,
- Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst,
- List *runCondition, List *qual, bool topWindow,
- Plan *lefttree);
+ Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, List *runCondition,
+ RPSkipTo rpSkipTo, List *patternVariable, List *patternRegexp,
+ List *defineClause,
+ List *defineInitial,
+ List *qual, bool topWindow, Plan *lefttree);
static Group *make_group(List *tlist, List *qual, int numGroupCols,
AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
Plan *lefttree);
@@ -2700,6 +2702,11 @@ create_windowagg_plan(PlannerInfo *root, WindowAggPath *best_path)
wc->inRangeAsc,
wc->inRangeNullsFirst,
best_path->runCondition,
+ wc->rpSkipTo,
+ wc->patternVariable,
+ wc->patternRegexp,
+ wc->defineClause,
+ wc->defineInitial,
best_path->qual,
best_path->topwindow,
subplan);
@@ -6706,8 +6713,10 @@ make_windowagg(List *tlist, Index winref,
int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations,
int frameOptions, Node *startOffset, Node *endOffset,
Oid startInRangeFunc, Oid endInRangeFunc,
- Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst,
- List *runCondition, List *qual, bool topWindow, Plan *lefttree)
+ Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, List *runCondition,
+ RPSkipTo rpSkipTo, List *patternVariable, List *patternRegexp, List *defineClause,
+ List *defineInitial,
+ List *qual, bool topWindow, Plan *lefttree)
{
WindowAgg *node = makeNode(WindowAgg);
Plan *plan = &node->plan;
@@ -6733,6 +6742,11 @@ make_windowagg(List *tlist, Index winref,
node->inRangeAsc = inRangeAsc;
node->inRangeNullsFirst = inRangeNullsFirst;
node->topWindow = topWindow;
+ node->rpSkipTo = rpSkipTo,
+ node->patternVariable = patternVariable;
+ node->patternRegexp = patternRegexp;
+ node->defineClause = defineClause;
+ node->defineInitial = defineInitial;
plan->targetlist = tlist;
plan->lefttree = lefttree;
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 0f423e9684..24f0744c7c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -879,6 +879,9 @@ subquery_planner(PlannerGlobal *glob, Query *parse, PlannerInfo *parent_root,
EXPRKIND_LIMIT);
wc->endOffset = preprocess_expression(root, wc->endOffset,
EXPRKIND_LIMIT);
+ wc->defineClause = (List *) preprocess_expression(root,
+ (Node *) wc->defineClause,
+ EXPRKIND_TARGET);
}
parse->limitOffset = preprocess_expression(root, parse->limitOffset,
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 91c7c4fe2f..b5310e0d72 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -211,7 +211,6 @@ static List *set_windowagg_runcondition_references(PlannerInfo *root,
List *runcondition,
Plan *plan);
-
/*****************************************************************************
*
* SUBPLAN REFERENCES
@@ -2495,6 +2494,32 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
NRM_EQUAL,
NUM_EXEC_QUAL(plan));
+ /*
+ * Modifies an expression tree in each DEFINE clause so that all Var
+ * nodes's varno refers to OUTER_VAR.
+ */
+ if (IsA(plan, WindowAgg))
+ {
+ WindowAgg *wplan = (WindowAgg *) plan;
+
+ if (wplan->defineClause != NIL)
+ {
+ foreach(l, wplan->defineClause)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(l);
+
+ tle->expr = (Expr *)
+ fix_upper_expr(root,
+ (Node *) tle->expr,
+ subplan_itlist,
+ OUTER_VAR,
+ rtoffset,
+ NRM_EQUAL,
+ NUM_EXEC_QUAL(plan));
+ }
+ }
+ }
+
pfree(subplan_itlist);
}
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 4d7f972caf..1dcfcf338d 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -2271,6 +2271,15 @@ perform_pullup_replace_vars(PlannerInfo *root,
parse->returningList = (List *)
pullup_replace_vars((Node *) parse->returningList, rvcontext);
+ foreach(lc, parse->windowClause)
+ {
+ WindowClause *wc = lfirst_node(WindowClause, lc);
+
+ if (wc->defineClause != NIL)
+ wc->defineClause = (List *)
+ pullup_replace_vars((Node *) wc->defineClause, rvcontext);
+ }
+
if (parse->onConflict)
{
parse->onConflict->onConflictSet = (List *)
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 52f29bcdb6..294597461b 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -20,6 +20,7 @@
#include "lib/stringinfo.h"
#include "nodes/bitmapset.h"
#include "nodes/lockoptions.h"
+#include "nodes/parsenodes.h"
#include "nodes/primnodes.h"
@@ -1099,6 +1100,24 @@ typedef struct WindowAgg
/* nulls sort first for in_range tests? */
bool inRangeNullsFirst;
+ /* Row Pattern Recognition AFTER MACH SKIP clause */
+ RPSkipTo rpSkipTo; /* Row Pattern Skip To type */
+
+ /* Row Pattern PATTERN variable name (list of String) */
+ List *patternVariable;
+
+ /*
+ * Row Pattern RPATTERN regular expression quantifier ('+' or ''. list of
+ * String)
+ */
+ List *patternRegexp;
+
+ /* Row Pattern DEFINE clause (list of TargetEntry) */
+ List *defineClause;
+
+ /* Row Pattern DEFINE variable initial names (list of String) */
+ List *defineInitial;
+
/*
* false for all apart from the WindowAgg that's closest to the root of
* the plan
--
2.25.1
----Next_Part(Fri_Oct_25_13_04_53_2024_648)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v23-0005-Row-pattern-recognition-patch-executor.patch"
^ permalink raw reply [nested|flat] 59+ messages in thread
end of thread, other threads:[~2024-10-25 03:56 UTC | newest]
Thread overview: 59+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-04-04 08:41 WIP: Aggregation push-down Antonin Houska <[email protected]>
2017-04-28 08:46 ` Antonin Houska <[email protected]>
2017-08-17 15:22 ` Antonin Houska <[email protected]>
2017-09-07 09:14 ` Jeevan Chalke <[email protected]>
2017-09-08 12:36 ` Antonin Houska <[email protected]>
2017-09-07 11:15 ` Ashutosh Bapat <[email protected]>
2017-09-08 13:34 ` Antonin Houska <[email protected]>
2017-09-11 08:30 ` Ashutosh Bapat <[email protected]>
2017-09-07 14:21 ` Merlin Moncure <[email protected]>
2017-11-03 15:33 ` Antonin Houska <[email protected]>
2017-11-30 04:15 ` Michael Paquier <[email protected]>
2017-12-22 15:43 ` Antonin Houska <[email protected]>
2018-01-15 21:15 ` Robert Haas <[email protected]>
2018-01-26 13:04 ` Antonin Houska <[email protected]>
2018-01-26 13:58 ` Robert Haas <[email protected]>
2018-01-29 08:32 ` Antonin Houska <[email protected]>
2018-01-29 14:01 ` Chapman Flack <[email protected]>
2018-01-29 14:43 ` Robert Haas <[email protected]>
2018-02-23 16:08 ` Antonin Houska <[email protected]>
2018-02-24 14:08 ` Robert Haas <[email protected]>
2018-07-06 11:11 ` Antonin Houska <[email protected]>
2018-02-28 16:02 ` Antonin Houska <[email protected]>
2018-08-31 15:05 ` Antonin Houska <[email protected]>
2018-11-18 21:43 ` Tom Lane <[email protected]>
2018-12-17 15:58 ` Antonin Houska <[email protected]>
2019-01-15 14:06 ` Antonin Houska <[email protected]>
2019-02-04 06:03 ` Michael Paquier <[email protected]>
2019-02-05 09:40 ` Antonin Houska <[email protected]>
2019-02-21 20:18 ` Tom Lane <[email protected]>
2019-02-28 16:02 ` Antonin Houska <[email protected]>
2019-07-03 06:26 ` Richard Guo <[email protected]>
2019-07-09 13:47 ` Antonin Houska <[email protected]>
2019-07-11 10:27 ` Richard Guo <[email protected]>
2019-07-12 08:41 ` Antonin Houska <[email protected]>
2019-07-15 10:28 ` Richard Guo <[email protected]>
2019-07-17 14:39 ` Antonin Houska <[email protected]>
2019-09-04 20:23 ` Alvaro Herrera <[email protected]>
2019-09-05 06:10 ` Antonin Houska <[email protected]>
2020-01-11 01:11 ` Tomas Vondra <[email protected]>
2020-01-16 15:25 ` Antonin Houska <[email protected]>
2020-01-31 21:59 ` legrand legrand <[email protected]>
2020-02-03 07:46 ` Antonin Houska <[email protected]>
2020-02-06 08:30 ` Richard Guo <[email protected]>
2020-02-07 09:05 ` Antonin Houska <[email protected]>
2020-02-26 21:10 ` legrand legrand <[email protected]>
2020-02-27 08:51 ` Antonin Houska <[email protected]>
2020-04-21 08:37 ` Andy Fan <[email protected]>
2020-04-22 03:39 ` Andy Fan <[email protected]>
2020-04-24 13:01 ` Antonin Houska <[email protected]>
2020-04-24 12:11 ` Antonin Houska <[email protected]>
2020-04-26 08:12 ` Andy Fan <[email protected]>
2020-05-19 08:17 ` Antonin Houska <[email protected]>
2020-07-02 12:39 ` Daniel Gustafsson <[email protected]>
2020-07-03 10:07 ` Laurenz Albe <[email protected]>
2021-03-03 16:42 ` David Steele <[email protected]>
2021-03-11 10:10 ` Antonin Houska <[email protected]>
2021-03-11 13:34 ` David Steele <[email protected]>
2022-03-21 21:09 ` [email protected] <[email protected]>
2024-10-25 03:56 [PATCH v23 4/8] Row pattern recognition patch (planner). Tatsuo Ishii <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox