Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fkpZK-0003PR-TZ for pgsql-hackers@arkaria.postgresql.org; Wed, 01 Aug 2018 11:45:31 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fkpZI-00076a-GF for pgsql-hackers@arkaria.postgresql.org; Wed, 01 Aug 2018 11:45:28 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fkpZH-000766-SC for pgsql-hackers@lists.postgresql.org; Wed, 01 Aug 2018 11:45:28 +0000 Received: from tama500.ecl.ntt.co.jp ([129.60.39.148]) by makus.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fkpZ4-0003TC-WA for pgsql-hackers@postgresql.org; Wed, 01 Aug 2018 11:45:26 +0000 Received: from vc2.ecl.ntt.co.jp (vc2.ecl.ntt.co.jp [129.60.86.154]) by tama500.ecl.ntt.co.jp (8.13.8/8.13.8) with ESMTP id w71Bj96w017750; Wed, 1 Aug 2018 20:45:09 +0900 Received: from vc2.ecl.ntt.co.jp (localhost [127.0.0.1]) by vc2.ecl.ntt.co.jp (Postfix) with ESMTP id 570F96394A6; Wed, 1 Aug 2018 20:45:09 +0900 (JST) Received: from jcms-pop21.ecl.ntt.co.jp (jcms-pop21.ecl.ntt.co.jp [129.60.87.134]) by vc2.ecl.ntt.co.jp (Postfix) with ESMTP id 4A2EA6391E7; Wed, 1 Aug 2018 20:45:09 +0900 (JST) Received: from [IPv6:::1] (unknown [129.60.241.75]) by jcms-pop21.ecl.ntt.co.jp (Postfix) with ESMTPSA id 3D960400232; Wed, 1 Aug 2018 20:45:09 +0900 (JST) Message-ID: <5B619D23.4010000@lab.ntt.co.jp> Date: Wed, 01 Aug 2018 20:44:35 +0900 From: Etsuro Fujita User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20120327 Thunderbird/11.0.1 MIME-Version: 1.0 Subject: Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled. References: <5AFC0865.8050802@lab.ntt.co.jp> <5AFD6580.5090308@lab.ntt.co.jp> <5AFE81CC.8000508@lab.ntt.co.jp> <5B17C5EC.1090602@lab.ntt.co.jp> <5B1E5071.1000704@lab.ntt.co.jp> <5B23A964.9020800@lab.ntt.co.jp> <5B28FB19.3090809@lab.ntt.co.jp> <20180703163519.tsigyic5xefunv74@alap3.anarazel.de> <5B3C2A4E.5030901@lab.ntt.co.jp> <5B4F2BE9.3000304@lab.ntt.co.jp> <5B55886F.5090906@lab.ntt.co.jp> <5B5712C8.9030404@lab.ntt.co.jp> <5B59BA55.30200@lab.ntt.co.jp> In-Reply-To: <5B59BA55.30200@lab.ntt.co.jp> Content-Type: multipart/mixed; boundary="------------040405050600000402020703" X-CC-Mail-RelayStamp: 1 To: Robert Haas Cc: Andres Freund , Ashutosh Bapat , Rajkumar Raghuwanshi , pgsql-hackers , Tom Lane X-TM-AS-MML: disable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------040405050600000402020703 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit (2018/07/26 21:11), Etsuro Fujita wrote: > (2018/07/26 5:27), Robert Haas wrote: >> Well, I could have the wrong idea here, but I tend to think allowing >> for ConvertRowTypeExpr elsewhere won't be that bad. > > I still don't like that because in my opinion, changes needed for that > would not be localized, and that would make code complicated more than > necessary. > > As I mentioned in a previous email, another idea to avoid that would be > to adjust tlists for children at path creation time, not plan creation > time; we could adjust the tlist for each of subpaths accumulated for an > Append/MergeAppend path in add_paths_to_append_rel called from > set_append_rel_pathlist or generate_partitionwise_join_paths, with > create_projection_path adding ConvertRowTypeExpr. It seems unlikely that > performing create_projection_path to such a subpath would change its > property of being the cheapest, so it would be safe to adjust the tlists > that way. This would not require making create_plan complicated anymore. > I might be missing something, though. I updated the patch that way. Updated patch attached. I fixed a bug and did a bit of cleanups as well. Best regards, Etsuro Fujita --------------040405050600000402020703 Content-Type: text/x-diff; name="redesign-pwj-child-tlist-handling-3.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="redesign-pwj-child-tlist-handling-3.patch" *** a/contrib/postgres_fdw/expected/postgres_fdw.out --- b/contrib/postgres_fdw/expected/postgres_fdw.out *************** *** 8337,8344 **** ALTER TABLE fprt2_p1 SET (autovacuum_enabled = 'false'); ALTER TABLE fprt2_p2 SET (autovacuum_enabled = 'false'); INSERT INTO fprt2_p1 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(0, 249, 3) i; INSERT INTO fprt2_p2 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(250, 499, 3) i; ! CREATE FOREIGN TABLE ftprt2_p1 PARTITION OF fprt2 FOR VALUES FROM (0) TO (250) SERVER loopback OPTIONS (table_name 'fprt2_p1', use_remote_estimate 'true'); CREATE FOREIGN TABLE ftprt2_p2 PARTITION OF fprt2 FOR VALUES FROM (250) TO (500) SERVER loopback OPTIONS (table_name 'fprt2_p2', use_remote_estimate 'true'); ANALYZE fprt2; --- 8337,8345 ---- ALTER TABLE fprt2_p2 SET (autovacuum_enabled = 'false'); INSERT INTO fprt2_p1 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(0, 249, 3) i; INSERT INTO fprt2_p2 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(250, 499, 3) i; ! CREATE FOREIGN TABLE ftprt2_p1 (b int, c varchar, a int) SERVER loopback OPTIONS (table_name 'fprt2_p1', use_remote_estimate 'true'); + ALTER TABLE fprt2 ATTACH PARTITION ftprt2_p1 FOR VALUES FROM (0) TO (250); CREATE FOREIGN TABLE ftprt2_p2 PARTITION OF fprt2 FOR VALUES FROM (250) TO (500) SERVER loopback OPTIONS (table_name 'fprt2_p2', use_remote_estimate 'true'); ANALYZE fprt2; *************** *** 8391,8416 **** SELECT t1.a,t2.b,t2.c FROM fprt1 t1 LEFT JOIN (SELECT * FROM fprt2 WHERE a < 10) -- with whole-row reference EXPLAIN (COSTS OFF) ! SELECT t1,t2 FROM fprt1 t1 JOIN fprt2 t2 ON (t1.a = t2.b and t1.b = t2.a) WHERE t1.a % 25 =0 ORDER BY 1,2; ! QUERY PLAN ! --------------------------------------------------------------------------------- Sort Sort Key: ((t1.*)::fprt1), ((t2.*)::fprt2) -> Append -> Foreign Scan ! Relations: (public.ftprt1_p1 t1) INNER JOIN (public.ftprt2_p1 t2) -> Foreign Scan ! Relations: (public.ftprt1_p2 t1) INNER JOIN (public.ftprt2_p2 t2) (7 rows) ! SELECT t1,t2 FROM fprt1 t1 JOIN fprt2 t2 ON (t1.a = t2.b and t1.b = t2.a) WHERE t1.a % 25 =0 ORDER BY 1,2; ! t1 | t2 ----------------+---------------- (0,0,0000) | (0,0,0000) (150,150,0003) | (150,150,0003) (250,250,0005) | (250,250,0005) (400,400,0008) | (400,400,0008) ! (4 rows) -- join with lateral reference EXPLAIN (COSTS OFF) --- 8392,8427 ---- -- with whole-row reference EXPLAIN (COSTS OFF) ! SELECT t1.wr, t2.wr FROM (SELECT t1 wr, a FROM fprt1 t1 WHERE t1.a % 25 = 0) t1 FULL JOIN (SELECT t2 wr, b FROM fprt2 t2 WHERE t2.b % 25 = 0) t2 ON (t1.a = t2.b) ORDER BY 1,2; ! QUERY PLAN ! -------------------------------------------------------------------------------- Sort Sort Key: ((t1.*)::fprt1), ((t2.*)::fprt2) -> Append -> Foreign Scan ! Relations: (public.ftprt1_p1 t1) FULL JOIN (public.ftprt2_p1 t2) -> Foreign Scan ! Relations: (public.ftprt1_p2 t1) FULL JOIN (public.ftprt2_p2 t2) (7 rows) ! SELECT t1.wr, t2.wr FROM (SELECT t1 wr, a FROM fprt1 t1 WHERE t1.a % 25 = 0) t1 FULL JOIN (SELECT t2 wr, b FROM fprt2 t2 WHERE t2.b % 25 = 0) t2 ON (t1.a = t2.b) ORDER BY 1,2; ! wr | wr ----------------+---------------- (0,0,0000) | (0,0,0000) + (50,50,0001) | + (100,100,0002) | (150,150,0003) | (150,150,0003) + (200,200,0004) | (250,250,0005) | (250,250,0005) + (300,300,0006) | + (350,350,0007) | (400,400,0008) | (400,400,0008) ! (450,450,0009) | ! | (75,75,0001) ! | (225,225,0004) ! | (325,325,0006) ! | (475,475,0009) ! (14 rows) -- join with lateral reference EXPLAIN (COSTS OFF) *************** *** 8474,8479 **** SELECT t1.a, t1.phv, t2.b, t2.phv FROM (SELECT 't1_phv' phv, * FROM fprt1 WHERE --- 8485,8520 ---- | | 475 | t2_phv (14 rows) + -- test FOR UPDATE + EXPLAIN (COSTS OFF) + SELECT t1.a, t2.b FROM fprt1 t1 INNER JOIN fprt2 t2 ON (t1.a = t2.b) WHERE t1.a % 25 = 0 ORDER BY 1,2 FOR UPDATE OF t1; + QUERY PLAN + --------------------------------------------------------------------------------------- + LockRows + -> Sort + Sort Key: t1.a + -> Append + -> Foreign Scan + Relations: (public.ftprt1_p1 t1) INNER JOIN (public.ftprt2_p1 t2) + -> Nested Loop + -> Foreign Scan on ftprt1_p1 t1 + -> Foreign Scan on ftprt2_p1 t2 + -> Foreign Scan + Relations: (public.ftprt1_p2 t1) INNER JOIN (public.ftprt2_p2 t2) + -> Nested Loop + -> Foreign Scan on ftprt1_p2 t1_1 + -> Foreign Scan on ftprt2_p2 t2_1 + (14 rows) + + SELECT t1.a, t2.b FROM fprt1 t1 INNER JOIN fprt2 t2 ON (t1.a = t2.b) WHERE t1.a % 25 = 0 ORDER BY 1,2 FOR UPDATE OF t1; + a | b + -----+----- + 0 | 0 + 150 | 150 + 250 | 250 + 400 | 400 + (4 rows) + RESET enable_partitionwise_join; -- =================================================================== -- test partitionwise aggregates *** a/contrib/postgres_fdw/sql/postgres_fdw.sql --- b/contrib/postgres_fdw/sql/postgres_fdw.sql *************** *** 2263,2270 **** ALTER TABLE fprt2_p1 SET (autovacuum_enabled = 'false'); ALTER TABLE fprt2_p2 SET (autovacuum_enabled = 'false'); INSERT INTO fprt2_p1 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(0, 249, 3) i; INSERT INTO fprt2_p2 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(250, 499, 3) i; ! CREATE FOREIGN TABLE ftprt2_p1 PARTITION OF fprt2 FOR VALUES FROM (0) TO (250) SERVER loopback OPTIONS (table_name 'fprt2_p1', use_remote_estimate 'true'); CREATE FOREIGN TABLE ftprt2_p2 PARTITION OF fprt2 FOR VALUES FROM (250) TO (500) SERVER loopback OPTIONS (table_name 'fprt2_p2', use_remote_estimate 'true'); ANALYZE fprt2; --- 2263,2271 ---- ALTER TABLE fprt2_p2 SET (autovacuum_enabled = 'false'); INSERT INTO fprt2_p1 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(0, 249, 3) i; INSERT INTO fprt2_p2 SELECT i, i, to_char(i/50, 'FM0000') FROM generate_series(250, 499, 3) i; ! CREATE FOREIGN TABLE ftprt2_p1 (b int, c varchar, a int) SERVER loopback OPTIONS (table_name 'fprt2_p1', use_remote_estimate 'true'); + ALTER TABLE fprt2 ATTACH PARTITION ftprt2_p1 FOR VALUES FROM (0) TO (250); CREATE FOREIGN TABLE ftprt2_p2 PARTITION OF fprt2 FOR VALUES FROM (250) TO (500) SERVER loopback OPTIONS (table_name 'fprt2_p2', use_remote_estimate 'true'); ANALYZE fprt2; *************** *** 2283,2290 **** SELECT t1.a,t2.b,t2.c FROM fprt1 t1 LEFT JOIN (SELECT * FROM fprt2 WHERE a < 10) -- with whole-row reference EXPLAIN (COSTS OFF) ! SELECT t1,t2 FROM fprt1 t1 JOIN fprt2 t2 ON (t1.a = t2.b and t1.b = t2.a) WHERE t1.a % 25 =0 ORDER BY 1,2; ! SELECT t1,t2 FROM fprt1 t1 JOIN fprt2 t2 ON (t1.a = t2.b and t1.b = t2.a) WHERE t1.a % 25 =0 ORDER BY 1,2; -- join with lateral reference EXPLAIN (COSTS OFF) --- 2284,2291 ---- -- with whole-row reference EXPLAIN (COSTS OFF) ! SELECT t1.wr, t2.wr FROM (SELECT t1 wr, a FROM fprt1 t1 WHERE t1.a % 25 = 0) t1 FULL JOIN (SELECT t2 wr, b FROM fprt2 t2 WHERE t2.b % 25 = 0) t2 ON (t1.a = t2.b) ORDER BY 1,2; ! SELECT t1.wr, t2.wr FROM (SELECT t1 wr, a FROM fprt1 t1 WHERE t1.a % 25 = 0) t1 FULL JOIN (SELECT t2 wr, b FROM fprt2 t2 WHERE t2.b % 25 = 0) t2 ON (t1.a = t2.b) ORDER BY 1,2; -- join with lateral reference EXPLAIN (COSTS OFF) *************** *** 2296,2301 **** EXPLAIN (COSTS OFF) --- 2297,2307 ---- SELECT t1.a, t1.phv, t2.b, t2.phv FROM (SELECT 't1_phv' phv, * FROM fprt1 WHERE a % 25 = 0) t1 FULL JOIN (SELECT 't2_phv' phv, * FROM fprt2 WHERE b % 25 = 0) t2 ON (t1.a = t2.b) ORDER BY t1.a, t2.b; SELECT t1.a, t1.phv, t2.b, t2.phv FROM (SELECT 't1_phv' phv, * FROM fprt1 WHERE a % 25 = 0) t1 FULL JOIN (SELECT 't2_phv' phv, * FROM fprt2 WHERE b % 25 = 0) t2 ON (t1.a = t2.b) ORDER BY t1.a, t2.b; + -- test FOR UPDATE + EXPLAIN (COSTS OFF) + SELECT t1.a, t2.b FROM fprt1 t1 INNER JOIN fprt2 t2 ON (t1.a = t2.b) WHERE t1.a % 25 = 0 ORDER BY 1,2 FOR UPDATE OF t1; + SELECT t1.a, t2.b FROM fprt1 t1 INNER JOIN fprt2 t2 ON (t1.a = t2.b) WHERE t1.a % 25 = 0 ORDER BY 1,2 FOR UPDATE OF t1; + RESET enable_partitionwise_join; *** a/src/backend/nodes/outfuncs.c --- b/src/backend/nodes/outfuncs.c *************** *** 2359,2365 **** _outRelOptInfo(StringInfo str, const RelOptInfo *node) --- 2359,2368 ---- WRITE_UINT_FIELD(baserestrict_min_security); WRITE_NODE_FIELD(joininfo); WRITE_BOOL_FIELD(has_eclass_joins); + WRITE_BOOL_FIELD(consider_partitionwise_join); + WRITE_OID_FIELD(top_parent_rowtype); WRITE_BITMAPSET_FIELD(top_parent_relids); + WRITE_BOOL_FIELD(has_target_wrvs); WRITE_NODE_FIELD(partitioned_child_rels); } *** a/src/backend/optimizer/path/allpaths.c --- b/src/backend/optimizer/path/allpaths.c *************** *** 910,915 **** set_append_rel_size(PlannerInfo *root, RelOptInfo *rel, --- 910,928 ---- } /* + * If this is a baserel, set the consider_partitionwise_join flag and + * check whether the baserel's targetlist contain a whole-row Var. + */ + if (rel->reloptkind == RELOPT_BASEREL) + { + if (enable_partitionwise_join && rel->part_scheme && + !bms_equal(rel->relids, root->all_baserels)) + rel->consider_partitionwise_join = true; + if (rel->attr_needed[InvalidAttrNumber - rel->min_attr] != NULL) + rel->has_target_wrvs = true; + } + + /* * Initialize to compute size estimates for whole append relation. * * We handle width estimates by weighting the widths of different child *************** *** 956,1025 **** set_append_rel_size(PlannerInfo *root, RelOptInfo *rel, childrel = find_base_rel(root, childRTindex); Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL); - if (rel->part_scheme) - { - AttrNumber attno; - - /* - * We need attr_needed data for building targetlist of a join - * relation representing join between matching partitions for - * partitionwise join. A given attribute of a child will be needed - * in the same highest joinrel where the corresponding attribute - * of parent is needed. Hence it suffices to use the same Relids - * set for parent and child. - */ - for (attno = rel->min_attr; attno <= rel->max_attr; attno++) - { - int index = attno - rel->min_attr; - Relids attr_needed = rel->attr_needed[index]; - - /* System attributes do not need translation. */ - if (attno <= 0) - { - Assert(rel->min_attr == childrel->min_attr); - childrel->attr_needed[index] = attr_needed; - } - else - { - Var *var = list_nth_node(Var, - appinfo->translated_vars, - attno - 1); - int child_index; - - /* - * Ignore any column dropped from the parent. - * Corresponding Var won't have any translation. It won't - * have attr_needed information, since it can not be - * referenced in the query. - */ - if (var == NULL) - { - Assert(attr_needed == NULL); - continue; - } - - child_index = var->varattno - childrel->min_attr; - childrel->attr_needed[child_index] = attr_needed; - } - } - } - /* ! * Copy/Modify targetlist. Even if this child is deemed empty, we need ! * its targetlist in case it falls on nullable side in a child-join ! * because of partitionwise join. * ! * NB: the resulting childrel->reltarget->exprs may contain arbitrary ! * expressions, which otherwise would not occur in a rel's targetlist. ! * Code that might be looking at an appendrel child must cope with ! * such. (Normally, a rel's targetlist would only include Vars and ! * PlaceHolderVars.) XXX we do not bother to update the cost or width ! * fields of childrel->reltarget; not clear if that would be useful. */ ! childrel->reltarget->exprs = (List *) ! adjust_appendrel_attrs(root, ! (Node *) rel->reltarget->exprs, ! 1, &appinfo); /* * We have to make child entries in the EquivalenceClass data --- 969,983 ---- childrel = find_base_rel(root, childRTindex); Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL); /* ! * We have to copy the parent's targetlist to the child, with ! * appropriate substitution of variables. * ! * Note: when considering partitionwise joining, we need the child's ! * targetlist in case it falls on nullable side in a child join even ! * if the child is deemed empty. */ ! build_childrel_tlist(root, rel, childrel, 1, &appinfo); /* * We have to make child entries in the EquivalenceClass data *************** *** 1181,1186 **** set_append_rel_size(PlannerInfo *root, RelOptInfo *rel, --- 1139,1152 ---- 1, &appinfo); /* + * Note: we could compute appropriate attr_needed data for the child's + * variables, by transforming the parent's attr_needed through the + * translated_vars mapping. However, currently there's no need + * because attr_needed is only examined for base relations not + * otherrels. So we just leave the child's attr_needed empty. + */ + + /* * If parallelism is allowable for this query in general, see whether * it's allowable for this childrel in particular. But if we've * already decided the appendrel is not parallel-safe as a whole, *** a/src/backend/optimizer/plan/setrefs.c --- b/src/backend/optimizer/plan/setrefs.c *************** *** 41,49 **** typedef struct int num_vars; /* number of plain Var tlist entries */ bool has_ph_vars; /* are there PlaceHolderVar entries? */ bool has_non_vars; /* are there other entries? */ - bool has_conv_whole_rows; /* are there ConvertRowtypeExpr - * entries encapsulating a whole-row - * Var? */ tlist_vinfo vars[FLEXIBLE_ARRAY_MEMBER]; /* has num_vars entries */ } indexed_tlist; --- 41,46 ---- *************** *** 143,149 **** static List *set_returning_clause_references(PlannerInfo *root, int rtoffset); static bool extract_query_dependencies_walker(Node *node, PlannerInfo *context); - static bool is_converted_whole_row_reference(Node *node); /***************************************************************************** * --- 140,145 ---- *************** *** 1997,2003 **** build_tlist_index(List *tlist) itlist->tlist = tlist; itlist->has_ph_vars = false; itlist->has_non_vars = false; - itlist->has_conv_whole_rows = false; /* Find the Vars and fill in the index array */ vinfo = itlist->vars; --- 1993,1998 ---- *************** *** 2016,2023 **** build_tlist_index(List *tlist) } else if (tle->expr && IsA(tle->expr, PlaceHolderVar)) itlist->has_ph_vars = true; - else if (is_converted_whole_row_reference((Node *) tle->expr)) - itlist->has_conv_whole_rows = true; else itlist->has_non_vars = true; } --- 2011,2016 ---- *************** *** 2033,2042 **** build_tlist_index(List *tlist) * This is like build_tlist_index, but we only index tlist entries that * are Vars belonging to some rel other than the one specified. We will set * has_ph_vars (allowing PlaceHolderVars to be matched), but not has_non_vars ! * (so nothing other than Vars and PlaceHolderVars can be matched). In case of ! * DML, where this function will be used, returning lists from child relations ! * will be appended similar to a simple append relation. That does not require ! * fixing ConvertRowtypeExpr references. So, those are not considered here. */ static indexed_tlist * build_tlist_index_other_vars(List *tlist, Index ignore_rel) --- 2026,2032 ---- * This is like build_tlist_index, but we only index tlist entries that * are Vars belonging to some rel other than the one specified. We will set * has_ph_vars (allowing PlaceHolderVars to be matched), but not has_non_vars ! * (so nothing other than Vars and PlaceHolderVars can be matched). */ static indexed_tlist * build_tlist_index_other_vars(List *tlist, Index ignore_rel) *************** *** 2053,2059 **** build_tlist_index_other_vars(List *tlist, Index ignore_rel) itlist->tlist = tlist; itlist->has_ph_vars = false; itlist->has_non_vars = false; - itlist->has_conv_whole_rows = false; /* Find the desired Vars and fill in the index array */ vinfo = itlist->vars; --- 2043,2048 ---- *************** *** 2256,2262 **** static Node * fix_join_expr_mutator(Node *node, fix_join_expr_context *context) { Var *newvar; - bool converted_whole_row; if (node == NULL) return NULL; --- 2245,2250 ---- *************** *** 2326,2337 **** fix_join_expr_mutator(Node *node, fix_join_expr_context *context) } if (IsA(node, Param)) return fix_param_node(context->root, (Param *) node); - /* Try matching more complex expressions too, if tlists have any */ ! converted_whole_row = is_converted_whole_row_reference(node); ! if (context->outer_itlist && ! (context->outer_itlist->has_non_vars || ! (context->outer_itlist->has_conv_whole_rows && converted_whole_row))) { newvar = search_indexed_tlist_for_non_var((Expr *) node, context->outer_itlist, --- 2314,2321 ---- } if (IsA(node, Param)) return fix_param_node(context->root, (Param *) node); /* Try matching more complex expressions too, if tlists have any */ ! if (context->outer_itlist && context->outer_itlist->has_non_vars) { newvar = search_indexed_tlist_for_non_var((Expr *) node, context->outer_itlist, *************** *** 2339,2347 **** fix_join_expr_mutator(Node *node, fix_join_expr_context *context) if (newvar) return (Node *) newvar; } ! if (context->inner_itlist && ! (context->inner_itlist->has_non_vars || ! (context->inner_itlist->has_conv_whole_rows && converted_whole_row))) { newvar = search_indexed_tlist_for_non_var((Expr *) node, context->inner_itlist, --- 2323,2329 ---- if (newvar) return (Node *) newvar; } ! if (context->inner_itlist && context->inner_itlist->has_non_vars) { newvar = search_indexed_tlist_for_non_var((Expr *) node, context->inner_itlist, *************** *** 2461,2469 **** fix_upper_expr_mutator(Node *node, fix_upper_expr_context *context) /* If no match, just fall through to process it normally */ } /* Try matching more complex expressions too, if tlist has any */ ! if (context->subplan_itlist->has_non_vars || ! (context->subplan_itlist->has_conv_whole_rows && ! is_converted_whole_row_reference(node))) { newvar = search_indexed_tlist_for_non_var((Expr *) node, context->subplan_itlist, --- 2443,2449 ---- /* If no match, just fall through to process it normally */ } /* Try matching more complex expressions too, if tlist has any */ ! if (context->subplan_itlist->has_non_vars) { newvar = search_indexed_tlist_for_non_var((Expr *) node, context->subplan_itlist, *************** *** 2670,2702 **** extract_query_dependencies_walker(Node *node, PlannerInfo *context) return expression_tree_walker(node, extract_query_dependencies_walker, (void *) context); } - - /* - * is_converted_whole_row_reference - * If the given node is a ConvertRowtypeExpr encapsulating a whole-row - * reference as implicit cast, return true. Otherwise return false. - */ - static bool - is_converted_whole_row_reference(Node *node) - { - ConvertRowtypeExpr *convexpr; - - if (!node || !IsA(node, ConvertRowtypeExpr)) - return false; - - /* Traverse nested ConvertRowtypeExpr's. */ - convexpr = castNode(ConvertRowtypeExpr, node); - while (convexpr->convertformat == COERCE_IMPLICIT_CAST && - IsA(convexpr->arg, ConvertRowtypeExpr)) - convexpr = castNode(ConvertRowtypeExpr, convexpr->arg); - - if (IsA(convexpr->arg, Var)) - { - Var *var = castNode(Var, convexpr->arg); - - if (var->varattno == 0) - return true; - } - - return false; - } --- 2650,2652 ---- *** a/src/backend/optimizer/util/pathnode.c --- b/src/backend/optimizer/util/pathnode.c *************** *** 53,58 **** typedef enum --- 53,62 ---- static List *translate_sub_tlist(List *tlist, int relid); static int append_total_cost_compare(const void *a, const void *b); static int append_startup_cost_compare(const void *a, const void *b); + static List *add_projection_to_subpaths(PlannerInfo *root, RelOptInfo *rel, + List *subpaths); + static Path *add_projection_to_subpath(PlannerInfo *root, RelOptInfo *rel, + Path *subpath); static List *reparameterize_pathlist_by_child(PlannerInfo *root, List *pathlist, RelOptInfo *child_rel); *************** *** 1274,1279 **** create_append_path(PlannerInfo *root, --- 1278,1288 ---- pathnode->first_partial_path = list_length(subpaths); pathnode->subpaths = list_concat(subpaths, partial_subpaths); + /* Add a projection step to each subpath if needed. */ + if (rel->consider_partitionwise_join && rel->has_target_wrvs) + pathnode->subpaths = add_projection_to_subpaths(root, rel, + pathnode->subpaths); + foreach(l, pathnode->subpaths) { Path *subpath = (Path *) lfirst(l); *************** *** 1366,1371 **** create_merge_append_path(PlannerInfo *root, --- 1375,1384 ---- pathnode->path.parallel_workers = 0; pathnode->path.pathkeys = pathkeys; pathnode->partitioned_rels = list_copy(partitioned_rels); + + /* Add a projection step to each subpath if needed. */ + if (rel->consider_partitionwise_join && rel->has_target_wrvs) + subpaths = add_projection_to_subpaths(root, rel, subpaths); pathnode->subpaths = subpaths; /* *************** *** 1429,1434 **** create_merge_append_path(PlannerInfo *root, --- 1442,1524 ---- } /* + * add_projection_to_subpaths + * Add a projection step for rowtype conversion to given subpaths. + * + * This function adjusts subpaths accumulated for creating a partitionwise + * join path, by adding a projection to each subpath to convert the tuple + * layouts of whole-row Vars in the targetlist for that subpath back to their + * respective parents' rowtypes. Note that that would be safe, because it + * seems unlikely that adding that projection to such a subpath would alter + * the parallel safety, the ordering (pathkeys), the parameterization + * (required_outer rels), and the property of being the cheapest. + */ + static List * + add_projection_to_subpaths(PlannerInfo *root, RelOptInfo *rel, List *subpaths) + { + List *result = NIL; + ListCell *l; + + Assert(rel->consider_partitionwise_join && rel->has_target_wrvs); + + foreach(l, subpaths) + { + Path *subpath = (Path *) lfirst(l); + + result = lappend(result, + add_projection_to_subpath(root, rel, subpath)); + } + return result; + } + + /* + * add_projection_to_subpath + * Add a projection step for rowtype conversion to given subpath. + */ + static Path * + add_projection_to_subpath(PlannerInfo *root, RelOptInfo *rel, Path *subpath) + { + PathTarget *target = copy_pathtarget(subpath->pathtarget); + List *new_exprs = NIL; + ListCell *parentvars; + ListCell *childvars; + + Assert(subpath->parent->has_target_wrvs); + + /* + * Build a new targetlist for the subpath by adding a conversion node + * above each (converted) whole-row Var in the subpath's targetlist to + * convert the tuple layout of that Var back to its parent rowtype. + * + * By construction, subpath's targetlist is 1-to-1 with parent's. + */ + forboth(parentvars, rel->reltarget->exprs, childvars, target->exprs) + { + Var *parentvar = (Var *) lfirst(parentvars); + Node *childvar = (Node *) lfirst(childvars); + + if (IsA(parentvar, Var) && parentvar->varattno == 0) + { + ConvertRowtypeExpr *cre = makeNode(ConvertRowtypeExpr); + + cre->arg = (Expr *) childvar; + cre->resulttype = parentvar->vartype; + cre->convertformat = COERCE_IMPLICIT_CAST; + cre->location = -1; + new_exprs = lappend(new_exprs, cre); + } + else + new_exprs = lappend(new_exprs, childvar); + } + target->exprs = new_exprs; + + return (Path *) create_projection_path(root, + subpath->parent, + subpath, + target); + } + + /* * create_result_path * Creates a path representing a Result-and-nothing-else plan. * *************** *** 2392,2399 **** create_projection_path(PlannerInfo *root, pathnode->path.pathtype = T_Result; pathnode->path.parent = rel; pathnode->path.pathtarget = target; ! /* For now, assume we are above any joins, so no parameterization */ ! pathnode->path.param_info = NULL; pathnode->path.parallel_aware = false; pathnode->path.parallel_safe = rel->consider_parallel && subpath->parallel_safe && --- 2482,2488 ---- pathnode->path.pathtype = T_Result; pathnode->path.parent = rel; pathnode->path.pathtarget = target; ! pathnode->path.param_info = subpath->param_info; pathnode->path.parallel_aware = false; pathnode->path.parallel_safe = rel->consider_parallel && subpath->parallel_safe && *** a/src/backend/optimizer/util/placeholder.c --- b/src/backend/optimizer/util/placeholder.c *************** *** 20,26 **** #include "optimizer/pathnode.h" #include "optimizer/placeholder.h" #include "optimizer/planmain.h" - #include "optimizer/prep.h" #include "optimizer/var.h" #include "utils/lsyscache.h" --- 20,25 ---- *************** *** 415,424 **** add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel, Relids relids = joinrel->relids; ListCell *lc; - /* This function is called only on the parent relations. */ - Assert(!IS_OTHER_REL(joinrel) && !IS_OTHER_REL(outer_rel) && - !IS_OTHER_REL(inner_rel)); - foreach(lc, root->placeholder_list) { PlaceHolderInfo *phinfo = (PlaceHolderInfo *) lfirst(lc); --- 414,419 ---- *************** *** 464,519 **** add_placeholders_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel, } } } - - /* - * add_placeholders_to_child_joinrel - * Translate the PHVs in parent's targetlist and add them to the child's - * targetlist. Also adjust the cost - */ - void - add_placeholders_to_child_joinrel(PlannerInfo *root, RelOptInfo *childrel, - RelOptInfo *parentrel) - { - ListCell *lc; - AppendRelInfo **appinfos; - int nappinfos; - - Assert(IS_JOIN_REL(childrel) && IS_JOIN_REL(parentrel)); - Assert(IS_OTHER_REL(childrel)); - - /* Nothing to do if no PHVs. */ - if (root->placeholder_list == NIL) - return; - - appinfos = find_appinfos_by_relids(root, childrel->relids, &nappinfos); - foreach(lc, parentrel->reltarget->exprs) - { - PlaceHolderVar *phv = lfirst(lc); - - if (IsA(phv, PlaceHolderVar)) - { - /* - * In case the placeholder Var refers to any of the parent - * relations, translate it to refer to the corresponding child. - */ - if (bms_overlap(phv->phrels, parentrel->relids) && - childrel->reloptkind == RELOPT_OTHER_JOINREL) - { - phv = (PlaceHolderVar *) adjust_appendrel_attrs(root, - (Node *) phv, - nappinfos, - appinfos); - } - - childrel->reltarget->exprs = lappend(childrel->reltarget->exprs, - phv); - } - } - - /* Adjust the cost and width of child targetlist. */ - childrel->reltarget->cost.startup = parentrel->reltarget->cost.startup; - childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple; - childrel->reltarget->width = parentrel->reltarget->width; - - pfree(appinfos); - } --- 459,461 ---- *** a/src/backend/optimizer/util/relnode.c --- b/src/backend/optimizer/util/relnode.c *************** *** 16,21 **** --- 16,22 ---- #include + #include "catalog/pg_class.h" #include "miscadmin.h" #include "optimizer/clauses.h" #include "optimizer/cost.h" *************** *** 28,33 **** --- 29,35 ---- #include "optimizer/tlist.h" #include "partitioning/partbounds.h" #include "utils/hsearch.h" + #include "utils/lsyscache.h" typedef struct JoinHashEntry *************** *** 37,43 **** typedef struct JoinHashEntry } JoinHashEntry; static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel, ! RelOptInfo *input_rel); static List *build_joinrel_restrictlist(PlannerInfo *root, RelOptInfo *joinrel, RelOptInfo *outer_rel, --- 39,45 ---- } JoinHashEntry; static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel, ! RelOptInfo *input_rel, bool *has_wholerow); static List *build_joinrel_restrictlist(PlannerInfo *root, RelOptInfo *joinrel, RelOptInfo *outer_rel, *************** *** 57,62 **** static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel); --- 59,69 ---- static void build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel, RelOptInfo *inner_rel, List *restrictlist, JoinType jointype); + static void build_child_join_reltarget(PlannerInfo *root, + RelOptInfo *parentrel, + RelOptInfo *childrel, + int nappinfos, + AppendRelInfo **appinfos); /* *************** *** 188,193 **** build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent) --- 195,204 ---- rel->baserestrict_min_security = UINT_MAX; rel->joininfo = NIL; rel->has_eclass_joins = false; + rel->consider_partitionwise_join = false; /* might get changed later */ + rel->top_parent_rowtype = InvalidOid; + rel->top_parent_relids = NULL; + rel->has_target_wrvs = false; rel->part_scheme = NULL; rel->nparts = 0; rel->boundinfo = NULL; *************** *** 201,207 **** build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent) * Pass top parent's relids down the inheritance hierarchy. If the parent * has top_parent_relids set, it's a direct or an indirect child of the * top parent indicated by top_parent_relids. By extension this child is ! * also an indirect child of that parent. */ if (parent) { --- 212,219 ---- * Pass top parent's relids down the inheritance hierarchy. If the parent * has top_parent_relids set, it's a direct or an indirect child of the * top parent indicated by top_parent_relids. By extension this child is ! * also an indirect child of that parent. Likeweise for the OID of that ! * parent's rowtype. */ if (parent) { *************** *** 209,218 **** build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent) --- 221,237 ---- rel->top_parent_relids = parent->top_parent_relids; else rel->top_parent_relids = bms_copy(parent->relids); + + rel->top_parent_rowtype = parent->top_parent_rowtype; } else + { rel->top_parent_relids = NULL; + if (rte->relkind == RELKIND_PARTITIONED_TABLE) + rel->top_parent_rowtype = get_rel_type_id(rte->relid); + } + /* Check type of rtable entry */ switch (rte->rtekind) { *************** *** 525,530 **** build_join_rel(PlannerInfo *root, --- 544,550 ---- { RelOptInfo *joinrel; List *restrictlist; + bool has_wholerow; /* This function should be used only for join between parents. */ Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel)); *************** *** 602,608 **** build_join_rel(PlannerInfo *root, --- 622,631 ---- joinrel->baserestrict_min_security = UINT_MAX; joinrel->joininfo = NIL; joinrel->has_eclass_joins = false; + joinrel->consider_partitionwise_join = false; /* might get changed later */ + joinrel->top_parent_rowtype = InvalidOid; joinrel->top_parent_relids = NULL; + joinrel->has_target_wrvs = false; joinrel->part_scheme = NULL; joinrel->nparts = 0; joinrel->boundinfo = NULL; *************** *** 623,630 **** build_join_rel(PlannerInfo *root, * and inner rels we first try to build it from. But the contents should * be the same regardless. */ ! build_joinrel_tlist(root, joinrel, outer_rel); ! build_joinrel_tlist(root, joinrel, inner_rel); add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel); /* --- 646,654 ---- * and inner rels we first try to build it from. But the contents should * be the same regardless. */ ! has_wholerow = false; ! build_joinrel_tlist(root, joinrel, outer_rel, &has_wholerow); ! build_joinrel_tlist(root, joinrel, inner_rel, &has_wholerow); add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel); /* *************** *** 660,665 **** build_join_rel(PlannerInfo *root, --- 684,698 ---- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist, sjinfo->jointype); + /* Set the consider_partitionwise_join flag. */ + if (IS_PARTITIONED_REL(joinrel) && + !bms_equal(joinrel->relids, root->all_baserels)) + joinrel->consider_partitionwise_join = true; + + /* Check whether the joinrel's targetlist contain whole-row Vars. */ + if (has_wholerow) + joinrel->has_target_wrvs = true; + /* * Set estimates of the joinrel's size. */ *************** *** 773,779 **** build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel, --- 806,815 ---- joinrel->baserestrictcost.per_tuple = 0; joinrel->joininfo = NIL; joinrel->has_eclass_joins = false; + joinrel->consider_partitionwise_join = false; /* might get changed later */ + joinrel->top_parent_rowtype = InvalidOid; joinrel->top_parent_relids = NULL; + joinrel->has_target_wrvs = false; joinrel->part_scheme = NULL; joinrel->nparts = 0; joinrel->boundinfo = NULL; *************** *** 789,802 **** build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel, /* Compute information relevant to foreign relations. */ set_foreign_rel_properties(joinrel, outer_rel, inner_rel); ! /* Build targetlist */ ! build_joinrel_tlist(root, joinrel, outer_rel); ! build_joinrel_tlist(root, joinrel, inner_rel); ! /* Add placeholder variables. */ ! add_placeholders_to_child_joinrel(root, joinrel, parent_joinrel); /* Construct joininfo list. */ - appinfos = find_appinfos_by_relids(root, joinrel->relids, &nappinfos); joinrel->joininfo = (List *) adjust_appendrel_attrs(root, (Node *) parent_joinrel->joininfo, nappinfos, --- 825,837 ---- /* Compute information relevant to foreign relations. */ set_foreign_rel_properties(joinrel, outer_rel, inner_rel); ! appinfos = find_appinfos_by_relids(root, joinrel->relids, &nappinfos); ! ! /* Set up reltarget struct */ ! build_child_join_reltarget(root, parent_joinrel, joinrel, ! nappinfos, appinfos); /* Construct joininfo list. */ joinrel->joininfo = (List *) adjust_appendrel_attrs(root, (Node *) parent_joinrel->joininfo, nappinfos, *************** *** 889,909 **** min_join_parameterization(PlannerInfo *root, * Vars from the specified input rel's tlist to the join rel's tlist. * * We also compute the expected width of the join's output, making use ! * of data that was cached at the baserel level by set_rel_width(). */ static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel, ! RelOptInfo *input_rel) { ! Relids relids; ListCell *vars; - /* attrs_needed refers to parent relids and not those of a child. */ - if (joinrel->top_parent_relids) - relids = joinrel->top_parent_relids; - else - relids = joinrel->relids; - foreach(vars, input_rel->reltarget->exprs) { Var *var = (Var *) lfirst(vars); --- 924,940 ---- * Vars from the specified input rel's tlist to the join rel's tlist. * * We also compute the expected width of the join's output, making use ! * of data that was cached at the baserel level by set_rel_width(), and ! * check whether the join's output contains whole-row Vars for later use ! * in partitionwise joins. */ static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel, ! RelOptInfo *input_rel, bool *has_wholerow) { ! Relids relids = joinrel->relids; ListCell *vars; foreach(vars, input_rel->reltarget->exprs) { Var *var = (Var *) lfirst(vars); *************** *** 919,973 **** build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel, /* * Otherwise, anything in a baserel or joinrel targetlist ought to be ! * a Var. Children of a partitioned table may have ConvertRowtypeExpr ! * translating whole-row Var of a child to that of the parent. ! * Children of an inherited table or subquery child rels can not ! * directly participate in a join, so other kinds of nodes here. */ ! if (IsA(var, Var)) ! { ! baserel = find_base_rel(root, var->varno); ! ndx = var->varattno - baserel->min_attr; ! } ! else if (IsA(var, ConvertRowtypeExpr)) ! { ! ConvertRowtypeExpr *child_expr = (ConvertRowtypeExpr *) var; ! Var *childvar = (Var *) child_expr->arg; ! ! /* ! * Child's whole-row references are converted to look like those ! * of parent using ConvertRowtypeExpr. There can be as many ! * ConvertRowtypeExpr decorations as the depth of partition tree. ! * The argument to the deepest ConvertRowtypeExpr is expected to ! * be a whole-row reference of the child. ! */ ! while (IsA(childvar, ConvertRowtypeExpr)) ! { ! child_expr = (ConvertRowtypeExpr *) childvar; ! childvar = (Var *) child_expr->arg; ! } ! Assert(IsA(childvar, Var) &&childvar->varattno == 0); ! ! baserel = find_base_rel(root, childvar->varno); ! ndx = 0 - baserel->min_attr; ! } ! else elog(ERROR, "unexpected node type in rel targetlist: %d", (int) nodeTag(var)); ! /* Is the target expression still needed above this joinrel? */ if (bms_nonempty_difference(baserel->attr_needed[ndx], relids)) { /* Yup, add it to the output */ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs, var); ! ! /* ! * Vars have cost zero, so no need to adjust reltarget->cost. Even ! * if it's a ConvertRowtypeExpr, it will be computed only for the ! * base relation, costing nothing for a join. ! */ joinrel->reltarget->width += baserel->attr_widths[ndx]; } } } --- 950,977 ---- /* * Otherwise, anything in a baserel or joinrel targetlist ought to be ! * a Var. (More general cases can only appear in appendrel child ! * rels, which will never be seen here.) */ ! if (!IsA(var, Var)) elog(ERROR, "unexpected node type in rel targetlist: %d", (int) nodeTag(var)); + /* Get the Var's original base rel */ + baserel = find_base_rel(root, var->varno); ! /* Is it still needed above this joinrel? */ ! ndx = var->varattno - baserel->min_attr; if (bms_nonempty_difference(baserel->attr_needed[ndx], relids)) { /* Yup, add it to the output */ joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs, var); ! /* Vars have cost zero, so no need to adjust reltarget->cost */ joinrel->reltarget->width += baserel->attr_widths[ndx]; + + /* If it's a whole-row Var, set the flag */ + if (var->varattno == InvalidAttrNumber) + *has_wholerow = true; } } } *************** *** 1215,1220 **** fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids) --- 1219,1227 ---- upperrel->cheapest_unique_path = NULL; upperrel->cheapest_parameterized_paths = NIL; + upperrel->consider_partitionwise_join = false; + upperrel->has_target_wrvs = false; + root->upper_rels[kind] = lappend(root->upper_rels[kind], upperrel); return upperrel; *************** *** 1768,1770 **** build_joinrel_partition_info(RelOptInfo *joinrel, RelOptInfo *outer_rel, --- 1775,1893 ---- joinrel->nullable_partexprs[cnt] = nullable_partexpr; } } + + /* + * build_childrel_tlist + * Build the child's targetlist from the parent's with + * adjust_appendrel_attrs + */ + void + build_childrel_tlist(PlannerInfo *root, + RelOptInfo *parentrel, + RelOptInfo *childrel, + int nappinfos, + AppendRelInfo **appinfos) + { + ListCell *lc; + + /* + * If we don't consider partitionwise join paths, or if the parent's + * targetlist doesn't contain any whole-row Vars, just apply + * adjust_appendrel_attrs to the parent's targetlist. + * + * NB: the resulting childrel->reltarget->exprs may contain arbitrary + * expressions, which otherwise would not occur in a rel's targetlist. + * Code that might be looking at an appendrel child must cope with such. + * (Normally, a rel's targetlist would only include Vars and + * PlaceHolderVars.) XXX we do not bother to update the cost or width + * fields of childrel->reltarget; not clear if that would be useful. + */ + if (!parentrel->consider_partitionwise_join || + !parentrel->has_target_wrvs) + { + childrel->reltarget->exprs = (List *) + adjust_appendrel_attrs(root, + (Node *) parentrel->reltarget->exprs, + nappinfos, appinfos); + return; + } + + /* The parent should be partitioned */ + Assert(parentrel->part_scheme); + + /* + * We build the child's targetlist not to contain ConvertRowtypeExprs: + * if the entry in the parent's targetlist is a whole-row Var for one of + * the parent's rels, create a whole-row Var for the corresponding child + * rel, instead of applying adjust_appendrel_attrs to the Var, and add it + * to the child's targetlist; otherwise apply adjust_appendrel_attrs to + * the entry in that targetlist. + * + * We also create an output that should be emitted by the child's rel for + * the topmost parent's rel. + */ + foreach(lc, parentrel->reltarget->exprs) + { + Node *node = (Node *) lfirst(lc); + + /* Should be a simple Var or PlaceHolderVar ... */ + Assert(IsA(node, Var) || IsA(node, PlaceHolderVar)); + + if (IsA(node, Var) && ((Var *) node)->varattno == 0) + { + Var *var = (Var *) copyObject(node); + AppendRelInfo *appinfo = NULL; + int cnt; + + /* Find the AppendRelInfo associated with the parent */ + for (cnt = 0; cnt < nappinfos; cnt++) + { + if (var->varno == appinfos[cnt]->parent_relid) + { + appinfo = appinfos[cnt]; + break; + } + } + /* Should have found the entry ... */ + Assert(appinfo); + var->varno = appinfo->child_relid; + var->varnoold = appinfo->child_relid; + var->vartype = appinfo->child_reltype; + childrel->reltarget->exprs = + lappend(childrel->reltarget->exprs, var); + childrel->has_target_wrvs = true; + } + else + { + childrel->reltarget->exprs = + lappend(childrel->reltarget->exprs, + adjust_appendrel_attrs(root, node, + nappinfos, + appinfos)); + } + } + Assert(childrel->has_target_wrvs); + + if (childrel->part_scheme) + childrel->consider_partitionwise_join = true; + } + + /* + * build_child_join_reltarget + * Set up a child-join relation's reltarget from a parent-join relation. + */ + static void + build_child_join_reltarget(PlannerInfo *root, + RelOptInfo *parentrel, + RelOptInfo *childrel, + int nappinfos, + AppendRelInfo **appinfos) + { + /* Build the targetlist */ + build_childrel_tlist(root, parentrel, childrel, nappinfos, appinfos); + + /* Set the cost and width fields */ + childrel->reltarget->cost.startup = parentrel->reltarget->cost.startup; + childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple; + childrel->reltarget->width = parentrel->reltarget->width; + } *** a/src/include/nodes/relation.h --- b/src/include/nodes/relation.h *************** *** 687,694 **** typedef struct RelOptInfo * involving this rel */ bool has_eclass_joins; /* T means joininfo is incomplete */ ! /* used by "other" relations */ ! Relids top_parent_relids; /* Relids of topmost parents */ /* used for partitioned relations */ PartitionScheme part_scheme; /* Partitioning scheme. */ --- 687,701 ---- * involving this rel */ bool has_eclass_joins; /* T means joininfo is incomplete */ ! /* per-partitioned-relation planner control flags */ ! bool consider_partitionwise_join; /* consider partitionwise ! * join paths? */ ! ! /* used by partitionwise joins: */ ! Oid top_parent_rowtype; /* OID of topmost parent's rowtype */ ! Relids top_parent_relids; /* Relids of topmost parent(s) */ ! bool has_target_wrvs; /* T means reltarget->exprs contains ! * whole-row Vars */ /* used for partitioned relations */ PartitionScheme part_scheme; /* Partitioning scheme. */ *** a/src/include/optimizer/pathnode.h --- b/src/include/optimizer/pathnode.h *************** *** 297,301 **** extern RelOptInfo *build_child_join_rel(PlannerInfo *root, --- 297,306 ---- RelOptInfo *outer_rel, RelOptInfo *inner_rel, RelOptInfo *parent_joinrel, List *restrictlist, SpecialJoinInfo *sjinfo, JoinType jointype); + extern void build_childrel_tlist(PlannerInfo *root, + RelOptInfo *parentrel, + RelOptInfo *childrel, + int nappinfos, + AppendRelInfo **appinfos); #endif /* PATHNODE_H */ *** a/src/test/regress/expected/partition_join.out --- b/src/test/regress/expected/partition_join.out *************** *** 476,481 **** SELECT t1.a, ss.t2a, ss.t2c FROM prt1 t1 LEFT JOIN LATERAL --- 476,554 ---- 550 | | (12 rows) + -- Check whole-row-Var handling in situations where the partitioned rel is not + -- the topmost scan/join relation + CREATE TABLE nonprt (a int, x text default 'testing'); + INSERT INTO nonprt (a) SELECT i FROM generate_series(0, 599) i WHERE i % 6 = 0; + ANALYZE nonprt; + EXPLAIN (COSTS OFF) + SELECT t1, t3 FROM prt1 t1, nonprt t3 WHERE t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + QUERY PLAN + -------------------------------------------------- + Sort + Sort Key: t1.a + -> Hash Join + Hash Cond: (t3.a = t1.a) + -> Seq Scan on nonprt t3 + -> Hash + -> Append + -> Seq Scan on prt1_p1 t1 + Filter: (b = 0) + -> Seq Scan on prt1_p2 t1_1 + Filter: (b = 0) + -> Seq Scan on prt1_p3 t1_2 + Filter: (b = 0) + (13 rows) + + SELECT t1, t3 FROM prt1 t1, nonprt t3 WHERE t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + t1 | t3 + --------------+--------------- + (0,0,0000) | (0,testing) + (150,0,0150) | (150,testing) + (300,0,0300) | (300,testing) + (450,0,0450) | (450,testing) + (4 rows) + + EXPLAIN (COSTS OFF) + SELECT t1, t2, t3 FROM prt1 t1, prt2 t2, nonprt t3 WHERE t1.a = t2.b AND t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + QUERY PLAN + -------------------------------------------------------------- + Sort + Sort Key: t1.a + -> Hash Join + Hash Cond: (t3.a = t1.a) + -> Seq Scan on nonprt t3 + -> Hash + -> Append + -> Hash Join + Hash Cond: (t2.b = t1.a) + -> Seq Scan on prt2_p1 t2 + -> Hash + -> Seq Scan on prt1_p1 t1 + Filter: (b = 0) + -> Hash Join + Hash Cond: (t2_1.b = t1_1.a) + -> Seq Scan on prt2_p2 t2_1 + -> Hash + -> Seq Scan on prt1_p2 t1_1 + Filter: (b = 0) + -> Hash Join + Hash Cond: (t2_2.b = t1_2.a) + -> Seq Scan on prt2_p3 t2_2 + -> Hash + -> Seq Scan on prt1_p3 t1_2 + Filter: (b = 0) + (25 rows) + + SELECT t1, t2, t3 FROM prt1 t1, prt2 t2, nonprt t3 WHERE t1.a = t2.b AND t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + t1 | t2 | t3 + --------------+--------------+--------------- + (0,0,0000) | (0,0,0000) | (0,testing) + (150,0,0150) | (0,150,0150) | (150,testing) + (300,0,0300) | (0,300,0300) | (300,testing) + (450,0,0450) | (0,450,0450) | (450,testing) + (4 rows) + -- -- partitioned by expression -- *************** *** 1042,1047 **** SELECT t1.a, t2.b FROM (SELECT * FROM prt1 WHERE a < 450) t1 LEFT JOIN (SELECT * --- 1115,1163 ---- 400 | (9 rows) + -- merge join when expression with whole-row reference needs to be sorted + EXPLAIN (COSTS OFF) + SELECT t1.a, t2.b FROM prt1 t1, prt2 t2 WHERE t1::text = t2::text AND t1.a = t2.b ORDER BY t1.a; + QUERY PLAN + ----------------------------------------------------------------------------------- + Merge Append + Sort Key: t1.a + -> Merge Join + Merge Cond: ((t1.a = t2.b) AND (((t1.*)::text) = ((t2.*)::text))) + -> Sort + Sort Key: t1.a, ((t1.*)::text) + -> Seq Scan on prt1_p1 t1 + -> Sort + Sort Key: t2.b, ((t2.*)::text) + -> Seq Scan on prt2_p1 t2 + -> Merge Join + Merge Cond: ((t1_1.a = t2_1.b) AND (((t1_1.*)::text) = ((t2_1.*)::text))) + -> Sort + Sort Key: t1_1.a, ((t1_1.*)::text) + -> Seq Scan on prt1_p2 t1_1 + -> Sort + Sort Key: t2_1.b, ((t2_1.*)::text) + -> Seq Scan on prt2_p2 t2_1 + -> Merge Join + Merge Cond: ((t1_2.a = t2_2.b) AND (((t1_2.*)::text) = ((t2_2.*)::text))) + -> Sort + Sort Key: t1_2.a, ((t1_2.*)::text) + -> Seq Scan on prt1_p3 t1_2 + -> Sort + Sort Key: t2_2.b, ((t2_2.*)::text) + -> Seq Scan on prt2_p3 t2_2 + (26 rows) + + SELECT t1.a, t2.b FROM prt1 t1, prt2 t2 WHERE t1::text = t2::text AND t1.a = t2.b ORDER BY t1.a; + a | b + ----+---- + 0 | 0 + 6 | 6 + 12 | 12 + 18 | 18 + 24 | 24 + (5 rows) + RESET enable_hashjoin; RESET enable_nestloop; -- *** a/src/test/regress/sql/partition_join.sql --- b/src/test/regress/sql/partition_join.sql *************** *** 89,94 **** SELECT t1.a, ss.t2a, ss.t2c FROM prt1 t1 LEFT JOIN LATERAL --- 89,108 ---- (SELECT t2.a AS t2a, t3.a AS t3a, t2.b t2b, t2.c t2c, least(t1.a,t2.a,t3.a) FROM prt1 t2 JOIN prt2 t3 ON (t2.a = t3.b)) ss ON t1.c = ss.t2c WHERE (t1.b + coalesce(ss.t2b, 0)) = 0 ORDER BY t1.a; + -- Check whole-row-Var handling in situations where the partitioned rel is not + -- the topmost scan/join relation + CREATE TABLE nonprt (a int, x text default 'testing'); + INSERT INTO nonprt (a) SELECT i FROM generate_series(0, 599) i WHERE i % 6 = 0; + ANALYZE nonprt; + + EXPLAIN (COSTS OFF) + SELECT t1, t3 FROM prt1 t1, nonprt t3 WHERE t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + SELECT t1, t3 FROM prt1 t1, nonprt t3 WHERE t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + + EXPLAIN (COSTS OFF) + SELECT t1, t2, t3 FROM prt1 t1, prt2 t2, nonprt t3 WHERE t1.a = t2.b AND t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + SELECT t1, t2, t3 FROM prt1 t1, prt2 t2, nonprt t3 WHERE t1.a = t2.b AND t1.a = t3.a AND t1.b = 0 ORDER BY t1.a; + -- -- partitioned by expression -- *************** *** 160,165 **** EXPLAIN (COSTS OFF) --- 174,184 ---- SELECT t1.a, t2.b FROM (SELECT * FROM prt1 WHERE a < 450) t1 LEFT JOIN (SELECT * FROM prt2 WHERE b > 250) t2 ON t1.a = t2.b WHERE t1.b = 0 ORDER BY t1.a, t2.b; SELECT t1.a, t2.b FROM (SELECT * FROM prt1 WHERE a < 450) t1 LEFT JOIN (SELECT * FROM prt2 WHERE b > 250) t2 ON t1.a = t2.b WHERE t1.b = 0 ORDER BY t1.a, t2.b; + -- merge join when expression with whole-row reference needs to be sorted + EXPLAIN (COSTS OFF) + SELECT t1.a, t2.b FROM prt1 t1, prt2 t2 WHERE t1::text = t2::text AND t1.a = t2.b ORDER BY t1.a; + SELECT t1.a, t2.b FROM prt1 t1, prt2 t2 WHERE t1::text = t2::text AND t1.a = t2.b ORDER BY t1.a; + RESET enable_hashjoin; RESET enable_nestloop; --------------040405050600000402020703--