public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v52 5/7] Doc part of shared-memory based stats collector.
9+ messages / 5 participants
[nested] [flat]
* [PATCH v52 5/7] Doc part of shared-memory based stats collector.
@ 2020-03-19 06:11 Kyotaro Horiguchi <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-19 06:11 UTC (permalink / raw)
---
doc/src/sgml/catalogs.sgml | 6 +-
doc/src/sgml/config.sgml | 27 +++---
doc/src/sgml/high-availability.sgml | 13 +--
doc/src/sgml/monitoring.sgml | 127 +++++++++++++---------------
doc/src/sgml/ref/pg_dump.sgml | 9 +-
src/backend/postmaster/postmaster.c | 2 -
6 files changed, 86 insertions(+), 98 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0674..0ef684d4d0 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9261,9 +9261,9 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l
<para>
<xref linkend="view-table"/> lists the system views described here.
More detailed documentation of each view follows below.
- There are some additional views that provide access to the results of
- the statistics collector; they are described in <xref
- linkend="monitoring-stats-views-table"/>.
+ There are some additional views that provide access to the activity
+ statistics; they are described in
+ <xref linkend="monitoring-stats-views-table"/>.
</para>
<para>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 529876895b..99a8664026 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -7375,11 +7375,11 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
<title>Run-time Statistics</title>
<sect2 id="runtime-config-statistics-collector">
- <title>Query and Index Statistics Collector</title>
+ <title>Query and Index Activity Statistics</title>
<para>
- These parameters control server-wide statistics collection features.
- When statistics collection is enabled, the data that is produced can be
+ These parameters control server-wide activity statistics features.
+ When activity statistics is enabled, the data that is produced can be
accessed via the <structname>pg_stat</structname> and
<structname>pg_statio</structname> family of system views.
Refer to <xref linkend="monitoring"/> for more information.
@@ -7395,14 +7395,13 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</term>
<listitem>
<para>
- Enables the collection of information on the currently
- executing command of each session, along with the time when
- that command began execution. This parameter is on by
- default. Note that even when enabled, this information is not
- visible to all users, only to superusers and the user owning
- the session being reported on, so it should not represent a
- security risk.
- Only superusers can change this setting.
+ Enables activity tracking on the currently executing command of
+ each session, along with the time when that command began
+ execution. This parameter is on by default. Note that even when
+ enabled, this information is not visible to all users, only to
+ superusers and the user owning the session being reported on, so it
+ should not represent a security risk. Only superusers can change this
+ setting.
</para>
</listitem>
</varlistentry>
@@ -7433,9 +7432,9 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</term>
<listitem>
<para>
- Enables collection of statistics on database activity.
+ Enables tracking of database activity.
This parameter is on by default, because the autovacuum
- daemon needs the collected information.
+ daemon needs the activity information.
Only superusers can change this setting.
</para>
</listitem>
@@ -8554,7 +8553,7 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
<listitem>
<para>
Specifies the fraction of the total number of heap tuples counted in
- the previous statistics collection that can be inserted without
+ the previously collected statistics that can be inserted without
incurring an index scan at the <command>VACUUM</command> cleanup stage.
This setting currently applies to B-tree indexes only.
</para>
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index f49f5c0108..45095857eb 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -2217,12 +2217,13 @@ HINT: You can then restart the server after making the necessary configuration
</para>
<para>
- The statistics collector is active during recovery. All scans, reads, blocks,
- index usage, etc., will be recorded normally on the standby. Replayed
- actions will not duplicate their effects on primary, so replaying an
- insert will not increment the Inserts column of pg_stat_user_tables.
- The stats file is deleted at the start of recovery, so stats from primary
- and standby will differ; this is considered a feature, not a bug.
+ The activity statistics is collected during recovery. All scans, reads,
+ blocks, index usage, etc., will be recorded normally on the
+ standby. Replayed actions will not duplicate their effects on primary, so
+ replaying an insert will not increment the Inserts column of
+ pg_stat_user_tables. The activity statistics is reset at the start of
+ recovery, so stats from primary and standby will differ; this is
+ considered a feature, not a bug.
</para>
<para>
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 3335d71eba..6217ed9792 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -22,7 +22,7 @@
<para>
Several tools are available for monitoring database activity and
analyzing performance. Most of this chapter is devoted to describing
- <productname>PostgreSQL</productname>'s statistics collector,
+ <productname>PostgreSQL</productname>'s activity statistics,
but one should not neglect regular Unix monitoring programs such as
<command>ps</command>, <command>top</command>, <command>iostat</command>, and <command>vmstat</command>.
Also, once one has identified a
@@ -53,7 +53,6 @@ postgres 15554 0.0 0.0 57536 1184 ? Ss 18:02 0:00 postgres: back
postgres 15555 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: checkpointer
postgres 15556 0.0 0.0 57536 916 ? Ss 18:02 0:00 postgres: walwriter
postgres 15557 0.0 0.0 58504 2244 ? Ss 18:02 0:00 postgres: autovacuum launcher
-postgres 15558 0.0 0.0 17512 1068 ? Ss 18:02 0:00 postgres: stats collector
postgres 15582 0.0 0.0 58772 3080 ? Ss 18:04 0:00 postgres: joe runbug 127.0.0.1 idle
postgres 15606 0.0 0.0 58772 3052 ? Ss 18:07 0:00 postgres: tgl regression [local] SELECT waiting
postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl regression [local] idle in transaction
@@ -65,9 +64,8 @@ postgres 15610 0.0 0.0 58772 3056 ? Ss 18:07 0:00 postgres: tgl
primary server process. The command arguments
shown for it are the same ones used when it was launched. The next five
processes are background worker processes automatically launched by the
- primary process. (The <quote>stats collector</quote> process will not be present
- if you have set the system not to start the statistics collector; likewise
- the <quote>autovacuum launcher</quote> process can be disabled.)
+ primary process. (The <quote>autovacuum launcher</quote> process will not
+ be present if you have set the system not to start it.)
Each of the remaining
processes is a server process handling one client connection. Each such
process sets its command line display in the form
@@ -130,20 +128,21 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</sect1>
<sect1 id="monitoring-stats">
- <title>The Statistics Collector</title>
+ <title>The Activity Statistics</title>
<indexterm zone="monitoring-stats">
<primary>statistics</primary>
</indexterm>
<para>
- <productname>PostgreSQL</productname>'s <firstterm>statistics collector</firstterm>
- is a subsystem that supports collection and reporting of information about
- server activity. Presently, the collector can count accesses to tables
- and indexes in both disk-block and individual-row terms. It also tracks
- the total number of rows in each table, and information about vacuum and
- analyze actions for each table. It can also count calls to user-defined
- functions and the total time spent in each one.
+ <productname>PostgreSQL</productname>'s <firstterm>activity
+ statistics</firstterm> is a subsystem that supports tracking and reporting
+ of information about server activity. Presently, the activity statistics
+ tracks the count of accesses to tables and indexes in both disk-block and
+ individual-row terms. It also tracks the total number of rows in each
+ table, and information about vacuum and analyze actions for each table. It
+ can also track calls to user-defined functions and the total time spent in
+ each one.
</para>
<para>
@@ -151,15 +150,15 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
information about exactly what is going on in the system right now, such as
the exact command currently being executed by other server processes, and
which other connections exist in the system. This facility is independent
- of the collector process.
+ of the activity statistics.
</para>
<sect2 id="monitoring-stats-setup">
- <title>Statistics Collection Configuration</title>
+ <title>Activity Statistics Configuration</title>
<para>
- Since collection of statistics adds some overhead to query execution,
- the system can be configured to collect or not collect information.
+ Since tracking for the activity statistics adds some overhead to query
+ execution, the system can be configured to track or not track activity.
This is controlled by configuration parameters that are normally set in
<filename>postgresql.conf</filename>. (See <xref linkend="runtime-config"/> for
details about setting configuration parameters.)
@@ -172,7 +171,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<para>
The parameter <xref linkend="guc-track-counts"/> controls whether
- statistics are collected about table and index accesses.
+ to track activity about table and index accesses.
</para>
<para>
@@ -201,18 +200,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</para>
<para>
- The statistics collector transmits the collected information to other
- <productname>PostgreSQL</productname> processes through temporary files.
- These files are stored in the directory named by the
- <xref linkend="guc-stats-temp-directory"/> parameter,
- <filename>pg_stat_tmp</filename> by default.
- For better performance, <varname>stats_temp_directory</varname> can be
- pointed at a RAM-based file system, decreasing physical I/O requirements.
- When the server shuts down cleanly, a permanent copy of the statistics
- data is stored in the <filename>pg_stat</filename> subdirectory, so that
- statistics can be retained across server restarts. When recovery is
- performed at server start (e.g., after immediate shutdown, server crash,
- and point-in-time recovery), all statistics counters are reset.
+ down cleanly, a permanent copy of the statistics data is stored in
+ the <filename>pg_stat</filename> subdirectory, so that statistics can be
+ retained across server restarts. When recovery is performed at server
+ start (e.g. after immediate shutdown, server crash, and point-in-time
+ recovery), all statistics counters are reset.
</para>
</sect2>
@@ -225,48 +217,46 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
linkend="monitoring-stats-dynamic-views-table"/>, are available to show
the current state of the system. There are also several other
views, listed in <xref
- linkend="monitoring-stats-views-table"/>, available to show the results
- of statistics collection. Alternatively, one can
- build custom views using the underlying statistics functions, as discussed
- in <xref linkend="monitoring-stats-functions"/>.
+ linkend="monitoring-stats-views-table"/>, available to show the activity
+ statistics. Alternatively, one can build custom views using the underlying
+ statistics functions, as discussed in
+ <xref linkend="monitoring-stats-functions"/>.
</para>
<para>
- When using the statistics to monitor collected data, it is important
- to realize that the information does not update instantaneously.
- Each individual server process transmits new statistical counts to
- the collector just before going idle; so a query or transaction still in
- progress does not affect the displayed totals. Also, the collector itself
- emits a new report at most once per <varname>PGSTAT_STAT_INTERVAL</varname>
- milliseconds (500 ms unless altered while building the server). So the
- displayed information lags behind actual activity. However, current-query
- information collected by <varname>track_activities</varname> is
- always up-to-date.
+ When using the activity statistics, it is important to realize that the
+ information does not update instantaneously. Each individual server writes
+ out new statistical counts just before going idle, not frequent than once
+ per <varname>PGSTAT_STAT_INTERVAL</varname> milliseconds (1 second unless
+ altered while building the server); so a query or transaction still in
+ progress does not affect the displayed totals. However, current-query
+ information tracked by <varname>track_activities</varname> is always
+ up-to-date.
</para>
<para>
Another important point is that when a server process is asked to display
- any of these statistics, it first fetches the most recent report emitted by
- the collector process and then continues to use this snapshot for all
- statistical views and functions until the end of its current transaction.
- So the statistics will show static information as long as you continue the
- current transaction. Similarly, information about the current queries of
- all sessions is collected when any such information is first requested
- within a transaction, and the same information will be displayed throughout
- the transaction.
- This is a feature, not a bug, because it allows you to perform several
- queries on the statistics and correlate the results without worrying that
- the numbers are changing underneath you. But if you want to see new
- results with each query, be sure to do the queries outside any transaction
- block. Alternatively, you can invoke
+ any of these statistics, it first reads the current statistics and then
+ continues to use this snapshot for all statistical views and functions
+ until the end of its current transaction. So the statistics will show
+ static information as long as you continue the current transaction.
+ Similarly, information about the current queries of all sessions is tracked
+ when any such information is first requested within a transaction, and the
+ same information will be displayed throughout the transaction. This is a
+ feature, not a bug, because it allows you to perform several queries on the
+ statistics and correlate the results without worrying that the numbers are
+ changing underneath you. But if you want to see new results with each
+ query, be sure to do the queries outside any transaction block.
+ Alternatively, you can invoke
<function>pg_stat_clear_snapshot</function>(), which will discard the
current transaction's statistics snapshot (if any). The next use of
statistical information will cause a new snapshot to be fetched.
</para>
-
+
<para>
- A transaction can also see its own statistics (as yet untransmitted to the
- collector) in the views <structname>pg_stat_xact_all_tables</structname>,
+ A transaction can also see its own statistics (as yet unwritten to the
+ server-wide activity statistics) in the
+ views <structname>pg_stat_xact_all_tables</structname>,
<structname>pg_stat_xact_sys_tables</structname>,
<structname>pg_stat_xact_user_tables</structname>, and
<structname>pg_stat_xact_user_functions</structname>. These numbers do not act as
@@ -648,7 +638,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
kernel's I/O cache, and might therefore still be fetched without
requiring a physical read. Users interested in obtaining more
detailed information on <productname>PostgreSQL</productname> I/O behavior are
- advised to use the <productname>PostgreSQL</productname> statistics collector
+ advised to use the <productname>PostgreSQL</productname> activity statistics
in combination with operating system utilities that allow insight
into the kernel's handling of I/O.
</para>
@@ -1085,10 +1075,6 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
<entry><literal>LogicalLauncherMain</literal></entry>
<entry>Waiting in main loop of logical replication launcher process.</entry>
</row>
- <row>
- <entry><literal>PgStatMain</literal></entry>
- <entry>Waiting in main loop of statistics collector process.</entry>
- </row>
<row>
<entry><literal>RecoveryWalStream</literal></entry>
<entry>Waiting in main loop of startup process for WAL to arrive, during
@@ -1843,6 +1829,10 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
</thead>
<tbody>
+ <row>
+ <entry><literal>ActivityStatistics</literal></entry>
+ <entry>Waiting to write out activity statistics to shared memory.</entry>
+ </row>
<row>
<entry><literal>AddinShmemInit</literal></entry>
<entry>Waiting to manage an extension's space allocation in shared
@@ -6123,9 +6113,10 @@ SELECT pg_stat_get_backend_pid(s.backendid) AS pid,
<entry><literal>performing final cleanup</literal></entry>
<entry>
<command>VACUUM</command> is performing final cleanup. During this phase,
- <command>VACUUM</command> will vacuum the free space map, update statistics
- in <literal>pg_class</literal>, and report statistics to the statistics
- collector. When this phase is completed, <command>VACUUM</command> will end.
+ <command>VACUUM</command> will vacuum the free space map, update
+ statistics in <literal>pg_class</literal>, and system-wide activity
+ statistics. When this phase is completed, <command>VACUUM</command>
+ will end.
</entry>
</row>
</tbody>
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index bcbb7a25fb..1fa59a2fdf 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -1280,11 +1280,10 @@ PostgreSQL documentation
</para>
<para>
- The database activity of <application>pg_dump</application> is
- normally collected by the statistics collector. If this is
- undesirable, you can set parameter <varname>track_counts</varname>
- to false via <envar>PGOPTIONS</envar> or the <literal>ALTER
- USER</literal> command.
+ The database activity of <application>pg_dump</application> is normally
+ collected. If this is undesirable, you can set
+ parameter <varname>track_counts</varname> to false
+ via <envar>PGOPTIONS</envar> or the <literal>ALTER USER</literal> command.
</para>
</refsect1>
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 0abc41c33c..d74b5eed82 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -6328,8 +6328,6 @@ restore_backend_variables(BackendParameters *param, Port *port)
if (postmaster_alive_fds[1] >= 0)
ReserveExternalFD();
#endif
- if (pgStatSock != PGINVALID_SOCKET)
- ReserveExternalFD();
}
--
2.27.0
----Next_Part(Wed_Mar_10_17_51_37_2021_192)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v52-0006-Remove-the-GUC-stats_temp_directory.patch"
^ permalink raw reply [nested|flat] 9+ messages in thread
* Retiring is_pushed_down
@ 2023-07-25 07:39 Richard Guo <[email protected]>
0 siblings, 2 replies; 9+ messages in thread
From: Richard Guo @ 2023-07-25 07:39 UTC (permalink / raw)
To: pgsql-hackers
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, I think we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way. (Interesting that optimizer/README actually describes this way in
section 'Relation Identification and Qual Clause Placement'.)
So I give it a try to retire is_pushed_down as attached. But there are
several points that may need more thoughts.
* When we form an outer join, it's possible that more than one outer
join relid is added to the join's relid set, if there are any pushed
down outer joins per identity 3. And it's also possible that no outer
join relid is added, for an outer join that has been pushed down. So
instead of checking if a qual clause's required_relids include the outer
join's relid, I think we should check if its required_relids overlap
the outer join relids that are being formed, which means that we should
use bms_overlap(rinfo->required_relids, ojrelids) rather than
bms_is_member(ojrelid, rinfo->required_relids). And we should do this
check only for outer joins.
* This patch calculates the outer join relids that are being formed
generally in this way:
bms_difference(joinrelids, bms_union(outerrelids, innerrelids))
Of course this can only be used after the outer join relids has been
added by add_outer_joins_to_relids(). This calculation is performed
multiple times during planning. I'm not sure if this has performance
issues. Maybe we can calculate it only once and store the result in
some place (such as in JoinPath)?
* ANTI joins are treated as outer joins but sometimes they do not have
rtindex (such as ANTI joins derived from SEMI). This would be a problem
with this new check. As an example, consider query
select * from a where not exists (select 1 from b where a.i = b.i) and
CURRENT_USER = SESSION_USER;
The pseudoconstant clause 'CURRENT_USER = SESSION_USER' is supposed to
be treated as a filter clause but the new check would treat it as a join
clause because the outer join relid set being formed is empty since the
ANTI join here does not have an rtindex. To solve this problem, this
patch manually adds a RTE for a ANTI join derived from SEMI.
Any thoughts?
Thanks
Richard
Attachments:
[application/octet-stream] v1-0001-Retiring-is_pushed_down.patch (39.2K, ../../CAMbWs49pDTmHbgjXa4xjxbA+pY=7ebZ=YUg1waQqQw8DERgQ-g@mail.gmail.com/3-v1-0001-Retiring-is_pushed_down.patch)
download | inline diff:
From 7c4f5b4625a23e69f650cfd206ad6b33adced460 Mon Sep 17 00:00:00 2001
From: pgsql-guo <[email protected]>
Date: Thu, 23 Mar 2023 11:53:40 +0800
Subject: [PATCH v1] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 13 +++--
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 18 +++++++
src/backend/optimizer/plan/initsplan.c | 62 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++--------
src/include/nodes/pathnodes.h | 53 +++++++++----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
16 files changed, 155 insertions(+), 141 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..60159e58d0 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5798,7 +5798,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6519,7 +6519,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index ef475d95a1..be80a8b327 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4738,7 +4738,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -4761,7 +4761,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -4829,8 +4829,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5195,13 +5195,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 7fa502d6e2..96a22f1389 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2005,7 +2004,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2033,7 +2031,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2061,7 +2058,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index f047ad9ba4..01f2e4fbb2 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -83,6 +83,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -149,6 +150,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -213,6 +217,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -222,7 +227,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2090,7 +2095,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2322,6 +2328,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2339,7 +2346,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 015a0b3cbe..b283d13bf0 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -895,6 +896,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -917,7 +922,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -931,12 +936,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -948,7 +953,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -984,7 +989,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1007,7 +1012,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1022,12 +1027,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1424,6 +1429,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1438,7 +1444,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 5f3cce873a..4c8c5d672b 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -284,7 +284,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -484,7 +486,9 @@ remove_rel_from_query(PlannerInfo *root, int relid, SpecialJoinInfo *sjinfo)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1286,6 +1290,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1303,7 +1310,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..13ab1d3eeb 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4347,8 +4347,14 @@ create_nestloop_plan(PlannerInfo *root,
/* Any pseudoconstant clauses are ignored here */
if (IS_OUTER_JOIN(best_path->jpath.jointype))
{
+ Relids ojrelids =
+ CALC_OUTER_JOIN_RELIDS(best_path->jpath.path.parent->relids,
+ best_path->jpath.outerjoinpath->parent->relids,
+ best_path->jpath.innerjoinpath->parent->relids);
+
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4435,8 +4441,14 @@ create_mergejoin_plan(PlannerInfo *root,
/* Any pseudoconstant clauses are ignored here */
if (IS_OUTER_JOIN(best_path->jpath.jointype))
{
+ Relids ojrelids =
+ CALC_OUTER_JOIN_RELIDS(best_path->jpath.path.parent->relids,
+ best_path->jpath.outerjoinpath->parent->relids,
+ best_path->jpath.innerjoinpath->parent->relids);
+
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4737,8 +4749,14 @@ create_hashjoin_plan(PlannerInfo *root,
/* Any pseudoconstant clauses are ignored here */
if (IS_OUTER_JOIN(best_path->jpath.jointype))
{
+ Relids ojrelids =
+ CALC_OUTER_JOIN_RELIDS(best_path->jpath.path.parent->relids,
+ best_path->jpath.outerjoinpath->parent->relids,
+ best_path->jpath.innerjoinpath->parent->relids);
+
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index b31d892121..c7e0369905 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -963,19 +963,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2209,7 +2206,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2319,37 +2315,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2373,7 +2341,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2390,12 +2357,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2420,7 +2381,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2792,7 +2752,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -2886,7 +2845,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 7a9fe88fec..f63afec871 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1519,7 +1519,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..46ce5692cf 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -893,7 +893,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
childquals = lappend(childquals,
make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -931,7 +930,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 6ef9d14b90..bc0ce171cc 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -265,7 +265,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 76dad17e33..9542954f0c 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -57,6 +57,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1286,9 +1287,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1328,6 +1329,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1349,11 +1351,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1364,7 +1370,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2059,6 +2069,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2079,7 +2092,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index d6d26a2b51..75945c3ab9 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c17b53f7ad..d1dc27be6f 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2408,27 +2408,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2515,9 +2504,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2661,16 +2647,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3170,6 +3160,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3179,6 +3171,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 6cf49705d3..11bdd120c2 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index e140e619ac..04658f0067 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.32.0
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Retiring is_pushed_down
@ 2023-07-27 02:55 Richard Guo <[email protected]>
parent: Richard Guo <[email protected]>
1 sibling, 1 reply; 9+ messages in thread
From: Richard Guo @ 2023-07-27 02:55 UTC (permalink / raw)
To: pgsql-hackers
On Tue, Jul 25, 2023 at 3:39 PM Richard Guo <[email protected]> wrote:
> * This patch calculates the outer join relids that are being formed
> generally in this way:
>
> bms_difference(joinrelids, bms_union(outerrelids, innerrelids))
>
> Of course this can only be used after the outer join relids has been
> added by add_outer_joins_to_relids(). This calculation is performed
> multiple times during planning. I'm not sure if this has performance
> issues. Maybe we can calculate it only once and store the result in
> some place (such as in JoinPath)?
>
In the v2 patch, I added a member in JoinPath to store the relid set of
any outer joins that will be calculated at this join, and this would
avoid repeating this calculation when creating nestloop/merge/hash join
plan nodes. Also fixed a comment in v2.
Thanks
Richard
Attachments:
[application/octet-stream] v2-0001-Retiring-is_pushed_down.patch (40.4K, ../../CAMbWs4-BGokURcVNDmEpQ8OG9Ur_2MP_+FVWRWb9OsC_DvX=4g@mail.gmail.com/3-v2-0001-Retiring-is_pushed_down.patch)
download | inline diff:
From 6aaa608a748b9c396ac13dae77125230a5b3c367 Mon Sep 17 00:00:00 2001
From: Richard Guo <[email protected]>
Date: Wed, 26 Jul 2023 16:16:49 +0800
Subject: [PATCH v2] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 15 +++---
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 3 ++
src/backend/optimizer/plan/initsplan.c | 62 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/pathnode.c | 3 ++
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++--------
src/include/nodes/pathnodes.h | 57 ++++++++++-----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
17 files changed, 148 insertions(+), 142 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index c5cada55fb..60159e58d0 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5798,7 +5798,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6519,7 +6519,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index ef475d95a1..a446cb265a 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4728,7 +4728,7 @@ get_restriction_qual_cost(PlannerInfo *root, RelOptInfo *baserel,
* sjinfo: SpecialJoinInfo relevant to this join
* restrictlist: join quals
* Output parameters:
- * *semifactors is filled in (see pathnodes.h for field definitions)
+ * extra->semifactors is filled in (see pathnodes.h for field definitions)
*/
void
compute_semi_anti_join_factors(PlannerInfo *root,
@@ -4738,7 +4738,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -4761,7 +4761,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -4829,8 +4829,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5195,13 +5195,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 7fa502d6e2..96a22f1389 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2005,7 +2004,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2033,7 +2031,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2061,7 +2058,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index f047ad9ba4..01f2e4fbb2 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -83,6 +83,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -149,6 +150,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -213,6 +217,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -222,7 +227,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2090,7 +2095,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2322,6 +2328,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2339,7 +2346,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 015a0b3cbe..b283d13bf0 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -895,6 +896,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -917,7 +922,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -931,12 +936,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -948,7 +953,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -984,7 +989,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1007,7 +1012,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1022,12 +1027,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1424,6 +1429,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1438,7 +1444,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 5f3cce873a..4c8c5d672b 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -284,7 +284,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -484,7 +486,9 @@ remove_rel_from_query(PlannerInfo *root, int relid, SpecialJoinInfo *sjinfo)
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1286,6 +1290,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1303,7 +1310,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index af48109058..3eeea338fe 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4349,6 +4349,7 @@ create_nestloop_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4437,6 +4438,7 @@ create_mergejoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4739,6 +4741,7 @@ create_hashjoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index b31d892121..c7e0369905 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -963,19 +963,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2209,7 +2206,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2319,37 +2315,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2373,7 +2341,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2390,12 +2357,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2420,7 +2381,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2792,7 +2752,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -2886,7 +2845,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 7a9fe88fec..f63afec871 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1519,7 +1519,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 94de855a22..46ce5692cf 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -893,7 +893,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
childquals = lappend(childquals,
make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -931,7 +930,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 6ef9d14b90..bc0ce171cc 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -265,7 +265,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index f123fcb41e..3ade50c0c5 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2477,6 +2477,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
final_cost_nestloop(root, pathnode, workspace, extra);
@@ -2541,6 +2542,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_mergeclauses = mergeclauses;
pathnode->outersortkeys = outersortkeys;
pathnode->innersortkeys = innersortkeys;
@@ -2618,6 +2620,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_hashclauses = hashclauses;
/* final_cost_hashjoin will fill in pathnode->num_batches */
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 76dad17e33..9542954f0c 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -57,6 +57,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1286,9 +1287,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1328,6 +1329,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1349,11 +1351,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1364,7 +1370,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2059,6 +2069,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2079,7 +2092,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index d6d26a2b51..75945c3ab9 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c17b53f7ad..c08f9551c3 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2036,6 +2036,10 @@ typedef struct JoinPath
* joinrestrictinfo is needed in JoinPath, and can't be merged into the
* parent RelOptInfo.
*/
+
+ Relids ojrelids; /* the relid set of any outer joins that will be
+ * calculated at this join for outer join, or NULL
+ * for inner join */
} JoinPath;
/*
@@ -2408,27 +2412,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2515,9 +2508,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2661,16 +2651,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3170,6 +3164,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3179,6 +3175,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index 6cf49705d3..11bdd120c2 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index e140e619ac..04658f0067 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.31.0
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Retiring is_pushed_down
@ 2024-01-21 12:37 vignesh C <[email protected]>
parent: Richard Guo <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: vignesh C @ 2024-01-21 12:37 UTC (permalink / raw)
To: Richard Guo <[email protected]>; +Cc: pgsql-hackers
On Thu, 27 Jul 2023 at 08:25, Richard Guo <[email protected]> wrote:
>
>
> On Tue, Jul 25, 2023 at 3:39 PM Richard Guo <[email protected]> wrote:
>>
>> * This patch calculates the outer join relids that are being formed
>> generally in this way:
>>
>> bms_difference(joinrelids, bms_union(outerrelids, innerrelids))
>>
>> Of course this can only be used after the outer join relids has been
>> added by add_outer_joins_to_relids(). This calculation is performed
>> multiple times during planning. I'm not sure if this has performance
>> issues. Maybe we can calculate it only once and store the result in
>> some place (such as in JoinPath)?
>
>
> In the v2 patch, I added a member in JoinPath to store the relid set of
> any outer joins that will be calculated at this join, and this would
> avoid repeating this calculation when creating nestloop/merge/hash join
> plan nodes. Also fixed a comment in v2.
I'm seeing that there has been no activity in this thread for nearly 6
months, I'm planning to close this in the current commitfest unless
someone is planning to take it forward. It can be opened again when
there is more interest.
Regards,
Vignesh
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Retiring is_pushed_down
@ 2024-01-29 11:50 Richard Guo <[email protected]>
parent: vignesh C <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Richard Guo @ 2024-01-29 11:50 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: pgsql-hackers
On Sun, Jan 21, 2024 at 8:37 PM vignesh C <[email protected]> wrote:
> I'm seeing that there has been no activity in this thread for nearly 6
> months, I'm planning to close this in the current commitfest unless
> someone is planning to take it forward. It can be opened again when
> there is more interest.
I'm planning to take it forward. The v2 patch does not compile any
more. Attached is an updated patch that is rebased over master and
fixes the compiling issues. Nothing else has changed.
Thanks
Richard
Attachments:
[application/octet-stream] v3-0001-Retiring-is_pushed_down.patch (41.3K, ../../CAMbWs4_SHxujmEkX4eXxVJE1bvTYSaJnb7Wj_w84ZuSSuO58Ww@mail.gmail.com/3-v3-0001-Retiring-is_pushed_down.patch)
download | inline diff:
From 642b89bdbca7cf1fbc4f54363f9ef8af72bced80 Mon Sep 17 00:00:00 2001
From: Richard Guo <[email protected]>
Date: Wed, 26 Jul 2023 16:16:49 +0800
Subject: [PATCH v3] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 15 +++---
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 3 ++
src/backend/optimizer/plan/initsplan.c | 63 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/joininfo.c | 1 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/pathnode.c | 3 ++
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++-------
src/include/nodes/pathnodes.h | 57 ++++++++++----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
18 files changed, 148 insertions(+), 144 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 142dcfc995..3fafc2760e 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5847,7 +5847,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6609,7 +6609,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 8b76e98529..32212e9e77 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4998,7 +4998,7 @@ get_restriction_qual_cost(PlannerInfo *root, RelOptInfo *baserel,
* sjinfo: SpecialJoinInfo relevant to this join
* restrictlist: join quals
* Output parameters:
- * *semifactors is filled in (see pathnodes.h for field definitions)
+ * extra->semifactors is filled in (see pathnodes.h for field definitions)
*/
void
compute_semi_anti_join_factors(PlannerInfo *root,
@@ -5008,7 +5008,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -5031,7 +5031,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -5099,8 +5099,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5465,13 +5465,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 4bd60a09c6..74fe422f49 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2016,7 +2015,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2044,7 +2042,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2072,7 +2069,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 6aca66f196..5cdee43d90 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -83,6 +83,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -149,6 +150,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -213,6 +217,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -222,7 +227,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2129,7 +2134,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2361,6 +2367,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2378,7 +2385,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 4750579b0a..5cc277e190 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -876,6 +877,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -898,7 +903,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -912,12 +917,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -929,7 +934,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -965,7 +970,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -988,7 +993,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1003,12 +1008,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1405,6 +1410,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1419,7 +1425,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 7dcb74572a..2cd44539a0 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -312,7 +312,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -542,7 +544,9 @@ remove_leftjoinrel_from_query(PlannerInfo *root, int relid,
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1393,6 +1397,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1410,7 +1417,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 610f4a56d6..4510325739 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4377,6 +4377,7 @@ create_nestloop_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4465,6 +4466,7 @@ create_mergejoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4767,6 +4769,7 @@ create_hashjoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index d4a9d77d7f..d6ccefdeff 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -963,19 +963,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2209,7 +2206,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2319,37 +2315,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2373,7 +2341,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2390,12 +2357,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2420,7 +2381,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2648,7 +2608,6 @@ add_base_clause_to_rel(PlannerInfo *root, Index relid,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2973,7 +2932,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -3067,7 +3025,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3115d79ad9..433f7ecf6f 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1519,7 +1519,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 5c7acf8a90..9e718f9360 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -889,7 +889,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
childquals = lappend(childquals,
make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -927,7 +926,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index 5fb0c17630..173aff761b 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -115,7 +115,6 @@ add_join_clause_to_rels(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 6a5234e2ce..c2e64e6cfa 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -265,7 +265,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 8dbf790e89..17b6675378 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2505,6 +2505,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
final_cost_nestloop(root, pathnode, workspace, extra);
@@ -2569,6 +2570,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_mergeclauses = mergeclauses;
pathnode->outersortkeys = outersortkeys;
pathnode->innersortkeys = innersortkeys;
@@ -2646,6 +2648,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_hashclauses = hashclauses;
/* final_cost_hashjoin will fill in pathnode->num_batches */
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index e5f4062bfb..72562ba767 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -58,6 +58,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1304,9 +1305,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1346,6 +1347,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1367,11 +1369,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1382,7 +1388,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2077,6 +2087,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2097,7 +2110,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 0b406e9334..5d05287bf3 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 534692bee1..de1122d6e0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2061,6 +2061,10 @@ typedef struct JoinPath
* joinrestrictinfo is needed in JoinPath, and can't be merged into the
* parent RelOptInfo.
*/
+
+ Relids ojrelids; /* the relid set of any outer joins that will be
+ * calculated at this join for outer join, or NULL
+ * for inner join */
} JoinPath;
/*
@@ -2433,27 +2437,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2540,9 +2533,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2689,16 +2679,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3198,6 +3192,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3207,6 +3203,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index b1c51a4e70..6b83499c62 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 1b42c832c5..295ce764d3 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.31.0
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Retiring is_pushed_down
@ 2024-04-18 07:34 Richard Guo <[email protected]>
parent: Richard Guo <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Richard Guo @ 2024-04-18 07:34 UTC (permalink / raw)
To: pgsql-hackers
Here is another rebase over 3af7040985. Nothing else has changed.
Thanks
Richard
Attachments:
[application/octet-stream] v4-0001-Retiring-is_pushed_down.patch (41.3K, ../../CAMbWs48awvB3Z5GKETGP2DkwGY7j6fx=192abvQ-QxaMW3fdEQ@mail.gmail.com/3-v4-0001-Retiring-is_pushed_down.patch)
download | inline diff:
From 3f2682d8c7b8116e5853e121f843d4c3ad8eacbe Mon Sep 17 00:00:00 2001
From: Richard Guo <[email protected]>
Date: Wed, 26 Jul 2023 16:16:49 +0800
Subject: [PATCH v4] Retiring is_pushed_down
When forming an outer join's joinrel, we have the is_pushed_down flag in
RestrictInfo nodes to distinguish those quals that are in that join's
JOIN/ON condition from those that were pushed down to the joinrel and
thus act as filter quals. Since now we have the outer-join-aware-Var
infrastructure, we can check to see whether a qual clause's
required_relids reference the outer join(s) being formed, in order to
tell if it's a join or filter clause. This seems like a more principled
way.
This patch is an attempt to retire the is_pushed_down flag.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/optimizer/path/costsize.c | 15 +++---
src/backend/optimizer/path/equivclass.c | 4 --
src/backend/optimizer/path/joinpath.c | 14 +++--
src/backend/optimizer/path/joinrels.c | 25 +++++----
src/backend/optimizer/plan/analyzejoins.c | 13 +++--
src/backend/optimizer/plan/createplan.c | 3 ++
src/backend/optimizer/plan/initsplan.c | 63 ++++-------------------
src/backend/optimizer/plan/subselect.c | 28 +++++++++-
src/backend/optimizer/util/inherit.c | 2 -
src/backend/optimizer/util/joininfo.c | 1 -
src/backend/optimizer/util/orclauses.c | 1 -
src/backend/optimizer/util/pathnode.c | 3 ++
src/backend/optimizer/util/relnode.c | 23 +++++++--
src/backend/optimizer/util/restrictinfo.c | 31 ++++-------
src/include/nodes/pathnodes.h | 57 ++++++++++----------
src/include/optimizer/cost.h | 2 +-
src/include/optimizer/restrictinfo.h | 4 +-
18 files changed, 148 insertions(+), 144 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 4053cd641c..635b5c7c95 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5847,7 +5847,7 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
rinfo->clause);
if (IS_OUTER_JOIN(jointype) &&
- !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ !RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
{
if (!is_remote_clause)
return false;
@@ -6609,7 +6609,6 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Assert(!IsA(expr, RestrictInfo));
rinfo = make_restrictinfo(root,
expr,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index ee23ed7835..625e500feb 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4998,7 +4998,7 @@ get_restriction_qual_cost(PlannerInfo *root, RelOptInfo *baserel,
* sjinfo: SpecialJoinInfo relevant to this join
* restrictlist: join quals
* Output parameters:
- * *semifactors is filled in (see pathnodes.h for field definitions)
+ * extra->semifactors is filled in (see pathnodes.h for field definitions)
*/
void
compute_semi_anti_join_factors(PlannerInfo *root,
@@ -5008,7 +5008,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors)
+ JoinPathExtraData *extra)
{
Selectivity jselec;
Selectivity nselec;
@@ -5031,7 +5031,7 @@ compute_semi_anti_join_factors(PlannerInfo *root,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (!RINFO_IS_PUSHED_DOWN(rinfo, extra->ojrelids, joinrel->relids))
joinquals = lappend(joinquals, rinfo);
}
}
@@ -5083,8 +5083,8 @@ compute_semi_anti_join_factors(PlannerInfo *root,
else
avgmatch = 1.0;
- semifactors->outer_match_frac = jselec;
- semifactors->match_count = avgmatch;
+ extra->semifactors.outer_match_frac = jselec;
+ extra->semifactors.match_count = avgmatch;
}
/*
@@ -5434,13 +5434,16 @@ calc_joinrel_size_estimate(PlannerInfo *root,
List *joinquals = NIL;
List *pushedquals = NIL;
ListCell *l;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outer_rel->relids,
+ inner_rel->relids);
/* Grovel through the clauses to separate into two lists */
foreach(l, restrictlist)
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
pushedquals = lappend(pushedquals, rinfo);
else
joinquals = lappend(joinquals, rinfo);
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 1d6bedb399..bb51bd97aa 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -196,7 +196,6 @@ process_equivalence(PlannerInfo *root,
*p_restrictinfo =
make_restrictinfo(root,
(Expr *) ntest,
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2031,7 +2030,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2059,7 +2057,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
@@ -2087,7 +2084,6 @@ reconsider_outer_join_clauses(PlannerInfo *root)
/* throw back a dummy replacement clause (see notes above) */
rinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(true, false),
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 5be8da9e09..9809260d18 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -84,6 +84,7 @@ static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed);
static void generate_mergejoin_paths(PlannerInfo *root,
RelOptInfo *joinrel,
@@ -150,6 +151,9 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ outerrel->relids,
+ innerrel->relids);
/*
* See if the inner relation is provably unique for this outer rel.
@@ -214,6 +218,7 @@ add_paths_to_joinrel(PlannerInfo *root,
innerrel,
restrictlist,
jointype,
+ &extra,
&mergejoin_allowed);
/*
@@ -223,7 +228,7 @@ add_paths_to_joinrel(PlannerInfo *root,
if (jointype == JOIN_SEMI || jointype == JOIN_ANTI || extra.inner_unique)
compute_semi_anti_join_factors(root, joinrel, outerrel, innerrel,
jointype, sjinfo, restrictlist,
- &extra.semifactors);
+ &extra);
/*
* Decide whether it's sensible to generate parameterized paths for this
@@ -2118,7 +2123,8 @@ hash_inner_and_outer(PlannerInfo *root,
* If processing an outer join, only use its own join clauses for
* hashing. For inner joins we need not be so picky.
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
if (!restrictinfo->can_join ||
@@ -2350,6 +2356,7 @@ select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *innerrel,
List *restrictlist,
JoinType jointype,
+ JoinPathExtraData *extra,
bool *mergejoin_allowed)
{
List *result_list = NIL;
@@ -2367,7 +2374,8 @@ select_mergejoin_clauses(PlannerInfo *root,
* we don't set have_nonmergeable_joinclause here because pushed-down
* clauses will become otherquals not joinquals.)
*/
- if (isouterjoin && RINFO_IS_PUSHED_DOWN(restrictinfo, joinrel->relids))
+ if (isouterjoin &&
+ RINFO_IS_PUSHED_DOWN(restrictinfo, extra->ojrelids, joinrel->relids))
continue;
/* Check that clause is a mergeable operator clause */
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index f3a9412d18..b27ad767c9 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -34,6 +34,7 @@ static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
@@ -894,6 +895,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist)
{
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -916,7 +921,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
{
case JOIN_INNER:
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -930,12 +935,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -947,7 +952,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
@@ -983,7 +988,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
bms_is_subset(sjinfo->min_righthand, rel2->relids))
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1006,7 +1011,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
sjinfo) != NULL)
{
if (is_dummy_rel(rel1) || is_dummy_rel(rel2) ||
- restriction_is_constant_false(restrictlist, joinrel, false))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, false))
{
mark_dummy_rel(joinrel);
break;
@@ -1021,12 +1026,12 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
case JOIN_ANTI:
if (is_dummy_rel(rel1) ||
- restriction_is_constant_false(restrictlist, joinrel, true))
+ restriction_is_constant_false(restrictlist, joinrel, ojrelids, true))
{
mark_dummy_rel(joinrel);
break;
}
- if (restriction_is_constant_false(restrictlist, joinrel, false) &&
+ if (restriction_is_constant_false(restrictlist, joinrel, ojrelids, false) &&
bms_is_subset(rel2->relids, sjinfo->syn_righthand))
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
@@ -1423,6 +1428,7 @@ mark_dummy_rel(RelOptInfo *rel)
static bool
restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
+ Relids ojrelids,
bool only_pushed_down)
{
ListCell *lc;
@@ -1437,7 +1443,8 @@ restriction_is_constant_false(List *restrictlist,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
- if (only_pushed_down && !RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ if (only_pushed_down &&
+ !RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
if (rinfo->clause && IsA(rinfo->clause, Const))
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c
index 506fccd20c..91831ea3ae 100644
--- a/src/backend/optimizer/plan/analyzejoins.c
+++ b/src/backend/optimizer/plan/analyzejoins.c
@@ -299,7 +299,9 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo)
* above the outer join, even if it references no other rels (it might
* be from WHERE, for example).
*/
- if (RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(restrictinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ joinrelids))
continue; /* ignore; not useful here */
/* Ignore if it's not a mergejoinable clause */
@@ -530,7 +532,9 @@ remove_leftjoinrel_from_query(PlannerInfo *root, int relid,
remove_join_clause_from_rels(root, rinfo, rinfo->required_relids);
- if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute))
+ if (RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_make_singleton(sjinfo->ojrelid),
+ join_plus_commute))
{
/*
* There might be references to relid or ojrelid in the
@@ -1381,6 +1385,9 @@ is_innerrel_unique_for(PlannerInfo *root,
{
List *clause_list = NIL;
ListCell *lc;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrelids,
+ outerrelids,
+ innerrel->relids);
/*
* Search for mergejoinable clauses that constrain the inner rel against
@@ -1398,7 +1405,7 @@ is_innerrel_unique_for(PlannerInfo *root,
* join, we can't use it.
*/
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(restrictinfo, joinrelids))
+ RINFO_IS_PUSHED_DOWN(restrictinfo, ojrelids, joinrelids))
continue;
/* Ignore if it's not a mergejoinable clause */
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 3b77886567..4de49cacdc 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -4396,6 +4396,7 @@ create_nestloop_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinrestrictclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4484,6 +4485,7 @@ create_mergejoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
@@ -4786,6 +4788,7 @@ create_hashjoin_plan(PlannerInfo *root,
{
extract_actual_join_clauses(joinclauses,
best_path->jpath.path.parent->relids,
+ best_path->jpath.ojrelids,
&joinclauses, &otherclauses);
}
else
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index e2c68fe6f9..d4353ff56a 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -962,19 +962,16 @@ deconstruct_recurse(PlannerInfo *root, Node *jtnode,
child_domain->jd_relids);
jtitem->qualscope = bms_union(left_item->qualscope,
right_item->qualscope);
- /* caution: ANTI join derived from SEMI will lack rtindex */
- if (j->rtindex != 0)
- {
- parent_domain->jd_relids =
- bms_add_member(parent_domain->jd_relids,
- j->rtindex);
- jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ Assert(j->rtindex != 0);
+ parent_domain->jd_relids =
+ bms_add_member(parent_domain->jd_relids,
+ j->rtindex);
+ jtitem->qualscope = bms_add_member(jtitem->qualscope,
+ j->rtindex);
+ root->outer_join_rels = bms_add_member(root->outer_join_rels,
j->rtindex);
- root->outer_join_rels = bms_add_member(root->outer_join_rels,
- j->rtindex);
- mark_rels_nulled_by_join(root, j->rtindex,
- right_item->qualscope);
- }
+ mark_rels_nulled_by_join(root, j->rtindex,
+ right_item->qualscope);
jtitem->inner_join_rels = bms_union(left_item->inner_join_rels,
right_item->inner_join_rels);
jtitem->left_rels = left_item->qualscope;
@@ -2208,7 +2205,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
List **postponed_oj_qual_list)
{
Relids relids;
- bool is_pushed_down;
bool pseudoconstant = false;
bool maybe_equivalence;
bool maybe_outer_join;
@@ -2318,37 +2314,9 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
}
- /*----------
+ /*
* Check to see if clause application must be delayed by outer-join
* considerations.
- *
- * A word about is_pushed_down: we mark the qual as "pushed down" if
- * it is (potentially) applicable at a level different from its original
- * syntactic level. This flag is used to distinguish OUTER JOIN ON quals
- * from other quals pushed down to the same joinrel. The rules are:
- * WHERE quals and INNER JOIN quals: is_pushed_down = true.
- * Non-degenerate OUTER JOIN quals: is_pushed_down = false.
- * Degenerate OUTER JOIN quals: is_pushed_down = true.
- * A "degenerate" OUTER JOIN qual is one that doesn't mention the
- * non-nullable side, and hence can be pushed down into the nullable side
- * without changing the join result. It is correct to treat it as a
- * regular filter condition at the level where it is evaluated.
- *
- * Note: it is not immediately obvious that a simple boolean is enough
- * for this: if for some reason we were to attach a degenerate qual to
- * its original join level, it would need to be treated as an outer join
- * qual there. However, this cannot happen, because all the rels the
- * clause mentions must be in the outer join's min_righthand, therefore
- * the join it needs must be formed before the outer join; and we always
- * attach quals to the lowest level where they can be evaluated. But
- * if we were ever to re-introduce a mechanism for delaying evaluation
- * of "expensive" quals, this area would need work.
- *
- * Note: generally, use of is_pushed_down has to go through the macro
- * RINFO_IS_PUSHED_DOWN, because that flag alone is not always sufficient
- * to tell whether a clause must be treated as pushed-down in context.
- * This seems like another reason why it should perhaps be rethought.
- *----------
*/
if (bms_overlap(relids, outerjoin_nonnullable))
{
@@ -2372,7 +2340,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
* deductions, if it is mergejoinable. So consider adding it to the
* lists of set-aside outer-join clauses.
*/
- is_pushed_down = false;
maybe_equivalence = false;
maybe_outer_join = true;
@@ -2389,12 +2356,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
}
else
{
- /*
- * Normal qual clause or degenerate outer-join clause. Either way, we
- * can mark it as pushed-down.
- */
- is_pushed_down = true;
-
/*
* It's possible that this is an IS NULL clause that's redundant with
* a lower antijoin; if so we can just discard it. We need not test
@@ -2419,7 +2380,6 @@ distribute_qual_to_rels(PlannerInfo *root, Node *clause,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -2665,7 +2625,6 @@ add_base_clause_to_rel(PlannerInfo *root, Index relid,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
@@ -2991,7 +2950,6 @@ process_implied_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
(Expr *) clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
pseudoconstant,
@@ -3085,7 +3043,6 @@ build_implied_join_equality(PlannerInfo *root,
*/
restrictinfo = make_restrictinfo(root,
clause,
- true, /* is_pushed_down */
false, /* !has_clone */
false, /* !is_clone */
false, /* pseudoconstant */
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index e35ebea8b4..496d362bfd 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -1515,7 +1515,33 @@ convert_EXISTS_sublink_to_join(PlannerInfo *root, SubLink *sublink,
result->join_using_alias = NULL;
result->quals = whereClause;
result->alias = NULL;
- result->rtindex = 0; /* we don't need an RTE for it */
+ result->rtindex = 0; /* we don't need an RTE for JOIN_SEMI */
+
+ /*
+ * Add a RTE for JOIN_ANTI
+ */
+ if (result->jointype == JOIN_ANTI)
+ {
+ ParseNamespaceItem *jnsitem;
+ ParseState *pstate;
+
+ /* Create a dummy ParseState for addRangeTableEntryForJoin */
+ pstate = make_parsestate(NULL);
+
+ jnsitem = addRangeTableEntryForJoin(pstate,
+ NULL,
+ NULL,
+ JOIN_ANTI,
+ 0,
+ NULL,
+ NIL,
+ NIL,
+ NULL,
+ NULL,
+ false);
+ parse->rtable = lappend(parse->rtable, jnsitem->p_rte);
+ result->rtindex = list_length(parse->rtable);
+ }
return result;
}
diff --git a/src/backend/optimizer/util/inherit.c b/src/backend/optimizer/util/inherit.c
index 4797312ae5..3050bf5e37 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -891,7 +891,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* reconstitute RestrictInfo with appropriate properties */
childrinfo = make_restrictinfo(root,
(Expr *) onecq,
- rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
@@ -938,7 +937,6 @@ apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
make_restrictinfo(root, qual,
- true,
false, false,
false,
security_level,
diff --git a/src/backend/optimizer/util/joininfo.c b/src/backend/optimizer/util/joininfo.c
index 5fb0c17630..173aff761b 100644
--- a/src/backend/optimizer/util/joininfo.c
+++ b/src/backend/optimizer/util/joininfo.c
@@ -115,7 +115,6 @@ add_join_clause_to_rels(PlannerInfo *root,
restrictinfo = make_restrictinfo(root,
(Expr *) makeBoolConst(false, false),
- restrictinfo->is_pushed_down,
restrictinfo->has_clone,
restrictinfo->is_clone,
restrictinfo->pseudoconstant,
diff --git a/src/backend/optimizer/util/orclauses.c b/src/backend/optimizer/util/orclauses.c
index 5e2bf26ec4..35500e40b9 100644
--- a/src/backend/optimizer/util/orclauses.c
+++ b/src/backend/optimizer/util/orclauses.c
@@ -264,7 +264,6 @@ consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
*/
or_rinfo = make_restrictinfo(root,
orclause,
- true,
false,
false,
false,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 3cf1dac087..0b392cba4f 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2524,6 +2524,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
final_cost_nestloop(root, pathnode, workspace, extra);
@@ -2588,6 +2589,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_mergeclauses = mergeclauses;
pathnode->outersortkeys = outersortkeys;
pathnode->innersortkeys = innersortkeys;
@@ -2665,6 +2667,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
+ pathnode->jpath.ojrelids = extra->ojrelids;
pathnode->path_hashclauses = hashclauses;
/* final_cost_hashjoin will fill in pathnode->num_batches */
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index e05b21c884..f03c8c2fa2 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -58,6 +58,7 @@ static List *subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist);
static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *joininfo_list,
@@ -1311,9 +1312,9 @@ build_joinrel_restrictlist(PlannerInfo *root,
* same clauses arriving from both input relations).
*/
result = subbuild_joinrel_restrictlist(root, joinrel, outer_rel,
- both_input_relids, NIL);
+ both_input_relids, sjinfo, NIL);
result = subbuild_joinrel_restrictlist(root, joinrel, inner_rel,
- both_input_relids, result);
+ both_input_relids, sjinfo, result);
/*
* Add on any clauses derived from EquivalenceClasses. These cannot be
@@ -1353,6 +1354,7 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *input_rel,
Relids both_input_relids,
+ SpecialJoinInfo *sjinfo,
List *new_restrictlist)
{
ListCell *l;
@@ -1374,11 +1376,15 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
*/
if (rinfo->has_clone || rinfo->is_clone)
{
- Assert(!RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids));
if (!bms_is_subset(rinfo->required_relids, both_input_relids))
continue;
if (bms_overlap(rinfo->incompatible_relids, both_input_relids))
continue;
+
+ Assert(!RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids));
}
else
{
@@ -1389,7 +1395,11 @@ subbuild_joinrel_restrictlist(PlannerInfo *root,
* (There is little point in checking incompatible_relids,
* because it'll be NULL.)
*/
- Assert(RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids) ||
+ Assert(sjinfo->ojrelid == 0 ||
+ RINFO_IS_PUSHED_DOWN(rinfo,
+ bms_difference(joinrel->relids,
+ both_input_relids),
+ joinrel->relids) ||
bms_is_subset(rinfo->required_relids,
both_input_relids));
}
@@ -2096,6 +2106,9 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
int cnt_pks;
bool pk_has_clause[PARTITION_MAX_KEYS];
bool strict_op;
+ Relids ojrelids = CALC_OUTER_JOIN_RELIDS(joinrel->relids,
+ rel1->relids,
+ rel2->relids);
/*
* This function must only be called when the joined relations have same
@@ -2116,7 +2129,7 @@ have_partkey_equi_join(PlannerInfo *root, RelOptInfo *joinrel,
/* If processing an outer join, only use its own join clauses. */
if (IS_OUTER_JOIN(jointype) &&
- RINFO_IS_PUSHED_DOWN(rinfo, joinrel->relids))
+ RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrel->relids))
continue;
/* Skip clauses which can not be used for a join. */
diff --git a/src/backend/optimizer/util/restrictinfo.c b/src/backend/optimizer/util/restrictinfo.c
index 0b406e9334..5d05287bf3 100644
--- a/src/backend/optimizer/util/restrictinfo.c
+++ b/src/backend/optimizer/util/restrictinfo.c
@@ -24,7 +24,6 @@
static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -34,7 +33,6 @@ static RestrictInfo *make_restrictinfo_internal(PlannerInfo *root,
Relids outer_relids);
static Expr *make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -49,11 +47,11 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
*
* Build a RestrictInfo node containing the given subexpression.
*
- * The is_pushed_down, has_clone, is_clone, and pseudoconstant flags for the
- * RestrictInfo must be supplied by the caller, as well as the correct values
- * for security_level, incompatible_relids, and outer_relids.
- * required_relids can be NULL, in which case it defaults to the actual clause
- * contents (i.e., clause_relids).
+ * The has_clone, is_clone, and pseudoconstant flags for the RestrictInfo
+ * must be supplied by the caller, as well as the correct values for
+ * security_level, incompatible_relids, and outer_relids. required_relids
+ * can be NULL, in which case it defaults to the actual clause contents
+ * (i.e., clause_relids).
*
* We initialize fields that depend only on the given subexpression, leaving
* others that depend on context (or may never be needed at all) to be filled
@@ -62,7 +60,6 @@ static Expr *make_sub_restrictinfos(PlannerInfo *root,
RestrictInfo *
make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -78,7 +75,6 @@ make_restrictinfo(PlannerInfo *root,
if (is_orclause(clause))
return (RestrictInfo *) make_sub_restrictinfos(root,
clause,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -93,7 +89,6 @@ make_restrictinfo(PlannerInfo *root,
return make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -112,7 +107,6 @@ static RestrictInfo *
make_restrictinfo_internal(PlannerInfo *root,
Expr *clause,
Expr *orclause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -126,7 +120,6 @@ make_restrictinfo_internal(PlannerInfo *root,
restrictinfo->clause = clause;
restrictinfo->orclause = orclause;
- restrictinfo->is_pushed_down = is_pushed_down;
restrictinfo->pseudoconstant = pseudoconstant;
restrictinfo->has_clone = has_clone;
restrictinfo->is_clone = is_clone;
@@ -259,9 +252,9 @@ make_restrictinfo_internal(PlannerInfo *root,
* implicit-AND lists at top level of RestrictInfo lists. Only ORs and
* simple clauses are valid RestrictInfos.
*
- * The same is_pushed_down, has_clone, is_clone, and pseudoconstant flag
- * values can be applied to all RestrictInfo nodes in the result. Likewise
- * for security_level, incompatible_relids, and outer_relids.
+ * The same has_clone, is_clone, and pseudoconstant flag values can be
+ * applied to all RestrictInfo nodes in the result. Likewise for
+ * security_level, incompatible_relids, and outer_relids.
*
* The given required_relids are attached to our top-level output,
* but any OR-clause constituents are allowed to default to just the
@@ -270,7 +263,6 @@ make_restrictinfo_internal(PlannerInfo *root,
static Expr *
make_sub_restrictinfos(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -288,7 +280,6 @@ make_sub_restrictinfos(PlannerInfo *root,
orlist = lappend(orlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -299,7 +290,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
make_orclause(orlist),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -317,7 +307,6 @@ make_sub_restrictinfos(PlannerInfo *root,
andlist = lappend(andlist,
make_sub_restrictinfos(root,
lfirst(temp),
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -331,7 +320,6 @@ make_sub_restrictinfos(PlannerInfo *root,
return (Expr *) make_restrictinfo_internal(root,
clause,
NULL,
- is_pushed_down,
has_clone,
is_clone,
pseudoconstant,
@@ -521,6 +509,7 @@ extract_actual_clauses(List *restrictinfo_list,
void
extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals)
{
@@ -533,7 +522,7 @@ extract_actual_join_clauses(List *restrictinfo_list,
{
RestrictInfo *rinfo = lfirst_node(RestrictInfo, l);
- if (RINFO_IS_PUSHED_DOWN(rinfo, joinrelids))
+ if (RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids))
{
if (!rinfo->pseudoconstant &&
!rinfo_is_constant_true(rinfo))
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 6c71098f2d..5778cfa345 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2070,6 +2070,10 @@ typedef struct JoinPath
* joinrestrictinfo is needed in JoinPath, and can't be merged into the
* parent RelOptInfo.
*/
+
+ Relids ojrelids; /* the relid set of any outer joins that will be
+ * calculated at this join for outer join, or NULL
+ * for inner join */
} JoinPath;
/*
@@ -2444,27 +2448,16 @@ typedef struct LimitPath
* level of the outer join, which is true except when it is a "degenerate"
* condition that references only Vars from the nullable side of the join.
*
- * RestrictInfo nodes contain a flag to indicate whether a qual has been
- * pushed down to a lower level than its original syntactic placement in the
- * join tree would suggest. If an outer join prevents us from pushing a qual
- * down to its "natural" semantic level (the level associated with just the
- * base rels used in the qual) then we mark the qual with a "required_relids"
- * value including more than just the base rels it actually uses. By
- * pretending that the qual references all the rels required to form the outer
- * join, we prevent it from being evaluated below the outer join's joinrel.
- * When we do form the outer join's joinrel, we still need to distinguish
- * those quals that are actually in that join's JOIN/ON condition from those
- * that appeared elsewhere in the tree and were pushed down to the join rel
- * because they used no other rels. That's what the is_pushed_down flag is
- * for; it tells us that a qual is not an OUTER JOIN qual for the set of base
- * rels listed in required_relids. A clause that originally came from WHERE
- * or an INNER JOIN condition will *always* have its is_pushed_down flag set.
- * It's possible for an OUTER JOIN clause to be marked is_pushed_down too,
- * if we decide that it can be pushed down into the nullable side of the join.
- * In that case it acts as a plain filter qual for wherever it gets evaluated.
- * (In short, is_pushed_down is only false for non-degenerate outer join
- * conditions. Possibly we should rename it to reflect that meaning? But
- * see also the comments for RINFO_IS_PUSHED_DOWN, below.)
+ * If an outer join prevents us from pushing a qual down to its "natural"
+ * semantic level (the level associated with just the base rels used in the
+ * qual) then we mark the qual with a "required_relids" value including more
+ * than just the base rels it actually uses. By pretending that the qual
+ * references all the rels required to form the outer join, we prevent it from
+ * being evaluated below the outer join's joinrel. When we do form the outer
+ * join's joinrel, we still need to distinguish those quals that are actually
+ * in that join's JOIN/ON condition from those that appeared elsewhere in the
+ * tree and were pushed down to the join rel because they used no other rels.
+ * See the comments for RINFO_IS_PUSHED_DOWN for how we do that.
*
* There is also an incompatible_relids field, which is a set of outer-join
* relids above which we cannot evaluate the clause (because they might null
@@ -2551,9 +2544,6 @@ typedef struct RestrictInfo
/* the represented clause of WHERE or JOIN */
Expr *clause;
- /* true if clause was pushed down in level */
- bool is_pushed_down;
-
/* see comment above */
bool can_join pg_node_attr(equal_ignore);
@@ -2700,16 +2690,20 @@ typedef struct RestrictInfo
* This macro embodies the correct way to test whether a RestrictInfo is
* "pushed down" to a given outer join, that is, should be treated as a filter
* clause rather than a join clause at that outer join. This is certainly so
- * if is_pushed_down is true; but examining that is not sufficient anymore,
- * because outer-join clauses will get pushed down to lower outer joins when
- * we generate a path for the lower outer join that is parameterized by the
- * LHS of the upper one. We can detect such a clause by noting that its
+ * if the outer join clause references that outer join in any varnullingrels or
+ * phnullingrels set; but examining that is not sufficient anymore, because
+ * outer-join clauses will get pushed down to lower outer joins when we
+ * generate a path for the lower outer join that is parameterized by the LHS of
+ * the upper one. We can detect such a clause by noting that its
* required_relids exceed the scope of the join.
*/
-#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
- ((rinfo)->is_pushed_down || \
+#define RINFO_IS_PUSHED_DOWN(rinfo, ojrelids, joinrelids) \
+ (bms_overlap((rinfo)->required_relids, (ojrelids)) || \
!bms_is_subset((rinfo)->required_relids, joinrelids))
+#define CALC_OUTER_JOIN_RELIDS(joinrelids, outerrelids, innerrelids) \
+ (bms_difference((joinrelids), bms_union((outerrelids), (innerrelids))))
+
/*
* Since mergejoinscansel() is a relatively expensive function, and would
* otherwise be invoked many times while planning a large join tree,
@@ -3212,6 +3206,8 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * ojrelids is the relid set of any outer joins that will be calculated at this
+ * join, or NULL for inner join
*/
typedef struct JoinPathExtraData
{
@@ -3221,6 +3217,7 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ Relids ojrelids;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index b1c51a4e70..6b83499c62 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -183,7 +183,7 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo,
List *restrictlist,
- SemiAntiJoinFactors *semifactors);
+ JoinPathExtraData *extra);
extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
diff --git a/src/include/optimizer/restrictinfo.h b/src/include/optimizer/restrictinfo.h
index 1b42c832c5..295ce764d3 100644
--- a/src/include/optimizer/restrictinfo.h
+++ b/src/include/optimizer/restrictinfo.h
@@ -19,12 +19,11 @@
/* Convenience macro for the common case of a valid-everywhere qual */
#define make_simple_restrictinfo(root, clause) \
- make_restrictinfo(root, clause, true, false, false, false, 0, \
+ make_restrictinfo(root, clause, false, false, false, 0, \
NULL, NULL, NULL)
extern RestrictInfo *make_restrictinfo(PlannerInfo *root,
Expr *clause,
- bool is_pushed_down,
bool has_clone,
bool is_clone,
bool pseudoconstant,
@@ -41,6 +40,7 @@ extern List *extract_actual_clauses(List *restrictinfo_list,
bool pseudoconstant);
extern void extract_actual_join_clauses(List *restrictinfo_list,
Relids joinrelids,
+ Relids ojrelids,
List **joinquals,
List **otherquals);
extern bool join_clause_is_movable_to(RestrictInfo *rinfo, RelOptInfo *baserel);
--
2.31.0
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Retiring is_pushed_down
@ 2024-09-10 12:22 Rafia Sabih <[email protected]>
parent: Richard Guo <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Rafia Sabih @ 2024-09-10 12:22 UTC (permalink / raw)
To: Richard Guo <[email protected]>; +Cc: pgsql-hackers
I see following issue with the latest patch,
relnode.c:2122:32: error: use of undeclared identifier 'ojrelids'
RINFO_IS_PUSHED_DOWN(rinfo, ojrelids,
joinrel->relids))
On Thu, 18 Apr 2024 at 09:34, Richard Guo <[email protected]> wrote:
> Here is another rebase over 3af7040985. Nothing else has changed.
>
> Thanks
> Richard
>
--
Regards,
Rafia Sabih
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Retiring is_pushed_down
@ 2024-09-26 20:06 Tom Lane <[email protected]>
parent: Richard Guo <[email protected]>
1 sibling, 1 reply; 9+ messages in thread
From: Tom Lane @ 2024-09-26 20:06 UTC (permalink / raw)
To: Richard Guo <[email protected]>; +Cc: pgsql-hackers
Richard Guo <[email protected]> writes:
> When forming an outer join's joinrel, we have the is_pushed_down flag in
> RestrictInfo nodes to distinguish those quals that are in that join's
> JOIN/ON condition from those that were pushed down to the joinrel and
> thus act as filter quals. Since now we have the outer-join-aware-Var
> infrastructure, I think we can check to see whether a qual clause's
> required_relids reference the outer join(s) being formed, in order to
> tell if it's a join or filter clause. This seems like a more principled
> way. (Interesting that optimizer/README actually describes this way in
> section 'Relation Identification and Qual Clause Placement'.)
Sorry for being so slow to look at this patch. The idea you're
following is one that I spent a fair amount of time on while working
on what became 2489d76c4 ("Make Vars be outer-join-aware"). I failed
to make it work though. Digging in my notes from the time:
-----
How about is_pushed_down?
Would really like to get rid of that, because it's ugly/sloppily defined,
and it's hard to determine the correct value for EquivClass-generated
clauses once we allow derivations from OJ clauses. However, my original
idea of checking for join's ojrelid present in clause's required_relids
has issues:
* fails if clause is not pushed as far down as it can possibly be (and
lateral refs mean that that's hard to do sometimes)
* getting the join's ojrelid to everywhere we need to check this is messy.
I'd tolerate the mess if it worked nicely, but ...
-----
So I'm worried that the point about lateral refs is still a problem
in your version. To be clear, the hazard is that if a WHERE clause
ends up getting placed at an outer join that's higher than any of
the OJs specifically listed in its required_relids, we'd misinterpret
it as being a join clause for that OJ although it should be a filter
clause.
The other thing I find in my old notes is speculation that we could
use the concept of JoinDomains to replace is_pushed_down. That is,
we'd have to label every RestrictInfo with the JoinDomain of its
syntactic source location, and then we could tell if the RI was
"pushed down" relative to a particular join by seeing if the JD was
above or below that join. This ought to be impervious to
not-pushed-down-all-the-way problems. The thing I'd not figured
out was how to make this work with quals of full joins: they don't
belong to either the upper JoinDomain or either of the lower ones.
We could possibly fix this by giving a full join its very own
JoinDomain that is understood to be a parent of both lower domains,
but I ran out of energy to pursue that.
If we went this route, we'd basically be replacing the is_pushed_down
field with a JoinDomain field, which is surely not simpler. But it
seems more crisply defined and perhaps more amenable to my long-term
desire to be able to use the EquivalenceClass machinery with outer
join clauses. (The idea being that an EC would describe equalities
that hold within a JoinDomain, but not necessarily elsewhere.)
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Retiring is_pushed_down
@ 2025-01-14 10:41 Richard Guo <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Richard Guo @ 2025-01-14 10:41 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: pgsql-hackers
On Fri, Sep 27, 2024 at 5:06 AM Tom Lane <[email protected]> wrote:
> So I'm worried that the point about lateral refs is still a problem
> in your version. To be clear, the hazard is that if a WHERE clause
> ends up getting placed at an outer join that's higher than any of
> the OJs specifically listed in its required_relids, we'd misinterpret
> it as being a join clause for that OJ although it should be a filter
> clause.
Sorry it took me so long to get back to this thread.
I don't quite understand how this could happen. If a WHERE clause is
placed on an outer join but does not include the outer join's ojrelid
in its required_relids, then it must only refer to the non-nullable
side. In that case, we should be able to push this clause down to the
non-nullable side of the outer join.
Perhaps this issue could occur with a lateral join, but I wasn't able
to construct such a query. I wonder if you happen to have an example
on hand.
> If we went this route, we'd basically be replacing the is_pushed_down
> field with a JoinDomain field, which is surely not simpler. But it
> seems more crisply defined and perhaps more amenable to my long-term
> desire to be able to use the EquivalenceClass machinery with outer
> join clauses. (The idea being that an EC would describe equalities
> that hold within a JoinDomain, but not necessarily elsewhere.)
Exactly. At first, I thought that with JoinDomain, we could avoid
checking if a clause's required_relids exceeds the scope of the join
in RINFO_IS_PUSHED_DOWN, so RINFO_IS_PUSHED_DOWN can be reduced to:
#define RINFO_IS_PUSHED_DOWN(rinfo, joinrelids) \
(bms_is_subset(joinrelids, (rinfo)->jdomain->jd_relids))
However, if the outer join was commuted with another one according to
Identity 3, simply checking whether the JoinDomain is above or below
the outer join is not sufficient. We'll still need to check if the
clause's required_relids exceeds the scope of the join, as we do
currently.
But I agree that changing to use JoinDomain is more amenable to the
goal of using ECs with outer join clauses.
Thanks
Richard
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2025-01-14 10:41 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-19 06:11 [PATCH v52 5/7] Doc part of shared-memory based stats collector. Kyotaro Horiguchi <[email protected]>
2023-07-25 07:39 Retiring is_pushed_down Richard Guo <[email protected]>
2023-07-27 02:55 ` Re: Retiring is_pushed_down Richard Guo <[email protected]>
2024-01-21 12:37 ` Re: Retiring is_pushed_down vignesh C <[email protected]>
2024-01-29 11:50 ` Re: Retiring is_pushed_down Richard Guo <[email protected]>
2024-04-18 07:34 ` Re: Retiring is_pushed_down Richard Guo <[email protected]>
2024-09-10 12:22 ` Re: Retiring is_pushed_down Rafia Sabih <[email protected]>
2024-09-26 20:06 ` Re: Retiring is_pushed_down Tom Lane <[email protected]>
2025-01-14 10:41 ` Re: Retiring is_pushed_down Richard Guo <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox