pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
22+ messages / 7 participants
[nested] [flat]
Commits from this thread:
2025-02-19 c39392ebae 727bc6ac33 Fix freeing a child join's SpecialJoinInfo
* BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
@ 2025-02-12 07:01 PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
0 siblings, 1 reply; 22+ messages in thread
From: PG Bug reporting form @ 2025-02-12 07:01 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: m_lingbin@126.com
The following bug has been logged on the website:
Bug reference: 18806
Logged by: lingbin meng
Email address: m_lingbin@126.com
PostgreSQL version: 17.2
Operating system: CentOS Linux release 7.9.2009 (Core)
Description:
After upgrading PG17.2, testing found that when enable_rartitionwise_join is
set to ON, when executing a query, if the main query and subquery have the
same table, the query will report an error.
The database log shows the following error:
LOG: server process (PID 24796) was terminated by signal 6: Aborted
DETAIL: Failed process was running: select * from test t join orders o on
t.order_id =o.order_id where t.order_id in (select order_id from orders);
LOG: terminating any other active server processes
The table structure is as follows, all of which are empty tables:
demo=# \d+ test
Table "bookings.test"
Column | Type | Collation | Nullable | Default | Storage |
Compression | Stats target | Description
----------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
order_id | integer | | | | plain |
| |
name | character varying | | | | extended |
| |
Access method: heap
demo=# \d+ orders
Partitioned table "bookings.orders"
Column | Type | Collation | Nullable | Default | Storage
| Compression | Stats target | Description
----------+-----------------------+-----------+----------+---------+----------+-------------+--------------+-------------
order_id | integer | | | | plain
| | |
name | character varying(10) | | | | extended
| | |
Partition key: HASH (order_id)
Partitions: orders_p1 FOR VALUES WITH (modulus 32, remainder 0),
orders_p10 FOR VALUES WITH (modulus 32, remainder 9),
orders_p11 FOR VALUES WITH (modulus 32, remainder 10),
orders_p12 FOR VALUES WITH (modulus 32, remainder 11),
orders_p13 FOR VALUES WITH (modulus 32, remainder 12),
orders_p14 FOR VALUES WITH (modulus 32, remainder 13),
orders_p15 FOR VALUES WITH (modulus 32, remainder 14),
orders_p16 FOR VALUES WITH (modulus 32, remainder 15),
orders_p17 FOR VALUES WITH (modulus 32, remainder 16),
orders_p18 FOR VALUES WITH (modulus 32, remainder 17),
orders_p19 FOR VALUES WITH (modulus 32, remainder 18),
orders_p2 FOR VALUES WITH (modulus 32, remainder 1),
orders_p20 FOR VALUES WITH (modulus 32, remainder 19),
orders_p21 FOR VALUES WITH (modulus 32, remainder 20),
orders_p22 FOR VALUES WITH (modulus 32, remainder 21),
orders_p23 FOR VALUES WITH (modulus 32, remainder 22),
orders_p24 FOR VALUES WITH (modulus 32, remainder 23),
orders_p25 FOR VALUES WITH (modulus 32, remainder 24),
orders_p26 FOR VALUES WITH (modulus 32, remainder 25),
orders_p28 FOR VALUES WITH (modulus 32, remainder 27),
orders_p29 FOR VALUES WITH (modulus 32, remainder 28),
orders_p3 FOR VALUES WITH (modulus 32, remainder 2),
orders_p30 FOR VALUES WITH (modulus 32, remainder 29),
orders_p31 FOR VALUES WITH (modulus 32, remainder 30),
orders_p32 FOR VALUES WITH (modulus 32, remainder 31),
orders_p4 FOR VALUES WITH (modulus 32, remainder 3),
orders_p5 FOR VALUES WITH (modulus 32, remainder 4),
orders_p6 FOR VALUES WITH (modulus 32, remainder 5),
orders_p7 FOR VALUES WITH (modulus 32, remainder 6),
orders_p8 FOR VALUES WITH (modulus 32, remainder 7),
orders_p9 FOR VALUES WITH (modulus 32, remainder 8)
demo=#
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
@ 2025-02-12 07:38 ` Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 08:55 ` Re:Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally =?GBK?B?w8/B7rHy?= <m_lingbin@126.com>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
0 siblings, 2 replies; 22+ messages in thread
From: Laurenz Albe @ 2025-02-12 07:38 UTC (permalink / raw)
To: m_lingbin@126.com; pgsql-bugs@lists.postgresql.org
On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
> PostgreSQL version: 17.2
> Operating system: CentOS Linux release 7.9.2009 (Core)
> Description:
>
> After upgrading PG17.2, testing found that when enable_rartitionwise_join is
> set to ON, when executing a query, if the main query and subquery have the
> same table, the query will report an error.
>
> The database log shows the following error:
> LOG: server process (PID 24796) was terminated by signal 6: Aborted
> DETAIL: Failed process was running: select * from test t join orders o on
> t.order_id =o.order_id where t.order_id in (select order_id from orders);
> LOG: terminating any other active server processes
That is a crash that may well indicate a PostgreSQL bug.
However, without a way to reproduce the behavior, we won't be able to fix
the problem. Try to come up with a self-contained test case. It would
also be interesting to know which PostgreSQL extensions are present.
Yours,
Laurenz Albe
--
*E-Mail Disclaimer*
Der Inhalt dieser E-Mail ist ausschliesslich fuer den
bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorgesehene Adressat
dieser E-Mail oder dessen Vertreter sein sollten, so beachten Sie bitte,
dass jede Form der Kenntnisnahme, Veroeffentlichung, Vervielfaeltigung oder
Weitergabe des Inhalts dieser E-Mail unzulaessig ist. Wir bitten Sie, sich
in diesem Fall mit dem Absender der E-Mail in Verbindung zu setzen.
*CONFIDENTIALITY NOTICE & DISCLAIMER
*This message and any attachment are
confidential and may be privileged or otherwise protected from disclosure
and solely for the use of the person(s) or entity to whom it is intended.
If you have received this message in error and are not the intended
recipient, please notify the sender immediately and delete this message and
any attachment from your system. If you are not the intended recipient, be
advised that any use of this message is prohibited and may be unlawful, and
you must not copy this message or attachment or disclose the contents to
any other person.
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re:Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
@ 2025-02-12 08:55 ` =?GBK?B?w8/B7rHy?= <m_lingbin@126.com>
1 sibling, 0 replies; 22+ messages in thread
From: 孟令彬 @ 2025-02-12 08:55 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org
The above email has provided the table structure. You can try creating the same table structure and executing 'select * from test t join orders o on'
t.order_id =o.order_id where t.order_id in (select order_id from orders);‘ sentence.
I believe that you will soon see the occurrence of errors.
demo=# \d+ test
Table "bookings.test"
Column | Type | Collation | Nullable | Default | Storage |
Compression | Stats target | Description
----------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
order_id | integer | | | | plain |
| |
name | character varying | | | | extended |
| |
Access method: heap
demo=# \d+ orders
Partitioned table "bookings.orders"
Column | Type | Collation | Nullable | Default | Storage
| Compression | Stats target | Description
----------+-----------------------+-----------+----------+---------+----------+-------------+--------------+-------------
order_id | integer | | | | plain
| | |
name | character varying(10) | | | | extended
| | |
Partition key: HASH (order_id)
Partitions: orders_p1 FOR VALUES WITH (modulus 32, remainder 0),
orders_p10 FOR VALUES WITH (modulus 32, remainder 9),
orders_p11 FOR VALUES WITH (modulus 32, remainder 10),
orders_p12 FOR VALUES WITH (modulus 32, remainder 11),
orders_p13 FOR VALUES WITH (modulus 32, remainder 12),
orders_p14 FOR VALUES WITH (modulus 32, remainder 13),
orders_p15 FOR VALUES WITH (modulus 32, remainder 14),
orders_p16 FOR VALUES WITH (modulus 32, remainder 15),
orders_p17 FOR VALUES WITH (modulus 32, remainder 16),
orders_p18 FOR VALUES WITH (modulus 32, remainder 17),
orders_p19 FOR VALUES WITH (modulus 32, remainder 18),
orders_p2 FOR VALUES WITH (modulus 32, remainder 1),
orders_p20 FOR VALUES WITH (modulus 32, remainder 19),
orders_p21 FOR VALUES WITH (modulus 32, remainder 20),
orders_p22 FOR VALUES WITH (modulus 32, remainder 21),
orders_p23 FOR VALUES WITH (modulus 32, remainder 22),
orders_p24 FOR VALUES WITH (modulus 32, remainder 23),
orders_p25 FOR VALUES WITH (modulus 32, remainder 24),
orders_p26 FOR VALUES WITH (modulus 32, remainder 25),
orders_p28 FOR VALUES WITH (modulus 32, remainder 27),
orders_p29 FOR VALUES WITH (modulus 32, remainder 28),
orders_p3 FOR VALUES WITH (modulus 32, remainder 2),
orders_p30 FOR VALUES WITH (modulus 32, remainder 29),
orders_p31 FOR VALUES WITH (modulus 32, remainder 30),
orders_p32 FOR VALUES WITH (modulus 32, remainder 31),
orders_p4 FOR VALUES WITH (modulus 32, remainder 3),
orders_p5 FOR VALUES WITH (modulus 32, remainder 4),
orders_p6 FOR VALUES WITH (modulus 32, remainder 5),
orders_p7 FOR VALUES WITH (modulus 32, remainder 6),
orders_p8 FOR VALUES WITH (modulus 32, remainder 7),
orders_p9 FOR VALUES WITH (modulus 32, remainder 8)
demo=#
good luck!
At 2025-02-12 15:38:34, "Laurenz Albe" <laurenz.albe@cybertec.at> wrote:
>On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
>> PostgreSQL version: 17.2
>> Operating system: CentOS Linux release 7.9.2009 (Core)
>> Description:
>>
>> After upgrading PG17.2, testing found that when enable_rartitionwise_join is
>> set to ON, when executing a query, if the main query and subquery have the
>> same table, the query will report an error.
>>
>> The database log shows the following error:
>> LOG: server process (PID 24796) was terminated by signal 6: Aborted
>> DETAIL: Failed process was running: select * from test t join orders o on
>> t.order_id =o.order_id where t.order_id in (select order_id from orders);
>> LOG: terminating any other active server processes
>
>That is a crash that may well indicate a PostgreSQL bug.
>
>However, without a way to reproduce the behavior, we won't be able to fix
>the problem. Try to come up with a self-contained test case. It would
>also be interesting to know which PostgreSQL extensions are present.
>
>Yours,
>Laurenz Albe
>
>--
>
>*E-Mail Disclaimer*
>Der Inhalt dieser E-Mail ist ausschliesslich fuer den
>bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorgesehene Adressat
>dieser E-Mail oder dessen Vertreter sein sollten, so beachten Sie bitte,
>dass jede Form der Kenntnisnahme, Veroeffentlichung, Vervielfaeltigung oder
>Weitergabe des Inhalts dieser E-Mail unzulaessig ist. Wir bitten Sie, sich
>in diesem Fall mit dem Absender der E-Mail in Verbindung zu setzen.
>
>*CONFIDENTIALITY NOTICE & DISCLAIMER
>*This message and any attachment are
>confidential and may be privileged or otherwise protected from disclosure
>and solely for the use of the person(s) or entity to whom it is intended.
>If you have received this message in error and are not the intended
>recipient, please notify the sender immediately and delete this message and
>any attachment from your system. If you are not the intended recipient, be
>advised that any use of this message is prohibited and may be unlawful, and
>you must not copy this message or attachment or disclose the contents to
>any other person.
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
@ 2025-02-12 09:48 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Tender Wang @ 2025-02-12 09:48 UTC (permalink / raw)
To: Laurenz Albe <laurenz.albe@cybertec.at>; +Cc: m_lingbin@126.com; pgsql-bugs@lists.postgresql.org
Laurenz Albe <laurenz.albe@cybertec.at> 于2025年2月12日周三 15:38写道：
> On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
> > PostgreSQL version: 17.2
> > Operating system: CentOS Linux release 7.9.2009 (Core)
> > Description:
> >
> > After upgrading PG17.2, testing found that when
> enable_rartitionwise_join is
> > set to ON, when executing a query, if the main query and subquery have
> the
> > same table, the query will report an error.
> >
> > The database log shows the following error:
> > LOG: server process (PID 24796) was terminated by signal 6: Aborted
> > DETAIL: Failed process was running: select * from test t join orders o
> on
> > t.order_id =o.order_id where t.order_id in (select order_id from orders);
> > LOG: terminating any other active server processes
>
> That is a crash that may well indicate a PostgreSQL bug.
>
> However, without a way to reproduce the behavior, we won't be able to fix
> the problem. Try to come up with a self-contained test case. It would
> also be interesting to know which PostgreSQL extensions are present.
>
>
I can reproduce this crash on HEAD.
postgres=# create table test(order_id int, name varchar);
create table orders(order_id int, name char(10)) partition by hash
(order_id);
CREATE TABLE
CREATE TABLE
postgres=# set enable_partitionwise_join = on;
SET
postgres=# create table orders_p1 partition of orders for values with (
modulus 32, remainder 0);
CREATE TABLE
postgres=# explain select * from test t join orders o on
t.order_id =o.order_id where t.order_id in (select order_id from orders);
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.
--
Thanks,
Tender Wang
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
@ 2025-02-12 11:48 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 13:55 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
0 siblings, 2 replies; 22+ messages in thread
From: Tender Wang @ 2025-02-12 11:48 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org, Amit Langote <amitlangote09@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; +Cc: m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 17:48写道：
>
>
> Laurenz Albe <laurenz.albe@cybertec.at> 于2025年2月12日周三 15:38写道：
>
>> On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
>> > PostgreSQL version: 17.2
>> > Operating system: CentOS Linux release 7.9.2009 (Core)
>> > Description:
>> >
>> > After upgrading PG17.2, testing found that when
>> enable_rartitionwise_join is
>> > set to ON, when executing a query, if the main query and subquery have
>> the
>> > same table, the query will report an error.
>> >
>> > The database log shows the following error:
>> > LOG: server process (PID 24796) was terminated by signal 6: Aborted
>> > DETAIL: Failed process was running: select * from test t join orders o
>> on
>> > t.order_id =o.order_id where t.order_id in (select order_id from
>> orders);
>> > LOG: terminating any other active server processes
>>
>> That is a crash that may well indicate a PostgreSQL bug.
>>
>> However, without a way to reproduce the behavior, we won't be able to fix
>> the problem. Try to come up with a self-contained test case. It would
>> also be interesting to know which PostgreSQL extensions are present.
>>
>>
> I can reproduce this crash on HEAD.
>
> postgres=# create table test(order_id int, name varchar);
> create table orders(order_id int, name char(10)) partition by hash
> (order_id);
> CREATE TABLE
> CREATE TABLE
> postgres=# set enable_partitionwise_join = on;
> SET
> postgres=# create table orders_p1 partition of orders for values with (
> modulus 32, remainder 0);
> CREATE TABLE
> postgres=# explain select * from test t join orders o on
> t.order_id =o.order_id where t.order_id in (select order_id from orders);
> server closed the connection unexpectedly
> This probably means the server terminated abnormally
> before or while processing the request.
> The connection to the server was lost. Attempting reset: Succeeded.
>
>
I debug this crash, I found this crash was related with below commit:
commit 5278d0a2e870c61f9374a7796b90e6f9f6a73638
Author: Amit Langote <amitlan@postgresql.org>
Date: Mon Mar 25 12:02:40 2024 +0900
Reduce memory used by partitionwise joins
the joinlist in this query is 1,2,5
1 means table t
2 means table orders
5 also means table orders, which subquery is pulled up.
When we create join rel for (2,5), then wen enter try_partitionwise_join().
In this func, we call build_child_join_sjinfo() to build SpecialJoinInfo
for child.
In build_child_join_sjinfo(), we use memcpy(), I think we should use
copyObject().
Otherwise, the parent_sjinfo will be freed, for example, after
call free_child_join_sjinfo(),
the parent_sjinfo min_lefthand looks like as below:
(gdb) p *parent_sjinfo->min_lefthand
$7 = {type = 3951489872, nwords = 21994, words = 0x55eaeb8696b0}
The above bitmap is invalid, so trigger the Assert(bms_is_valid_set(a));
Attached patch is my fix. Any thoughts?
--
Thanks,
Tender Wang
Attachments:
0001-Fix-parent_sjinfo-wrong-freed-issue.patch (text/plain, 3.1K) download
From 248cf398a3777fc20233dd4dea941179fcb4170e Mon Sep 17 00:00:00 2001
From: Tender Wang <tndrwang@gmail.com>
Date: Wed, 12 Feb 2025 19:35:33 +0800
Subject: [PATCH] Fix parent_sjinfo wrong freed issue.
---
src/backend/optimizer/path/joinrels.c | 2 +-
src/test/regress/expected/partition_join.out | 21 ++++++++++++++++++++
src/test/regress/sql/partition_join.sql | 7 +++++++
3 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index c2eb300ea9..dd598c79b5 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -1717,7 +1717,7 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
return sjinfo;
}
- memcpy(sjinfo, parent_sjinfo, sizeof(SpecialJoinInfo));
+ sjinfo = copyObject(parent_sjinfo);
left_appinfos = find_appinfos_by_relids(root, left_relids,
&left_nappinfos);
right_appinfos = find_appinfos_by_relids(root, right_relids,
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index af468682a2..c1b7df8659 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -713,6 +713,27 @@ SELECT a, b FROM prt1 FULL JOIN prt2 p2(b,a,c) USING(a,b)
RESET enable_partitionwise_aggregate;
RESET enable_hashjoin;
+-- bug free parent_sjinfo
+create table pw_test(order_id int, name varchar);
+create table pw_orders(order_id int, name char(10)) partition by hash (order_id);
+create table orders_p1 partition of pw_orders for values with ( modulus 32, remainder 0);
+explain(costs off) select * from pw_test t join pw_orders o on
+t.order_id =o.order_id where t.order_id in (select order_id from pw_orders);
+ QUERY PLAN
+---------------------------------------------------------------
+ Hash Join
+ Hash Cond: (o.order_id = t.order_id)
+ -> Seq Scan on orders_p1 o
+ -> Hash
+ -> Hash Join
+ Hash Cond: (t.order_id = pw_orders.order_id)
+ -> Seq Scan on pw_test t
+ -> Hash
+ -> HashAggregate
+ Group Key: pw_orders.order_id
+ -> Seq Scan on orders_p1 pw_orders
+(11 rows)
+
--
-- partitioned by expression
--
diff --git a/src/test/regress/sql/partition_join.sql b/src/test/regress/sql/partition_join.sql
index e84b65f444..d8ad4ce3dc 100644
--- a/src/test/regress/sql/partition_join.sql
+++ b/src/test/regress/sql/partition_join.sql
@@ -143,6 +143,13 @@ SELECT a, b FROM prt1 FULL JOIN prt2 p2(b,a,c) USING(a,b)
RESET enable_partitionwise_aggregate;
RESET enable_hashjoin;
+-- bug free parent_sjinfo
+create table pw_test(order_id int, name varchar);
+create table pw_orders(order_id int, name char(10)) partition by hash (order_id);
+create table orders_p1 partition of pw_orders for values with ( modulus 32, remainder 0);
+explain(costs off) select * from pw_test t join pw_orders o on
+t.order_id =o.order_id where t.order_id in (select order_id from pw_orders);
+
--
-- partitioned by expression
--
--
2.34.1
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
@ 2025-02-12 12:11 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Tender Wang @ 2025-02-12 12:11 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org, Amit Langote <amitlangote09@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; +Cc: m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 19:48写道：
>
>
> Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 17:48写道：
>
>>
>>
>> Laurenz Albe <laurenz.albe@cybertec.at> 于2025年2月12日周三 15:38写道：
>>
>>> On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
>>> > PostgreSQL version: 17.2
>>> > Operating system: CentOS Linux release 7.9.2009 (Core)
>>> > Description:
>>> >
>>> > After upgrading PG17.2, testing found that when
>>> enable_rartitionwise_join is
>>> > set to ON, when executing a query, if the main query and subquery have
>>> the
>>> > same table, the query will report an error.
>>> >
>>> > The database log shows the following error:
>>> > LOG: server process (PID 24796) was terminated by signal 6: Aborted
>>> > DETAIL: Failed process was running: select * from test t join orders
>>> o on
>>> > t.order_id =o.order_id where t.order_id in (select order_id from
>>> orders);
>>> > LOG: terminating any other active server processes
>>>
>>> That is a crash that may well indicate a PostgreSQL bug.
>>>
>>> However, without a way to reproduce the behavior, we won't be able to fix
>>> the problem. Try to come up with a self-contained test case. It would
>>> also be interesting to know which PostgreSQL extensions are present.
>>>
>>>
>> I can reproduce this crash on HEAD.
>>
>> postgres=# create table test(order_id int, name varchar);
>> create table orders(order_id int, name char(10)) partition by hash
>> (order_id);
>> CREATE TABLE
>> CREATE TABLE
>> postgres=# set enable_partitionwise_join = on;
>> SET
>> postgres=# create table orders_p1 partition of orders for values with (
>> modulus 32, remainder 0);
>> CREATE TABLE
>> postgres=# explain select * from test t join orders o on
>> t.order_id =o.order_id where t.order_id in (select order_id from orders);
>> server closed the connection unexpectedly
>> This probably means the server terminated abnormally
>> before or while processing the request.
>> The connection to the server was lost. Attempting reset: Succeeded.
>>
>>
> I debug this crash, I found this crash was related with below commit:
> commit 5278d0a2e870c61f9374a7796b90e6f9f6a73638
> Author: Amit Langote <amitlan@postgresql.org>
> Date: Mon Mar 25 12:02:40 2024 +0900
>
> Reduce memory used by partitionwise joins
>
>
> the joinlist in this query is 1,2,5
> 1 means table t
> 2 means table orders
> 5 also means table orders, which subquery is pulled up.
> When we create join rel for (2,5), then wen enter try_partitionwise_join().
> In this func, we call build_child_join_sjinfo() to build SpecialJoinInfo
> for child.
> In build_child_join_sjinfo(), we use memcpy(), I think we should use
> copyObject().
> Otherwise, the parent_sjinfo will be freed, for example, after
> call free_child_join_sjinfo(),
> the parent_sjinfo min_lefthand looks like as below:
>
> (gdb) p *parent_sjinfo->min_lefthand
> $7 = {type = 3951489872, nwords = 21994, words = 0x55eaeb8696b0}
>
> The above bitmap is invalid, so trigger the Assert(bms_is_valid_set(a));
>
> Attached patch is my fix. Any thoughts?
>
I forgot sjinfo had called makeNode(), in v1 patch, origin memory to
sjinfo will leak, so I do a little adjust in v2 patch.
--
Thanks,
Tender Wang
Attachments:
v2-0001-Fix-parent_sjinfo-wrong-freed-issue.patch (text/plain, 3.7K) download
From 09e75e99aab7933ce690ad1cf8b3e46be11bd9f0 Mon Sep 17 00:00:00 2001
From: Tender Wang <tndrwang@gmail.com>
Date: Wed, 12 Feb 2025 19:35:33 +0800
Subject: [PATCH v2] Fix parent_sjinfo wrong freed issue.
---
src/backend/optimizer/path/joinrels.c | 5 +++--
src/test/regress/expected/partition_join.out | 21 ++++++++++++++++++++
src/test/regress/sql/partition_join.sql | 7 +++++++
3 files changed, 31 insertions(+), 2 deletions(-)
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index c2eb300ea9..c33baeca6d 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -1703,7 +1703,7 @@ static SpecialJoinInfo *
build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
Relids left_relids, Relids right_relids)
{
- SpecialJoinInfo *sjinfo = makeNode(SpecialJoinInfo);
+ SpecialJoinInfo *sjinfo;
AppendRelInfo **left_appinfos;
int left_nappinfos;
AppendRelInfo **right_appinfos;
@@ -1713,11 +1713,12 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
if (parent_sjinfo->jointype == JOIN_INNER)
{
Assert(parent_sjinfo->ojrelid == 0);
+ sjinfo = makeNode(SpecialJoinInfo);
init_dummy_sjinfo(sjinfo, left_relids, right_relids);
return sjinfo;
}
- memcpy(sjinfo, parent_sjinfo, sizeof(SpecialJoinInfo));
+ sjinfo = copyObject(parent_sjinfo);
left_appinfos = find_appinfos_by_relids(root, left_relids,
&left_nappinfos);
right_appinfos = find_appinfos_by_relids(root, right_relids,
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index af468682a2..c1b7df8659 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -713,6 +713,27 @@ SELECT a, b FROM prt1 FULL JOIN prt2 p2(b,a,c) USING(a,b)
RESET enable_partitionwise_aggregate;
RESET enable_hashjoin;
+-- bug free parent_sjinfo
+create table pw_test(order_id int, name varchar);
+create table pw_orders(order_id int, name char(10)) partition by hash (order_id);
+create table orders_p1 partition of pw_orders for values with ( modulus 32, remainder 0);
+explain(costs off) select * from pw_test t join pw_orders o on
+t.order_id =o.order_id where t.order_id in (select order_id from pw_orders);
+ QUERY PLAN
+---------------------------------------------------------------
+ Hash Join
+ Hash Cond: (o.order_id = t.order_id)
+ -> Seq Scan on orders_p1 o
+ -> Hash
+ -> Hash Join
+ Hash Cond: (t.order_id = pw_orders.order_id)
+ -> Seq Scan on pw_test t
+ -> Hash
+ -> HashAggregate
+ Group Key: pw_orders.order_id
+ -> Seq Scan on orders_p1 pw_orders
+(11 rows)
+
--
-- partitioned by expression
--
diff --git a/src/test/regress/sql/partition_join.sql b/src/test/regress/sql/partition_join.sql
index e84b65f444..d8ad4ce3dc 100644
--- a/src/test/regress/sql/partition_join.sql
+++ b/src/test/regress/sql/partition_join.sql
@@ -143,6 +143,13 @@ SELECT a, b FROM prt1 FULL JOIN prt2 p2(b,a,c) USING(a,b)
RESET enable_partitionwise_aggregate;
RESET enable_hashjoin;
+-- bug free parent_sjinfo
+create table pw_test(order_id int, name varchar);
+create table pw_orders(order_id int, name char(10)) partition by hash (order_id);
+create table orders_p1 partition of pw_orders for values with ( modulus 32, remainder 0);
+explain(costs off) select * from pw_test t join pw_orders o on
+t.order_id =o.order_id where t.order_id in (select order_id from pw_orders);
+
--
-- partitioned by expression
--
--
2.34.1
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
@ 2025-02-12 12:20 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Ashutosh Bapat @ 2025-02-12 12:20 UTC (permalink / raw)
To: Tender Wang <tndrwang@gmail.com>; +Cc: pgsql-bugs@lists.postgresql.org, Amit Langote <amitlangote09@gmail.com>; m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Wed, Feb 12, 2025 at 5:41 PM Tender Wang <tndrwang@gmail.com> wrote:
>
>
>
> Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 19:48写道：
>>
>>
>>
>> Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 17:48写道：
>>>
>>>
>>>
>>> Laurenz Albe <laurenz.albe@cybertec.at> 于2025年2月12日周三 15:38写道：
>>>>
>>>> On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
>>>> > PostgreSQL version: 17.2
>>>> > Operating system: CentOS Linux release 7.9.2009 (Core)
>>>> > Description:
>>>> >
>>>> > After upgrading PG17.2, testing found that when enable_rartitionwise_join is
>>>> > set to ON, when executing a query, if the main query and subquery have the
>>>> > same table, the query will report an error.
>>>> >
>>>> > The database log shows the following error:
>>>> > LOG: server process (PID 24796) was terminated by signal 6: Aborted
>>>> > DETAIL: Failed process was running: select * from test t join orders o on
>>>> > t.order_id =o.order_id where t.order_id in (select order_id from orders);
>>>> > LOG: terminating any other active server processes
>>>>
>>>> That is a crash that may well indicate a PostgreSQL bug.
>>>>
>>>> However, without a way to reproduce the behavior, we won't be able to fix
>>>> the problem. Try to come up with a self-contained test case. It would
>>>> also be interesting to know which PostgreSQL extensions are present.
>>>>
>>>
>>> I can reproduce this crash on HEAD.
>>>
>>> postgres=# create table test(order_id int, name varchar);
>>> create table orders(order_id int, name char(10)) partition by hash (order_id);
>>> CREATE TABLE
>>> CREATE TABLE
>>> postgres=# set enable_partitionwise_join = on;
>>> SET
>>> postgres=# create table orders_p1 partition of orders for values with ( modulus 32, remainder 0);
>>> CREATE TABLE
>>> postgres=# explain select * from test t join orders o on
>>> t.order_id =o.order_id where t.order_id in (select order_id from orders);
>>> server closed the connection unexpectedly
>>> This probably means the server terminated abnormally
>>> before or while processing the request.
>>> The connection to the server was lost. Attempting reset: Succeeded.
>>>
>>
>> I debug this crash, I found this crash was related with below commit:
>> commit 5278d0a2e870c61f9374a7796b90e6f9f6a73638
>> Author: Amit Langote <amitlan@postgresql.org>
>> Date: Mon Mar 25 12:02:40 2024 +0900
>>
>> Reduce memory used by partitionwise joins
>>
>>
>> the joinlist in this query is 1,2,5
>> 1 means table t
>> 2 means table orders
>> 5 also means table orders, which subquery is pulled up.
>> When we create join rel for (2,5), then wen enter try_partitionwise_join().
>> In this func, we call build_child_join_sjinfo() to build SpecialJoinInfo for child.
>> In build_child_join_sjinfo(), we use memcpy(), I think we should use copyObject().
>> Otherwise, the parent_sjinfo will be freed, for example, after call free_child_join_sjinfo(),
>> the parent_sjinfo min_lefthand looks like as below:
>>
>> (gdb) p *parent_sjinfo->min_lefthand
>> $7 = {type = 3951489872, nwords = 21994, words = 0x55eaeb8696b0}
>>
>> The above bitmap is invalid, so trigger the Assert(bms_is_valid_set(a));
>>
>> Attached patch is my fix. Any thoughts?
Thanks for the report and the fix. Ideally, following line should
allocate separate memory for the child's bitmapset.
sjinfo->min_lefthand = adjust_child_relids(sjinfo->min_lefthand,
left_nappinfos, left_appinfos);
Unless no relid in sjinfo->min_lefthand was translated according to
the given appinfos, adjust_child_relids() should create a new
Bitmapset. Can you please check why no translation happened. I will
try your reproduction tomorrow.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
@ 2025-02-12 12:33 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Tender Wang @ 2025-02-12 12:33 UTC (permalink / raw)
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; +Cc: pgsql-bugs@lists.postgresql.org, Amit Langote <amitlangote09@gmail.com>; m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> 于2025年2月12日周三 20:20写道：
> On Wed, Feb 12, 2025 at 5:41 PM Tender Wang <tndrwang@gmail.com> wrote:
> >
> >
> >
> > Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 19:48写道：
> >>
> >>
> >>
> >> Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 17:48写道：
> >>>
> >>>
> >>>
> >>> Laurenz Albe <laurenz.albe@cybertec.at> 于2025年2月12日周三 15:38写道：
> >>>>
> >>>> On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
> >>>> > PostgreSQL version: 17.2
> >>>> > Operating system: CentOS Linux release 7.9.2009 (Core)
> >>>> > Description:
> >>>> >
> >>>> > After upgrading PG17.2, testing found that when
> enable_rartitionwise_join is
> >>>> > set to ON, when executing a query, if the main query and subquery
> have the
> >>>> > same table, the query will report an error.
> >>>> >
> >>>> > The database log shows the following error:
> >>>> > LOG: server process (PID 24796) was terminated by signal 6: Aborted
> >>>> > DETAIL: Failed process was running: select * from test t join
> orders o on
> >>>> > t.order_id =o.order_id where t.order_id in (select order_id from
> orders);
> >>>> > LOG: terminating any other active server processes
> >>>>
> >>>> That is a crash that may well indicate a PostgreSQL bug.
> >>>>
> >>>> However, without a way to reproduce the behavior, we won't be able to
> fix
> >>>> the problem. Try to come up with a self-contained test case. It
> would
> >>>> also be interesting to know which PostgreSQL extensions are present.
> >>>>
> >>>
> >>> I can reproduce this crash on HEAD.
> >>>
> >>> postgres=# create table test(order_id int, name varchar);
> >>> create table orders(order_id int, name char(10)) partition by hash
> (order_id);
> >>> CREATE TABLE
> >>> CREATE TABLE
> >>> postgres=# set enable_partitionwise_join = on;
> >>> SET
> >>> postgres=# create table orders_p1 partition of orders for values with
> ( modulus 32, remainder 0);
> >>> CREATE TABLE
> >>> postgres=# explain select * from test t join orders o on
> >>> t.order_id =o.order_id where t.order_id in (select order_id from
> orders);
> >>> server closed the connection unexpectedly
> >>> This probably means the server terminated abnormally
> >>> before or while processing the request.
> >>> The connection to the server was lost. Attempting reset: Succeeded.
> >>>
> >>
> >> I debug this crash, I found this crash was related with below commit:
> >> commit 5278d0a2e870c61f9374a7796b90e6f9f6a73638
> >> Author: Amit Langote <amitlan@postgresql.org>
> >> Date: Mon Mar 25 12:02:40 2024 +0900
> >>
> >> Reduce memory used by partitionwise joins
> >>
> >>
> >> the joinlist in this query is 1,2,5
> >> 1 means table t
> >> 2 means table orders
> >> 5 also means table orders, which subquery is pulled up.
> >> When we create join rel for (2,5), then wen enter
> try_partitionwise_join().
> >> In this func, we call build_child_join_sjinfo() to build
> SpecialJoinInfo for child.
> >> In build_child_join_sjinfo(), we use memcpy(), I think we should use
> copyObject().
> >> Otherwise, the parent_sjinfo will be freed, for example, after call
> free_child_join_sjinfo(),
> >> the parent_sjinfo min_lefthand looks like as below:
> >>
> >> (gdb) p *parent_sjinfo->min_lefthand
> >> $7 = {type = 3951489872, nwords = 21994, words = 0x55eaeb8696b0}
> >>
> >> The above bitmap is invalid, so trigger the Assert(bms_is_valid_set(a));
> >>
> >> Attached patch is my fix. Any thoughts?
>
> Thanks for the report and the fix. Ideally, following line should
> allocate separate memory for the child's bitmapset.
> sjinfo->min_lefthand = adjust_child_relids(sjinfo->min_lefthand,
> left_nappinfos, left_appinfos);
>
> Unless no relid in sjinfo->min_lefthand was translated according to
> the given appinfos, adjust_child_relids() should create a new
> Bitmapset. Can you please check why no translation happened. I will
> try your reproduction tomorrow.
>
I step into adjust_child_relids(), see below:
nappinfos = 1;
p appinfo->parent_relid
2
call bmsToString(relids)
"(b 1)"
So no translation you said happened, just return relids.
--
Thanks,
Tender Wang
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
@ 2025-02-12 14:04 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Ashutosh Bapat @ 2025-02-12 14:04 UTC (permalink / raw)
To: Tender Wang <tndrwang@gmail.com>; +Cc: pgsql-bugs@lists.postgresql.org, Amit Langote <amitlangote09@gmail.com>; m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Wed, Feb 12, 2025 at 6:03 PM Tender Wang <tndrwang@gmail.com> wrote:
>
>
>
> Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> 于2025年2月12日周三 20:20写道：
>>
>> On Wed, Feb 12, 2025 at 5:41 PM Tender Wang <tndrwang@gmail.com> wrote:
>> >
>> >
>> >
>> > Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 19:48写道：
>> >>
>> >>
>> >>
>> >> Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 17:48写道：
>> >>>
>> >>>
>> >>>
>> >>> Laurenz Albe <laurenz.albe@cybertec.at> 于2025年2月12日周三 15:38写道：
>> >>>>
>> >>>> On Wed, 2025-02-12 at 07:01 +0000, PG Bug reporting form wrote:
>> >>>> > PostgreSQL version: 17.2
>> >>>> > Operating system: CentOS Linux release 7.9.2009 (Core)
>> >>>> > Description:
>> >>>> >
>> >>>> > After upgrading PG17.2, testing found that when enable_rartitionwise_join is
>> >>>> > set to ON, when executing a query, if the main query and subquery have the
>> >>>> > same table, the query will report an error.
>> >>>> >
>> >>>> > The database log shows the following error:
>> >>>> > LOG: server process (PID 24796) was terminated by signal 6: Aborted
>> >>>> > DETAIL: Failed process was running: select * from test t join orders o on
>> >>>> > t.order_id =o.order_id where t.order_id in (select order_id from orders);
>> >>>> > LOG: terminating any other active server processes
>> >>>>
>> >>>> That is a crash that may well indicate a PostgreSQL bug.
>> >>>>
>> >>>> However, without a way to reproduce the behavior, we won't be able to fix
>> >>>> the problem. Try to come up with a self-contained test case. It would
>> >>>> also be interesting to know which PostgreSQL extensions are present.
>> >>>>
>> >>>
>> >>> I can reproduce this crash on HEAD.
>> >>>
>> >>> postgres=# create table test(order_id int, name varchar);
>> >>> create table orders(order_id int, name char(10)) partition by hash (order_id);
>> >>> CREATE TABLE
>> >>> CREATE TABLE
>> >>> postgres=# set enable_partitionwise_join = on;
>> >>> SET
>> >>> postgres=# create table orders_p1 partition of orders for values with ( modulus 32, remainder 0);
>> >>> CREATE TABLE
>> >>> postgres=# explain select * from test t join orders o on
>> >>> t.order_id =o.order_id where t.order_id in (select order_id from orders);
>> >>> server closed the connection unexpectedly
>> >>> This probably means the server terminated abnormally
>> >>> before or while processing the request.
>> >>> The connection to the server was lost. Attempting reset: Succeeded.
>> >>>
>> >>
>> >> I debug this crash, I found this crash was related with below commit:
>> >> commit 5278d0a2e870c61f9374a7796b90e6f9f6a73638
>> >> Author: Amit Langote <amitlan@postgresql.org>
>> >> Date: Mon Mar 25 12:02:40 2024 +0900
>> >>
>> >> Reduce memory used by partitionwise joins
>> >>
>> >>
>> >> the joinlist in this query is 1,2,5
>> >> 1 means table t
>> >> 2 means table orders
>> >> 5 also means table orders, which subquery is pulled up.
>> >> When we create join rel for (2,5), then wen enter try_partitionwise_join().
>> >> In this func, we call build_child_join_sjinfo() to build SpecialJoinInfo for child.
>> >> In build_child_join_sjinfo(), we use memcpy(), I think we should use copyObject().
>> >> Otherwise, the parent_sjinfo will be freed, for example, after call free_child_join_sjinfo(),
>> >> the parent_sjinfo min_lefthand looks like as below:
>> >>
>> >> (gdb) p *parent_sjinfo->min_lefthand
>> >> $7 = {type = 3951489872, nwords = 21994, words = 0x55eaeb8696b0}
>> >>
>> >> The above bitmap is invalid, so trigger the Assert(bms_is_valid_set(a));
>> >>
>> >> Attached patch is my fix. Any thoughts?
>>
>> Thanks for the report and the fix. Ideally, following line should
>> allocate separate memory for the child's bitmapset.
>> sjinfo->min_lefthand = adjust_child_relids(sjinfo->min_lefthand,
>> left_nappinfos, left_appinfos);
>>
>> Unless no relid in sjinfo->min_lefthand was translated according to
>> the given appinfos, adjust_child_relids() should create a new
>> Bitmapset. Can you please check why no translation happened. I will
>> try your reproduction tomorrow.
>
>
> I step into adjust_child_relids(), see below:
> nappinfos = 1;
> p appinfo->parent_relid
> 2
> call bmsToString(relids)
> "(b 1)"
>
> So no translation you said happened, just return relids.
>
Hmm. The code there assumes that all the Relids will at least have one
parent each of the children involved. For some reason
sjinfo->min_lefthand has only relid 1 but not 2 or 5. 2 and 5 are
actually the parent relids of the children passed in respectively. The
join is between 2 and 5, then why is 1 appearing in the min_lefthand.
It might be legitimate, but we need to find the reason. If it's
legitimate, I think we need to copy the Relids which haven't undergone
any translation so as to keep them isolated from the parent relids.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
@ 2025-02-12 14:30 ` Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Richard Guo @ 2025-02-12 14:30 UTC (permalink / raw)
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; +Cc: Tender Wang <tndrwang@gmail.com>; pgsql-bugs@lists.postgresql.org, Amit Langote <amitlangote09@gmail.com>; m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Wed, Feb 12, 2025 at 10:04 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
> Hmm. The code there assumes that all the Relids will at least have one
> parent each of the children involved. For some reason
> sjinfo->min_lefthand has only relid 1 but not 2 or 5. 2 and 5 are
> actually the parent relids of the children passed in respectively. The
> join is between 2 and 5, then why is 1 appearing in the min_lefthand.
> It might be legitimate, but we need to find the reason. If it's
> legitimate, I think we need to copy the Relids which haven't undergone
> any translation so as to keep them isolated from the parent relids.
Yes, it's legitimate. For the semijoin, its join clause only
references {1} and {5}, with no other ordering restrictions.
Therefore, the minimum LHS for this join consists only of {1}.
Instead of copying the untranslated Relids and freeing them later, I
think it might be better to modify free_child_join_sjinfo() to avoid
freeing the untranslated members of child_sjinfo.
Thanks
Richard
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
@ 2025-02-13 02:39 ` Amit Langote <amitlangote09@gmail.com>
2025-02-13 03:14 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 07:25 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
0 siblings, 2 replies; 22+ messages in thread
From: Amit Langote @ 2025-02-13 02:39 UTC (permalink / raw)
To: Richard Guo <guofenglinux@gmail.com>; +Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; Tender Wang <tndrwang@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
Hi Richard,
On Wed, Feb 12, 2025 at 11:30 PM Richard Guo <guofenglinux@gmail.com> wrote:
> On Wed, Feb 12, 2025 at 10:04 PM Ashutosh Bapat
> <ashutosh.bapat.oss@gmail.com> wrote:
> > Hmm. The code there assumes that all the Relids will at least have one
> > parent each of the children involved. For some reason
> > sjinfo->min_lefthand has only relid 1 but not 2 or 5. 2 and 5 are
> > actually the parent relids of the children passed in respectively. The
> > join is between 2 and 5, then why is 1 appearing in the min_lefthand.
> > It might be legitimate, but we need to find the reason. If it's
> > legitimate, I think we need to copy the Relids which haven't undergone
> > any translation so as to keep them isolated from the parent relids.
>
> Yes, it's legitimate. For the semijoin, its join clause only
> references {1} and {5}, with no other ordering restrictions.
> Therefore, the minimum LHS for this join consists only of {1}.
>
> Instead of copying the untranslated Relids and freeing them later, I
> think it might be better to modify free_child_join_sjinfo() to avoid
> freeing the untranslated members of child_sjinfo.
free_child_join_sjinfo() frees min_lefthand and other fields
initialized by build_child_join_sjinfo(), assuming that
adjust_child_relids() creates a copy. However, this does not always
seem to be the case, as demonstrated in this report.
I'm wondering if the following line in adjust_child_relids() should be
using bms_copy() instead:
/* Otherwise, return the original set without modification. */
return relids;
}
That is, should we copy relids not only when translation is needed but
also in the general case? Would that be a bigger band-aid than
necessary?
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
@ 2025-02-13 03:14 ` Richard Guo <guofenglinux@gmail.com>
2025-02-13 04:55 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Richard Guo @ 2025-02-13 03:14 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; Tender Wang <tndrwang@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Thu, Feb 13, 2025 at 10:39 AM Amit Langote <amitlangote09@gmail.com> wrote:
> On Wed, Feb 12, 2025 at 11:30 PM Richard Guo <guofenglinux@gmail.com> wrote:
> > On Wed, Feb 12, 2025 at 10:04 PM Ashutosh Bapat
> > <ashutosh.bapat.oss@gmail.com> wrote:
> > > Hmm. The code there assumes that all the Relids will at least have one
> > > parent each of the children involved. For some reason
> > > sjinfo->min_lefthand has only relid 1 but not 2 or 5. 2 and 5 are
> > > actually the parent relids of the children passed in respectively. The
> > > join is between 2 and 5, then why is 1 appearing in the min_lefthand.
> > > It might be legitimate, but we need to find the reason. If it's
> > > legitimate, I think we need to copy the Relids which haven't undergone
> > > any translation so as to keep them isolated from the parent relids.
> >
> > Yes, it's legitimate. For the semijoin, its join clause only
> > references {1} and {5}, with no other ordering restrictions.
> > Therefore, the minimum LHS for this join consists only of {1}.
> >
> > Instead of copying the untranslated Relids and freeing them later, I
> > think it might be better to modify free_child_join_sjinfo() to avoid
> > freeing the untranslated members of child_sjinfo.
>
> free_child_join_sjinfo() frees min_lefthand and other fields
> initialized by build_child_join_sjinfo(), assuming that
> adjust_child_relids() creates a copy. However, this does not always
> seem to be the case, as demonstrated in this report.
>
> I'm wondering if the following line in adjust_child_relids() should be
> using bms_copy() instead:
>
> /* Otherwise, return the original set without modification. */
> return relids;
> }
>
> That is, should we copy relids not only when translation is needed but
> also in the general case? Would that be a bigger band-aid than
> necessary?
Hmm, I'm concerned this might lead to unnecessary memory usage in
other translation scenarios. After all, the purpose of 5278d0a2e is
to reduce memory used by partitionwise joins. Even in the
SpecialJoinInfo case, this approach means that we are copying
untranslated Relids and then freeing them later, which seems
unnecessary.
I'm thinking that a better approach might be to check if each member
of the child_sjinfo is a translated copy and only free it if that's
the case, something like attached.
Thanks
Richard
Attachments:
v3-0001-Fix-freeing-a-child-join-s-SpecialJoinInfo.patch (application/octet-stream, 5.4K) download
From 934d41885e943ebcc8d0da301dde992d73661761 Mon Sep 17 00:00:00 2001
From: Richard Guo <guofenglinux@gmail.com>
Date: Thu, 13 Feb 2025 11:55:37 +0900
Subject: [PATCH v3] Fix freeing a child join's SpecialJoinInfo
---
src/backend/optimizer/path/joinrels.c | 34 ++++++++++++++-----
src/test/regress/expected/partition_join.out | 35 ++++++++++++++++++++
src/test/regress/sql/partition_join.sql | 4 +++
3 files changed, 64 insertions(+), 9 deletions(-)
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index c2eb300ea9..60d65762b5 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -45,7 +45,8 @@ static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
static SpecialJoinInfo *build_child_join_sjinfo(PlannerInfo *root,
SpecialJoinInfo *parent_sjinfo,
Relids left_relids, Relids right_relids);
-static void free_child_join_sjinfo(SpecialJoinInfo *sjinfo);
+static void free_child_join_sjinfo(SpecialJoinInfo *child_sjinfo,
+ SpecialJoinInfo *parent_sjinfo);
static void compute_partition_bounds(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
@@ -1687,7 +1688,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
*/
pfree(appinfos);
bms_free(child_relids);
- free_child_join_sjinfo(child_sjinfo);
+ free_child_join_sjinfo(child_sjinfo, parent_sjinfo);
}
}
@@ -1754,18 +1755,33 @@ build_child_join_sjinfo(PlannerInfo *root, SpecialJoinInfo *parent_sjinfo,
* SpecialJoinInfo are freed here.
*/
static void
-free_child_join_sjinfo(SpecialJoinInfo *sjinfo)
+free_child_join_sjinfo(SpecialJoinInfo *child_sjinfo,
+ SpecialJoinInfo *parent_sjinfo)
{
/*
* Dummy SpecialJoinInfos of inner joins do not have any translated fields
* and hence no fields that to be freed.
*/
- if (sjinfo->jointype != JOIN_INNER)
+ if (child_sjinfo->jointype != JOIN_INNER)
{
- bms_free(sjinfo->min_lefthand);
- bms_free(sjinfo->min_righthand);
- bms_free(sjinfo->syn_lefthand);
- bms_free(sjinfo->syn_righthand);
+ if (child_sjinfo->min_lefthand != parent_sjinfo->min_lefthand)
+ bms_free(child_sjinfo->min_lefthand);
+
+ if (child_sjinfo->min_righthand != parent_sjinfo->min_righthand)
+ bms_free(child_sjinfo->min_righthand);
+
+ if (child_sjinfo->syn_lefthand != parent_sjinfo->syn_lefthand)
+ bms_free(child_sjinfo->syn_lefthand);
+
+ if (child_sjinfo->syn_righthand != parent_sjinfo->syn_righthand)
+ bms_free(child_sjinfo->syn_righthand);
+
+ Assert(child_sjinfo->commute_above_l == parent_sjinfo->commute_above_l);
+ Assert(child_sjinfo->commute_above_r == parent_sjinfo->commute_above_r);
+ Assert(child_sjinfo->commute_below_l == parent_sjinfo->commute_below_l);
+ Assert(child_sjinfo->commute_below_r == parent_sjinfo->commute_below_r);
+
+ Assert(child_sjinfo->semi_operators == parent_sjinfo->semi_operators);
/*
* semi_rhs_exprs may in principle be freed, but a simple pfree() does
@@ -1773,7 +1789,7 @@ free_child_join_sjinfo(SpecialJoinInfo *sjinfo)
*/
}
- pfree(sjinfo);
+ pfree(child_sjinfo);
}
/*
diff --git a/src/test/regress/expected/partition_join.out b/src/test/regress/expected/partition_join.out
index af468682a2..34b963ce6f 100644
--- a/src/test/regress/expected/partition_join.out
+++ b/src/test/regress/expected/partition_join.out
@@ -713,6 +713,41 @@ SELECT a, b FROM prt1 FULL JOIN prt2 p2(b,a,c) USING(a,b)
RESET enable_partitionwise_aggregate;
RESET enable_hashjoin;
+-- bug in freeing the SpecialJoinInfo of a child-join
+EXPLAIN (COSTS OFF)
+SELECT * FROM prt1 t1 JOIN prt1 t2 ON t1.a = t2.a WHERE t1.a IN (SELECT a FROM prt1 t3);
+ QUERY PLAN
+--------------------------------------------------
+ Append
+ -> Hash Semi Join
+ Hash Cond: (t1_1.a = t3_1.a)
+ -> Hash Join
+ Hash Cond: (t1_1.a = t2_1.a)
+ -> Seq Scan on prt1_p1 t1_1
+ -> Hash
+ -> Seq Scan on prt1_p1 t2_1
+ -> Hash
+ -> Seq Scan on prt1_p1 t3_1
+ -> Hash Semi Join
+ Hash Cond: (t1_2.a = t3_2.a)
+ -> Hash Join
+ Hash Cond: (t1_2.a = t2_2.a)
+ -> Seq Scan on prt1_p2 t1_2
+ -> Hash
+ -> Seq Scan on prt1_p2 t2_2
+ -> Hash
+ -> Seq Scan on prt1_p2 t3_2
+ -> Hash Semi Join
+ Hash Cond: (t1_3.a = t3_3.a)
+ -> Hash Join
+ Hash Cond: (t1_3.a = t2_3.a)
+ -> Seq Scan on prt1_p3 t1_3
+ -> Hash
+ -> Seq Scan on prt1_p3 t2_3
+ -> Hash
+ -> Seq Scan on prt1_p3 t3_3
+(28 rows)
+
--
-- partitioned by expression
--
diff --git a/src/test/regress/sql/partition_join.sql b/src/test/regress/sql/partition_join.sql
index e84b65f444..26b8e3d063 100644
--- a/src/test/regress/sql/partition_join.sql
+++ b/src/test/regress/sql/partition_join.sql
@@ -143,6 +143,10 @@ SELECT a, b FROM prt1 FULL JOIN prt2 p2(b,a,c) USING(a,b)
RESET enable_partitionwise_aggregate;
RESET enable_hashjoin;
+-- bug in freeing the SpecialJoinInfo of a child-join
+EXPLAIN (COSTS OFF)
+SELECT * FROM prt1 t1 JOIN prt1 t2 ON t1.a = t2.a WHERE t1.a IN (SELECT a FROM prt1 t3);
+
--
-- partitioned by expression
--
--
2.43.0
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 03:14 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
@ 2025-02-13 04:55 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-18 02:53 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Ashutosh Bapat @ 2025-02-13 04:55 UTC (permalink / raw)
To: Richard Guo <guofenglinux@gmail.com>; +Cc: Amit Langote <amitlangote09@gmail.com>; Tender Wang <tndrwang@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Thu, Feb 13, 2025 at 8:44 AM Richard Guo <guofenglinux@gmail.com> wrote:
>
> On Thu, Feb 13, 2025 at 10:39 AM Amit Langote <amitlangote09@gmail.com> wrote:
> > On Wed, Feb 12, 2025 at 11:30 PM Richard Guo <guofenglinux@gmail.com> wrote:
> > > On Wed, Feb 12, 2025 at 10:04 PM Ashutosh Bapat
> > > <ashutosh.bapat.oss@gmail.com> wrote:
> > > > Hmm. The code there assumes that all the Relids will at least have one
> > > > parent each of the children involved. For some reason
> > > > sjinfo->min_lefthand has only relid 1 but not 2 or 5. 2 and 5 are
> > > > actually the parent relids of the children passed in respectively. The
> > > > join is between 2 and 5, then why is 1 appearing in the min_lefthand.
> > > > It might be legitimate, but we need to find the reason. If it's
> > > > legitimate, I think we need to copy the Relids which haven't undergone
> > > > any translation so as to keep them isolated from the parent relids.
> > >
> > > Yes, it's legitimate. For the semijoin, its join clause only
> > > references {1} and {5}, with no other ordering restrictions.
> > > Therefore, the minimum LHS for this join consists only of {1}.
> > >
> > > Instead of copying the untranslated Relids and freeing them later, I
> > > think it might be better to modify free_child_join_sjinfo() to avoid
> > > freeing the untranslated members of child_sjinfo.
> >
> > free_child_join_sjinfo() frees min_lefthand and other fields
> > initialized by build_child_join_sjinfo(), assuming that
> > adjust_child_relids() creates a copy. However, this does not always
> > seem to be the case, as demonstrated in this report.
> >
> > I'm wondering if the following line in adjust_child_relids() should be
> > using bms_copy() instead:
> >
> > /* Otherwise, return the original set without modification. */
> > return relids;
> > }
> >
> > That is, should we copy relids not only when translation is needed but
> > also in the general case? Would that be a bigger band-aid than
> > necessary?
>
> Hmm, I'm concerned this might lead to unnecessary memory usage in
> other translation scenarios. After all, the purpose of 5278d0a2e is
> to reduce memory used by partitionwise joins. Even in the
> SpecialJoinInfo case, this approach means that we are copying
> untranslated Relids and then freeing them later, which seems
> unnecessary.
When the translations happen, we do allocate memory for child bitmaps
so allocating and freeing memory is expected in the approach. However,
we can't blatantly copy the Relids returned by adjust_child_relids()
or we will leak the memory allocated if child Relids is a translation
of parent Relids. If the Relids output by adjust_child_relids() is not
the same as its input, we should copy bitmapset.
>
> I'm thinking that a better approach might be to check if each member
> of the child_sjinfo is a translated copy and only free it if that's
> the case, something like attached.
This approach doesn't allocate memory when translation is not required
but it requires parent SJinfo to be present when
free_child_join_sjinfo() is called. It's true right now but future
changes to the code may not ensure that. So there is a potential
maintenance overhead here.
We have to be always cautious of freeing or modifying the Relids
returned by adjust_child_relids() [1]. For example, we could free
intermediate Relids in adjust_child_relids_multilevel() but we need to
be cautious of them being same as the parent Relids. Amit's approach
of bms_copy() in adjust_child_relids() frees us from that caution. May
be we could pass a flag to adjust_child_relids() asking it to create a
new bitmapset always. So even if it's a larger bandaid, it may be
useful at multiple places.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 03:14 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 04:55 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
@ 2025-02-18 02:53 ` Amit Langote <amitlangote09@gmail.com>
2025-02-18 04:22 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Amit Langote @ 2025-02-18 02:53 UTC (permalink / raw)
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; +Cc: Richard Guo <guofenglinux@gmail.com>; Tender Wang <tndrwang@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Thu, Feb 13, 2025 at 1:55 PM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
> On Thu, Feb 13, 2025 at 8:44 AM Richard Guo <guofenglinux@gmail.com> wrote:
> > I'm thinking that a better approach might be to check if each member
> > of the child_sjinfo is a translated copy and only free it if that's
> > the case, something like attached.
>
> This approach doesn't allocate memory when translation is not required
> but it requires parent SJinfo to be present when
> free_child_join_sjinfo() is called. It's true right now but future
> changes to the code may not ensure that. So there is a potential
> maintenance overhead here.
Given the function's name, I don't see a problem with adding a new
parameter to pass parent_sjinfo. In fact, I'd be especially concerned
if any caller doesn’t have parent_sjinfo handy -- that could signal an
unexpected usage or future maintenance risk. Making it explicit helps
keep things correct and avoids silent errors.
> We have to be always cautious of freeing or modifying the Relids
> returned by adjust_child_relids() [1]. For example, we could free
> intermediate Relids in adjust_child_relids_multilevel() but we need to
> be cautious of them being same as the parent Relids. Amit's approach
> of bms_copy() in adjust_child_relids() frees us from that caution. May
> be we could pass a flag to adjust_child_relids() asking it to create a
> new bitmapset always. So even if it's a larger bandaid, it may be
> useful at multiple places.
We could do this separately, but for a bug fix, I'd prefer to keep the
changes focused on the issue at hand and avoid modifications unrelated
to the bug.
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 03:14 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 04:55 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-18 02:53 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
@ 2025-02-18 04:22 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
0 siblings, 0 replies; 22+ messages in thread
From: Ashutosh Bapat @ 2025-02-18 04:22 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Richard Guo <guofenglinux@gmail.com>; Tender Wang <tndrwang@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Tue, Feb 18, 2025 at 8:23 AM Amit Langote <amitlangote09@gmail.com> wrote:
>
> On Thu, Feb 13, 2025 at 1:55 PM Ashutosh Bapat
> <ashutosh.bapat.oss@gmail.com> wrote:
> > On Thu, Feb 13, 2025 at 8:44 AM Richard Guo <guofenglinux@gmail.com> wrote:
> > > I'm thinking that a better approach might be to check if each member
> > > of the child_sjinfo is a translated copy and only free it if that's
> > > the case, something like attached.
> >
> > This approach doesn't allocate memory when translation is not required
> > but it requires parent SJinfo to be present when
> > free_child_join_sjinfo() is called. It's true right now but future
> > changes to the code may not ensure that. So there is a potential
> > maintenance overhead here.
>
> Given the function's name, I don't see a problem with adding a new
> parameter to pass parent_sjinfo. In fact, I'd be especially concerned
> if any caller doesn’t have parent_sjinfo handy -- that could signal an
> unexpected usage or future maintenance risk. Making it explicit helps
> keep things correct and avoids silent errors.
Normal SJInfos are created somewhere and put in a list while they are
passed around. In future, we may do something like that for child
SJInfos as well. That was my thinking behind this suggestion. But we
will see what happens when it comes to that.
>
> > We have to be always cautious of freeing or modifying the Relids
> > returned by adjust_child_relids() [1]. For example, we could free
> > intermediate Relids in adjust_child_relids_multilevel() but we need to
> > be cautious of them being same as the parent Relids. Amit's approach
> > of bms_copy() in adjust_child_relids() frees us from that caution. May
> > be we could pass a flag to adjust_child_relids() asking it to create a
> > new bitmapset always. So even if it's a larger bandaid, it may be
> > useful at multiple places.
>
> We could do this separately, but for a bug fix, I'd prefer to keep the
> changes focused on the issue at hand and avoid modifications unrelated
> to the bug.
That's a fair point.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
@ 2025-02-13 07:25 ` Tender Wang <tndrwang@gmail.com>
2025-02-18 05:03 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Tender Wang @ 2025-02-13 07:25 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Richard Guo <guofenglinux@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
Amit Langote <amitlangote09@gmail.com> 于2025年2月13日周四 10:39写道：
> Hi Richard,
>
> On Wed, Feb 12, 2025 at 11:30 PM Richard Guo <guofenglinux@gmail.com>
> wrote:
> > On Wed, Feb 12, 2025 at 10:04 PM Ashutosh Bapat
> > <ashutosh.bapat.oss@gmail.com> wrote:
> > > Hmm. The code there assumes that all the Relids will at least have one
> > > parent each of the children involved. For some reason
> > > sjinfo->min_lefthand has only relid 1 but not 2 or 5. 2 and 5 are
> > > actually the parent relids of the children passed in respectively. The
> > > join is between 2 and 5, then why is 1 appearing in the min_lefthand.
> > > It might be legitimate, but we need to find the reason. If it's
> > > legitimate, I think we need to copy the Relids which haven't undergone
> > > any translation so as to keep them isolated from the parent relids.
> >
> > Yes, it's legitimate. For the semijoin, its join clause only
> > references {1} and {5}, with no other ordering restrictions.
> > Therefore, the minimum LHS for this join consists only of {1}.
> >
> > Instead of copying the untranslated Relids and freeing them later, I
> > think it might be better to modify free_child_join_sjinfo() to avoid
> > freeing the untranslated members of child_sjinfo.
>
> free_child_join_sjinfo() frees min_lefthand and other fields
> initialized by build_child_join_sjinfo(), assuming that
> adjust_child_relids() creates a copy. However, this does not always
> seem to be the case, as demonstrated in this report.
>
> I'm wondering if the following line in adjust_child_relids() should be
> using bms_copy() instead:
>
> /* Otherwise, return the original set without modification. */
> return relids;
> }
>
This fix is ok for me. Keeping the child's info isolated from the parents
seems safer.
--
Thanks,
Tender Wang
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 07:25 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
@ 2025-02-18 05:03 ` Amit Langote <amitlangote09@gmail.com>
2025-02-18 06:52 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Amit Langote @ 2025-02-18 05:03 UTC (permalink / raw)
To: Tender Wang <tndrwang@gmail.com>; +Cc: Richard Guo <guofenglinux@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Thu, Feb 13, 2025 at 4:25 PM Tender Wang <tndrwang@gmail.com> wrote:
> Amit Langote <amitlangote09@gmail.com> 于2025年2月13日周四 10:39写道：
>> On Wed, Feb 12, 2025 at 11:30 PM Richard Guo <guofenglinux@gmail.com> wrote:
>> > On Wed, Feb 12, 2025 at 10:04 PM Ashutosh Bapat
>> > <ashutosh.bapat.oss@gmail.com> wrote:
>> > > Hmm. The code there assumes that all the Relids will at least have one
>> > > parent each of the children involved. For some reason
>> > > sjinfo->min_lefthand has only relid 1 but not 2 or 5. 2 and 5 are
>> > > actually the parent relids of the children passed in respectively. The
>> > > join is between 2 and 5, then why is 1 appearing in the min_lefthand.
>> > > It might be legitimate, but we need to find the reason. If it's
>> > > legitimate, I think we need to copy the Relids which haven't undergone
>> > > any translation so as to keep them isolated from the parent relids.
>> >
>> > Yes, it's legitimate. For the semijoin, its join clause only
>> > references {1} and {5}, with no other ordering restrictions.
>> > Therefore, the minimum LHS for this join consists only of {1}.
>> >
>> > Instead of copying the untranslated Relids and freeing them later, I
>> > think it might be better to modify free_child_join_sjinfo() to avoid
>> > freeing the untranslated members of child_sjinfo.
>>
>> free_child_join_sjinfo() frees min_lefthand and other fields
>> initialized by build_child_join_sjinfo(), assuming that
>> adjust_child_relids() creates a copy. However, this does not always
>> seem to be the case, as demonstrated in this report.
>>
>> I'm wondering if the following line in adjust_child_relids() should be
>> using bms_copy() instead:
>>
>> /* Otherwise, return the original set without modification. */
>> return relids;
>> }
>
> This fix is ok for me. Keeping the child's info isolated from the parents seems safer.
I'm inclined to go with Richard's proposal, mainly to avoid the
overhead of copying in a bug fix just to isolate child sjinfo from
parent_sjinfo.
Richard, would you like to commit the patch you proposed (adding
parent_sjinfo as a parameter to free_child_join_sjinfo()), or would
you prefer that I do it?
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 07:25 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-18 05:03 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
@ 2025-02-18 06:52 ` Richard Guo <guofenglinux@gmail.com>
2025-02-19 02:28 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Richard Guo @ 2025-02-18 06:52 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Tender Wang <tndrwang@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Tue, Feb 18, 2025 at 2:03 PM Amit Langote <amitlangote09@gmail.com> wrote:
> I'm inclined to go with Richard's proposal, mainly to avoid the
> overhead of copying in a bug fix just to isolate child sjinfo from
> parent_sjinfo.
>
> Richard, would you like to commit the patch you proposed (adding
> parent_sjinfo as a parameter to free_child_join_sjinfo()), or would
> you prefer that I do it?
Thank you Amit for reviewing. I'll commit it and will do so shortly.
Thanks
Richard
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 07:25 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-18 05:03 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-18 06:52 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
@ 2025-02-19 02:28 ` Richard Guo <guofenglinux@gmail.com>
2025-02-19 02:42 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-19 04:59 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
0 siblings, 2 replies; 22+ messages in thread
From: Richard Guo @ 2025-02-19 02:28 UTC (permalink / raw)
To: Amit Langote <amitlangote09@gmail.com>; +Cc: Tender Wang <tndrwang@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Tue, Feb 18, 2025 at 3:52 PM Richard Guo <guofenglinux@gmail.com> wrote:
> On Tue, Feb 18, 2025 at 2:03 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > I'm inclined to go with Richard's proposal, mainly to avoid the
> > overhead of copying in a bug fix just to isolate child sjinfo from
> > parent_sjinfo.
> >
> > Richard, would you like to commit the patch you proposed (adding
> > parent_sjinfo as a parameter to free_child_join_sjinfo()), or would
> > you prefer that I do it?
>
> Thank you Amit for reviewing. I'll commit it and will do so shortly.
Pushed and back-patched to v17. The fix will be included in the next
minor release (17.5).
Thanks
Richard
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 07:25 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-18 05:03 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-18 06:52 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-19 02:28 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
@ 2025-02-19 02:42 ` Amit Langote <amitlangote09@gmail.com>
1 sibling, 0 replies; 22+ messages in thread
From: Amit Langote @ 2025-02-19 02:42 UTC (permalink / raw)
To: Richard Guo <guofenglinux@gmail.com>; +Cc: Tender Wang <tndrwang@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Wed, Feb 19, 2025 at 11:28 AM Richard Guo <guofenglinux@gmail.com> wrote:
> On Tue, Feb 18, 2025 at 3:52 PM Richard Guo <guofenglinux@gmail.com> wrote:
> > On Tue, Feb 18, 2025 at 2:03 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > > I'm inclined to go with Richard's proposal, mainly to avoid the
> > > overhead of copying in a bug fix just to isolate child sjinfo from
> > > parent_sjinfo.
> > >
> > > Richard, would you like to commit the patch you proposed (adding
> > > parent_sjinfo as a parameter to free_child_join_sjinfo()), or would
> > > you prefer that I do it?
> >
> > Thank you Amit for reviewing. I'll commit it and will do so shortly.
>
> Pushed and back-patched to v17. The fix will be included in the next
> minor release (17.5).
Thank you!
--
Thanks, Amit Langote
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-13 07:25 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-18 05:03 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Amit Langote <amitlangote09@gmail.com>
2025-02-18 06:52 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
2025-02-19 02:28 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Richard Guo <guofenglinux@gmail.com>
@ 2025-02-19 04:59 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
1 sibling, 0 replies; 22+ messages in thread
From: Ashutosh Bapat @ 2025-02-19 04:59 UTC (permalink / raw)
To: Richard Guo <guofenglinux@gmail.com>; +Cc: Amit Langote <amitlangote09@gmail.com>; Tender Wang <tndrwang@gmail.com>; pgsql-bugs@lists.postgresql.org, m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Wed, Feb 19, 2025 at 7:58 AM Richard Guo <guofenglinux@gmail.com> wrote:
>
> On Tue, Feb 18, 2025 at 3:52 PM Richard Guo <guofenglinux@gmail.com> wrote:
> > On Tue, Feb 18, 2025 at 2:03 PM Amit Langote <amitlangote09@gmail.com> wrote:
> > > I'm inclined to go with Richard's proposal, mainly to avoid the
> > > overhead of copying in a bug fix just to isolate child sjinfo from
> > > parent_sjinfo.
> > >
> > > Richard, would you like to commit the patch you proposed (adding
> > > parent_sjinfo as a parameter to free_child_join_sjinfo()), or would
> > > you prefer that I do it?
> >
> > Thank you Amit for reviewing. I'll commit it and will do so shortly.
>
> Pushed and back-patched to v17. The fix will be included in the next
> minor release (17.5).
Thanks Richard.
--
Best Wishes,
Ashutosh Bapat
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 09:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Re: BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally Tender Wang <tndrwang@gmail.com>
@ 2025-02-12 13:55 ` Richard Guo <guofenglinux@gmail.com>
1 sibling, 0 replies; 22+ messages in thread
From: Richard Guo @ 2025-02-12 13:55 UTC (permalink / raw)
To: Tender Wang <tndrwang@gmail.com>; +Cc: pgsql-bugs@lists.postgresql.org, Amit Langote <amitlangote09@gmail.com>; Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>; m_lingbin@126.com, Laurenz Albe <laurenz.albe@cybertec.at>
On Wed, Feb 12, 2025 at 7:48 PM Tender Wang <tndrwang@gmail.com> wrote:
> Tender Wang <tndrwang@gmail.com> 于2025年2月12日周三 17:48写道：
> When we create join rel for (2,5), then wen enter try_partitionwise_join().
> In this func, we call build_child_join_sjinfo() to build SpecialJoinInfo for child.
> In build_child_join_sjinfo(), we use memcpy(), I think we should use copyObject().
> Otherwise, the parent_sjinfo will be freed, for example, after call free_child_join_sjinfo(),
> the parent_sjinfo min_lefthand looks like as below:
>
> (gdb) p *parent_sjinfo->min_lefthand
> $7 = {type = 3951489872, nwords = 21994, words = 0x55eaeb8696b0}
>
> The above bitmap is invalid, so trigger the Assert(bms_is_valid_set(a));
Nice catch! We should avoid freeing the members of child_sjinfo
unless they are translated copies of their counterparts in
parent_sjinfo.
Thanks
Richard
^ permalink raw reply [nested|flat] 22+ messages in thread
end of thread, other threads:[~2025-02-19 04:59 UTC | newest]
Thread overview: 22+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-02-12 07:01 BUG #18806: When enable_rartitionwise_join is set to ON, the database shuts down abnormally PG Bug reporting form <noreply@postgresql.org>
2025-02-12 07:38 ` Laurenz Albe <laurenz.albe@cybertec.at>
2025-02-12 08:55 ` =?GBK?B?w8/B7rHy?= <m_lingbin@126.com>
2025-02-12 09:48 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 11:48 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 12:11 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 12:20 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 12:33 ` Tender Wang <tndrwang@gmail.com>
2025-02-12 14:04 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 14:30 ` Richard Guo <guofenglinux@gmail.com>
2025-02-13 02:39 ` Amit Langote <amitlangote09@gmail.com>
2025-02-13 03:14 ` Richard Guo <guofenglinux@gmail.com>
2025-02-13 04:55 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-18 02:53 ` Amit Langote <amitlangote09@gmail.com>
2025-02-18 04:22 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-13 07:25 ` Tender Wang <tndrwang@gmail.com>
2025-02-18 05:03 ` Amit Langote <amitlangote09@gmail.com>
2025-02-18 06:52 ` Richard Guo <guofenglinux@gmail.com>
2025-02-19 02:28 ` Richard Guo <guofenglinux@gmail.com>
2025-02-19 02:42 ` Amit Langote <amitlangote09@gmail.com>
2025-02-19 04:59 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2025-02-12 13:55 ` Richard Guo <guofenglinux@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox