pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Reduce memory used by partitionwise joins 2+ messages / 2 participants [nested] [flat]
* pgsql: Reduce memory used by partitionwise joins @ 2024-03-25 09:07 Amit Langote <amitlan@postgresql.org> 0 siblings, 0 replies; 2+ messages in thread From: Amit Langote @ 2024-03-25 09:07 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Reduce memory used by partitionwise joins Specifically, this commit reduces the memory consumed by the SpecialJoinInfos that are allocated for child joins in try_partitionwise_join() by freeing them at the end of creating paths for each child join. A SpecialJoinInfo allocated for a given child join is a copy of the parent join's SpecialJoinInfo, which contains the translated copies of the various Relids bitmapsets and semi_rhs_exprs, which is a List of Nodes. The newly added freeing step frees the struct itself and the various bitmapsets, but not semi_rhs_exprs, because there's no handy function to free the memory of Node trees. Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Richard Guo <guofenglinux@gmail.com> Reviewed-by: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Andrey Lepikhov <a.lepikhov@postgrespro.ru> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Discussion: https://postgr.es/m/CAExHW5tHqEf3ASVqvFFcghYGPfpy7o3xnvhHwBGbJFMRH8KjNw@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/5278d0a2e870c61f9374a7796b90e6f9f6a73638 Modified Files -------------- src/backend/optimizer/path/joinrels.c | 36 +++++++++++++++++++++++++++++++++++ src/backend/optimizer/util/pathnode.c | 5 +++-- src/include/nodes/pathnodes.h | 3 +++ 3 files changed, 42 insertions(+), 2 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
* pgsql: Reduce memory used by partitionwise joins @ 2024-07-29 02:37 Richard Guo <rguo@postgresql.org> 0 siblings, 0 replies; 2+ messages in thread From: Richard Guo @ 2024-07-29 02:37 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Reduce memory used by partitionwise joins In try_partitionwise_join, we aim to break down the join between two partitioned relations into joins between matching partitions. To achieve this, we iterate through each pair of partitions from the two joining relations and create child-join relations for them. With potentially thousands of partitions, the local objects allocated in each iteration can accumulate significant memory usage. Therefore, we opt to eagerly free these local objects at the end of each iteration. In line with this approach, this patch frees the bitmap set that represents the relids of child-join relations at the end of each iteration. Additionally, it modifies build_child_join_rel() to reuse the AppendRelInfo structures generated within each iteration. Author: Ashutosh Bapat Reviewed-by: David Christensen, Richard Guo Discussion: https://postgr.es/m/CAExHW5s4EqY43oB=ne6B2=-xLgrs9ZGeTr1NXwkGFt2j-OmaQQ@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/513f4472a4a0d294ca64123627ce7b48ce0ee7c1 Modified Files -------------- src/backend/optimizer/path/joinrels.c | 15 +++++++++++---- src/backend/optimizer/util/relnode.c | 18 +++--------------- src/include/optimizer/pathnode.h | 3 ++- 3 files changed, 16 insertions(+), 20 deletions(-) ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-07-29 02:37 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-03-25 09:07 pgsql: Reduce memory used by partitionwise joins Amit Langote <amitlan@postgresql.org> 2024-07-29 02:37 pgsql: Reduce memory used by partitionwise joins Richard Guo <rguo@postgresql.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox