pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alexander Lakhin <exclusion@gmail.com>
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16801: Invalid memory access on WITH RECURSIVE with nested WITHs
Date: Fri, 26 Feb 2021 10:17:50 +0900
Message-ID: <YDhMPvDL4iLP6gLo@paquier.xyz> (raw)
In-Reply-To: <119550.1614266383@sss.pgh.pa.us>
References: <eb358b3f-8272-d0a9-6645-983e7ad2bec4@gmail.com>
	<YDXeLBqjDeQQPGoO@paquier.xyz>
	<4057243.1614179963@sss.pgh.pa.us>
	<31724.1614213386@sss.pgh.pa.us>
	<YDb9wdB1GUkr+NAD@paquier.xyz>
	<51877.1614219226@sss.pgh.pa.us>
	<YDcLzKuQJmKOZYq+@paquier.xyz>
	<60272.1614221283@sss.pgh.pa.us>
	<YDcVh8R7OArj9dtV@paquier.xyz>
	<119550.1614266383@sss.pgh.pa.us>

On Thu, Feb 25, 2021 at 10:19:43AM -0500, Tom Lane wrote:
> For me, the example I gave fails in a non-debug build.  With the code as
> of HEAD, I get "ERROR: stack depth limit exceeded" or a segfault.  With
> your patch applied, it complains about w6 not having the correct form for
> a recursive CTE.

Yes, same here for the test stressing makeDependencyGraphWalker().

The second test I posted for checkWellFormedRecursionWalker() passes
on HEAD in a non-debug build, and triggers an assertion with debug
builds.  So the second test requires CLOBBER_FREED_MEMORY but not
DEBUG_LIST_MEMORY_USAGE.  That's not as good as the first one, but I
would vote for having this second test than none to stress more the
list handling when looking after invalid self-references in a CTE.
This gives me the attached.

I would rather have both tests in the version committed, but if you
think that this addition is not necessary I won't fight hard either :)
--
Michael

Attachments:

bug-16801-fix-v2.patch (text/x-diff, 4.7K) download diff --git a/src/backend/parser/parse_cte.c b/src/backend/parser/parse_cte.c index f4f7041ead..f46d63d451 100644 --- a/src/backend/parser/parse_cte.c +++ b/src/backend/parser/parse_cte.c @@ -730,15 +730,15 @@ makeDependencyGraphWalker(Node *node, CteState *cstate) * In the non-RECURSIVE case, query names are visible to the * WITH items after them and to the main query. */ - ListCell *cell1; - cstate->innerwiths = lcons(NIL, cstate->innerwiths); - cell1 = list_head(cstate->innerwiths); foreach(lc, stmt->withClause->ctes) { CommonTableExpr *cte = (CommonTableExpr *) lfirst(lc); + ListCell *cell1; (void) makeDependencyGraphWalker(cte->ctequery, cstate); + /* note that recursion could mutate innerwiths list */ + cell1 = list_head(cstate->innerwiths); lfirst(cell1) = lappend((List *) lfirst(cell1), cte); } (void) raw_expression_tree_walker(node, @@ -1006,15 +1006,15 @@ checkWellFormedRecursionWalker(Node *node, CteState *cstate) * In the non-RECURSIVE case, query names are visible to the * WITH items after them and to the main query. */ - ListCell *cell1; - cstate->innerwiths = lcons(NIL, cstate->innerwiths); - cell1 = list_head(cstate->innerwiths); foreach(lc, stmt->withClause->ctes) { CommonTableExpr *cte = (CommonTableExpr *) lfirst(lc); + ListCell *cell1; (void) checkWellFormedRecursionWalker(cte->ctequery, cstate); + /* note that recursion could mutate innerwiths list */ + cell1 = list_head(cstate->innerwiths); lfirst(cell1) = lappend((List *) lfirst(cell1), cte); } checkWellFormedSelectStmt(stmt, cstate); diff --git a/src/test/regress/expected/with.out b/src/test/regress/expected/with.out index c519a61c4f..a7a652822c 100644 --- a/src/test/regress/expected/with.out +++ b/src/test/regress/expected/with.out @@ -176,6 +176,65 @@ ERROR: operator does not exist: text + integer LINE 4: SELECT n+1 FROM t WHERE n < 10 ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts. +-- Deeply nested WITH caused a list-munging problem in v13 +-- Detection of cross-references and self-references +WITH RECURSIVE w1(c1) AS + (WITH w2(c2) AS + (WITH w3(c3) AS + (WITH w4(c4) AS + (WITH w5(c5) AS + (WITH RECURSIVE w6(c6) AS + (WITH w6(c6) AS + (WITH w8(c8) AS + (SELECT 1) + SELECT * FROM w8) + SELECT * FROM w6) + SELECT * FROM w6) + SELECT * FROM w5) + SELECT * FROM w4) + SELECT * FROM w3) + SELECT * FROM w2) +SELECT * FROM w1; + c1 +---- + 1 +(1 row) + +-- Detection of invalid self-references +WITH RECURSIVE outermost(x) AS ( + SELECT 1 + UNION (WITH innermost1 AS ( + SELECT 2 + UNION (WITH innermost2 AS ( + SELECT 3 + UNION (WITH innermost3 AS ( + SELECT 4 + UNION (WITH innermost4 AS ( + SELECT 5 + UNION (WITH innermost5 AS ( + SELECT 6 + UNION (WITH innermost6 AS + (SELECT 7) + SELECT * FROM innermost6)) + SELECT * FROM innermost5)) + SELECT * FROM innermost4)) + SELECT * FROM innermost3)) + SELECT * FROM innermost2)) + SELECT * FROM outermost + UNION SELECT * FROM innermost1) + ) + SELECT * FROM outermost ORDER BY 1; + x +--- + 1 + 2 + 3 + 4 + 5 + 6 + 7 +(7 rows) + -- -- Some examples with a tree -- diff --git a/src/test/regress/sql/with.sql b/src/test/regress/sql/with.sql index f4ba0d8e39..85a671c630 100644 --- a/src/test/regress/sql/with.sql +++ b/src/test/regress/sql/with.sql @@ -95,6 +95,50 @@ UNION ALL ) SELECT n, pg_typeof(n) FROM t; +-- Deeply nested WITH caused a list-munging problem in v13 +-- Detection of cross-references and self-references +WITH RECURSIVE w1(c1) AS + (WITH w2(c2) AS + (WITH w3(c3) AS + (WITH w4(c4) AS + (WITH w5(c5) AS + (WITH RECURSIVE w6(c6) AS + (WITH w6(c6) AS + (WITH w8(c8) AS + (SELECT 1) + SELECT * FROM w8) + SELECT * FROM w6) + SELECT * FROM w6) + SELECT * FROM w5) + SELECT * FROM w4) + SELECT * FROM w3) + SELECT * FROM w2) +SELECT * FROM w1; +-- Detection of invalid self-references +WITH RECURSIVE outermost(x) AS ( + SELECT 1 + UNION (WITH innermost1 AS ( + SELECT 2 + UNION (WITH innermost2 AS ( + SELECT 3 + UNION (WITH innermost3 AS ( + SELECT 4 + UNION (WITH innermost4 AS ( + SELECT 5 + UNION (WITH innermost5 AS ( + SELECT 6 + UNION (WITH innermost6 AS + (SELECT 7) + SELECT * FROM innermost6)) + SELECT * FROM innermost5)) + SELECT * FROM innermost4)) + SELECT * FROM innermost3)) + SELECT * FROM innermost2)) + SELECT * FROM outermost + UNION SELECT * FROM innermost1) + ) + SELECT * FROM outermost ORDER BY 1; + -- -- Some examples with a tree --
signature.asc (application/pgp-signature, 832B) download

view thread (18+ messages)  latest in thread

Message-ID: <YDhMPvDL4iLP6gLo@paquier.xyz>
Permalink:  ../YDhMPvDL4iLP6gLo@paquier.xyz/
Also on:    postgresql.org/message-id/YDhMPvDL4iLP6gLo@paquier.xyz

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: michael@paquier.xyz, tgl@sss.pgh.pa.us, exclusion@gmail.com, pgsql-bugs@lists.postgresql.org
  Subject: Re: BUG #16801: Invalid memory access on WITH RECURSIVE with nested WITHs
  In-Reply-To: <YDhMPvDL4iLP6gLo@paquier.xyz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox