Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lF4dw-0007qL-BI for pgsql-bugs@arkaria.postgresql.org; Thu, 25 Feb 2021 00:36:36 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lF4du-0004IS-4c for pgsql-bugs@arkaria.postgresql.org; Thu, 25 Feb 2021 00:36:34 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lF4dt-0004IK-V1 for pgsql-bugs@lists.postgresql.org; Thu, 25 Feb 2021 00:36:33 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lF4do-0003lv-Vz for pgsql-bugs@lists.postgresql.org; Thu, 25 Feb 2021 00:36:32 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 11P0aQ2c031725; Wed, 24 Feb 2021 19:36:26 -0500 From: Tom Lane To: Michael Paquier cc: Alexander Lakhin , pgsql-bugs@lists.postgresql.org Subject: Re: BUG #16801: Invalid memory access on WITH RECURSIVE with nested WITHs In-reply-to: <4057243.1614179963@sss.pgh.pa.us> References: <16801-393c7922143eaa4d@postgresql.org> <4057243.1614179963@sss.pgh.pa.us> Comments: In-reply-to Tom Lane message dated "Wed, 24 Feb 2021 10:19:23 -0500" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0" Content-ID: <31699.1614213356.0@sss.pgh.pa.us> Date: Wed, 24 Feb 2021 19:36:26 -0500 Message-ID: <31724.1614213386@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk ------- =_aaaaaaaaaa0 Content-Type: text/plain; charset="us-ascii" Content-ID: <31699.1614213356.1@sss.pgh.pa.us> I wrote: > Surely that breaks things entirely (if it doesn't, then we are badly > under-testing this area). A nil list is just a null pointer, so > appending to "new_cte_list" later isn't going to affect what was > previously put into the innerwiths list. Here's a patch that I think fixes it correctly, including a test case that doesn't work with your patch. regards, tom lane ------- =_aaaaaaaaaa0 Content-Type: text/x-diff; name="bug-16801-fix.patch"; charset="us-ascii" Content-ID: <31699.1614213356.2@sss.pgh.pa.us> Content-Description: bug-16801-fix.patch Content-Transfer-Encoding: quoted-printable diff --git a/src/backend/parser/parse_cte.c b/src/backend/parser/parse_cte= .c index f4f7041ead..f46d63d451 100644 --- a/src/backend/parser/parse_cte.c +++ b/src/backend/parser/parse_cte.c @@ -730,15 +730,15 @@ makeDependencyGraphWalker(Node *node, CteState *csta= te) * In the non-RECURSIVE case, query names are visible to the * WITH items after them and to the main query. */ - ListCell *cell1; - cstate->innerwiths =3D lcons(NIL, cstate->innerwiths); - cell1 =3D list_head(cstate->innerwiths); foreach(lc, stmt->withClause->ctes) { CommonTableExpr *cte =3D (CommonTableExpr *) lfirst(lc); + ListCell *cell1; = (void) makeDependencyGraphWalker(cte->ctequery, cstate); + /* note that recursion could mutate innerwiths list */ + cell1 =3D list_head(cstate->innerwiths); lfirst(cell1) =3D lappend((List *) lfirst(cell1), cte); } (void) raw_expression_tree_walker(node, @@ -1006,15 +1006,15 @@ checkWellFormedRecursionWalker(Node *node, CteStat= e *cstate) * In the non-RECURSIVE case, query names are visible to the * WITH items after them and to the main query. */ - ListCell *cell1; - cstate->innerwiths =3D lcons(NIL, cstate->innerwiths); - cell1 =3D list_head(cstate->innerwiths); foreach(lc, stmt->withClause->ctes) { CommonTableExpr *cte =3D (CommonTableExpr *) lfirst(lc); + ListCell *cell1; = (void) checkWellFormedRecursionWalker(cte->ctequery, cstate); + /* note that recursion could mutate innerwiths list */ + cell1 =3D list_head(cstate->innerwiths); lfirst(cell1) =3D lappend((List *) lfirst(cell1), cte); } checkWellFormedSelectStmt(stmt, cstate); diff --git a/src/test/regress/expected/with.out b/src/test/regress/expecte= d/with.out index c519a61c4f..be4ba1825f 100644 --- a/src/test/regress/expected/with.out +++ b/src/test/regress/expected/with.out @@ -176,6 +176,29 @@ ERROR: operator does not exist: text + integer LINE 4: SELECT n+1 FROM t WHERE n < 10 ^ HINT: No operator matches the given name and argument types. You might n= eed to add explicit type casts. +-- Deeply nested WITH caused a list-munging problem in v13 +with recursive w1(c1) as + (with w2(c2) as + (with w3(c3) as + (with w4(c4) as + (with w5(c5) as + (with recursive w6(c6) as + (with w6(c6) as + (with w8(c8) as + (select 1) + select * from w8) + select * from w6) + select * from w6) + select * from w5) + select * from w4) + select * from w3) + select * from w2) +select * from w1; + c1 = +---- + 1 +(1 row) + -- -- Some examples with a tree -- diff --git a/src/test/regress/sql/with.sql b/src/test/regress/sql/with.sql index f4ba0d8e39..b2f9ac07e8 100644 --- a/src/test/regress/sql/with.sql +++ b/src/test/regress/sql/with.sql @@ -95,6 +95,25 @@ UNION ALL ) SELECT n, pg_typeof(n) FROM t; = +-- Deeply nested WITH caused a list-munging problem in v13 +with recursive w1(c1) as + (with w2(c2) as + (with w3(c3) as + (with w4(c4) as + (with w5(c5) as + (with recursive w6(c6) as + (with w6(c6) as + (with w8(c8) as + (select 1) + select * from w8) + select * from w6) + select * from w6) + select * from w5) + select * from w4) + select * from w3) + select * from w2) +select * from w1; + -- -- Some examples with a tree -- ------- =_aaaaaaaaaa0--