Content-Type: multipart/alternative;
 boundary="------------5hu4UE4Uv0fp0z7CM9Ni0e2t"
Message-ID: <4103db46-d888-6d1d-e88d-87c21ed99472@gmail.com>
Date: Mon, 13 Feb 2023 13:00:00 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Subject: Re: array_agg(DISTINCT) caused a segmentation fault
To: David Rowley <dgrowleyml@gmail.com>,
 Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: pgsql-bugs@lists.postgresql.org
References: <860c6d6f-a3c5-3ae9-9da2-827177bede06@oss.nttdata.com>
 <CAApHDvrWu0G1caebi21=pELRtpcyu+1sDvo_c-vkogYJ=tQrvA@mail.gmail.com>
From: Alexander Lakhin <exclusion@gmail.com>
Content-Language: en-US
In-Reply-To: 
 <CAApHDvrWu0G1caebi21=pELRtpcyu+1sDvo_c-vkogYJ=tQrvA@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/4103db46-d888-6d1d-e88d-87c21ed99472%40gmail.com>
Precedence: bulk

This is a multi-part message in MIME format.
--------------5hu4UE4Uv0fp0z7CM9Ni0e2t
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hello David,

13.02.2023 10:44, David Rowley wrote:
> On Mon, 13 Feb 2023 at 18:29, Fujii Masao<masao.fujii@oss.nttdata.com>  wrote:
>> =# SELECT array_agg(distinct val) FROM (SELECT NULL AS val FROM generate_series(1, 2)) hoge;
>> LOG:  server process (PID 76507) was terminated by signal 11: Segmentation fault: 11
>> DETAIL:  Failed process was running: SELECT array_agg(distinct val) FROM (SELECT NULL AS val FROM generate_series(1, 2)) hoge;
> I've now pushed a fix for it and included your test.  To get it to
> crash it needed to be a byref aggregate without a strict transition
> function.  There are not too many of those, which is probably why
> nobody noticed this before.
I've encountered an issue that could have the same title but it still 
reproduced after the fix.
The following query:
SELECT array_agg(DISTINCT a ORDER BY a DESC)
           FROM (VALUES (1),(1.0),(NULL)) v(a);

Invokes valgrind-detected error:
==00:00:00:03.708 2686358== Invalid read of size 4
==00:00:00:03.708 2686358==    at 0x76C4AE: GetMemoryChunkMethodID 
(mcxt.c:195)
==00:00:00:03.708 2686358==    by 0x76C4AE: pfree (mcxt.c:1439)
==00:00:00:03.708 2686358==    by 0x3FD547: 
ExecEvalPreOrderedDistinctSingle (execExprInterp.c:4258)
==00:00:00:03.708 2686358==    by 0x3FF203: ExecInterpExpr 
(execExprInterp.c:1772)
==00:00:00:03.708 2686358==    by 0x418792: ExecEvalExprSwitchContext 
(executor.h:344)
==00:00:00:03.708 2686358==    by 0x418792: advance_aggregates 
(nodeAgg.c:823)
==00:00:00:03.708 2686358==    by 0x41A12A: agg_retrieve_direct 
(nodeAgg.c:2446)
==00:00:00:03.708 2686358==    by 0x41A294: ExecAgg (nodeAgg.c:2171)
==00:00:00:03.708 2686358==    by 0x40AD3F: ExecProcNodeFirst 
(execProcnode.c:464)
==00:00:00:03.708 2686358==    by 0x40337F: ExecProcNode (executor.h:262)
==00:00:00:03.708 2686358==    by 0x40337F: ExecutePlan (execMain.c:1633)
==00:00:00:03.708 2686358==    by 0x403542: standard_ExecutorRun 
(execMain.c:364)
==00:00:00:03.708 2686358==    by 0x40360E: ExecutorRun (execMain.c:308)
==00:00:00:03.708 2686358==    by 0x5EB971: PortalRunSelect (pquery.c:924)
==00:00:00:03.708 2686358==    by 0x5ED31B: PortalRun (pquery.c:768)
==00:00:00:03.708 2686358==  Address 0xfffffffffffffff8 is not stack'd, 
malloc'd or (recently) free'd
==00:00:00:03.708 2686358==
...
==00:00:00:03.708 2686358==
==00:00:00:03.708 2686358== Exit program on first error 
(--exit-on-first-error=yes)
2023-02-13 10:26:39.276 MSK [2686332] LOG:  server process (PID 2686358) 
exited with exit code 1
2023-02-13 10:26:39.276 MSK [2686332] DETAIL:  Failed process was 
running: SELECT array_agg(DISTINCT a ORDER BY a DESC)
               FROM (VALUES (1),(1.0),(NULL)) v(a);

(Without valgrind I get SIGSEGV here.)
The first bad commit is 1349d2790 again (but before 80ef92675 an 
assertion failure can be seen).

Best regards,
Alexander
--------------5hu4UE4Uv0fp0z7CM9Ni0e2t
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <div class="moz-cite-prefix">Hello David,<br>
      <br>
      13.02.2023 10:44, David Rowley wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAApHDvrWu0G1caebi21=pELRtpcyu+1sDvo_c-vkogYJ=tQrvA@mail.gmail.com">
      <pre class="moz-quote-pre" wrap="">On Mon, 13 Feb 2023 at 18:29, Fujii Masao <a class="moz-txt-link-rfc2396E" href="mailto:masao.fujii@oss.nttdata.com">&lt;masao.fujii@oss.nttdata.com&gt;</a> wrote:
</pre>
      <blockquote type="cite">
        <pre class="moz-quote-pre" wrap="">=# SELECT array_agg(distinct val) FROM (SELECT NULL AS val FROM generate_series(1, 2)) hoge;
LOG:  server process (PID 76507) was terminated by signal 11: Segmentation fault: 11
DETAIL:  Failed process was running: SELECT array_agg(distinct val) FROM (SELECT NULL AS val FROM generate_series(1, 2)) hoge;
</pre>
      </blockquote>
      <pre class="moz-quote-pre" wrap="">
I've now pushed a fix for it and included your test.  To get it to
crash it needed to be a byref aggregate without a strict transition
function.  There are not too many of those, which is probably why
nobody noticed this before.
</pre>
    </blockquote>
    I've encountered an issue that could have the same title but it
    still reproduced after the fix.<br>
    The following query:<br>
    SELECT array_agg(DISTINCT a ORDER BY a DESC)<br>
              FROM (VALUES (1),(1.0),(NULL)) v(a);<br>
    <br>
    Invokes valgrind-detected error:<br>
    <font face="monospace">==00:00:00:03.708 2686358== Invalid read of
      size 4<br>
      ==00:00:00:03.708 2686358==    at 0x76C4AE: GetMemoryChunkMethodID
      (mcxt.c:195)<br>
      ==00:00:00:03.708 2686358==    by 0x76C4AE: pfree (mcxt.c:1439)<br>
      ==00:00:00:03.708 2686358==    by 0x3FD547:
      ExecEvalPreOrderedDistinctSingle (execExprInterp.c:4258)<br>
      ==00:00:00:03.708 2686358==    by 0x3FF203: ExecInterpExpr
      (execExprInterp.c:1772)<br>
      ==00:00:00:03.708 2686358==    by 0x418792:
      ExecEvalExprSwitchContext (executor.h:344)<br>
      ==00:00:00:03.708 2686358==    by 0x418792: advance_aggregates
      (nodeAgg.c:823)<br>
      ==00:00:00:03.708 2686358==    by 0x41A12A: agg_retrieve_direct
      (nodeAgg.c:2446)<br>
      ==00:00:00:03.708 2686358==    by 0x41A294: ExecAgg
      (nodeAgg.c:2171)<br>
      ==00:00:00:03.708 2686358==    by 0x40AD3F: ExecProcNodeFirst
      (execProcnode.c:464)<br>
      ==00:00:00:03.708 2686358==    by 0x40337F: ExecProcNode
      (executor.h:262)<br>
      ==00:00:00:03.708 2686358==    by 0x40337F: ExecutePlan
      (execMain.c:1633)<br>
      ==00:00:00:03.708 2686358==    by 0x403542: standard_ExecutorRun
      (execMain.c:364)<br>
      ==00:00:00:03.708 2686358==    by 0x40360E: ExecutorRun
      (execMain.c:308)<br>
      ==00:00:00:03.708 2686358==    by 0x5EB971: PortalRunSelect
      (pquery.c:924)<br>
      ==00:00:00:03.708 2686358==    by 0x5ED31B: PortalRun
      (pquery.c:768)<br>
      ==00:00:00:03.708 2686358==  Address 0xfffffffffffffff8 is not
      stack'd, malloc'd or (recently) free'd<br>
      ==00:00:00:03.708 2686358==<br>
      ...<br>
      ==00:00:00:03.708 2686358==<br>
      ==00:00:00:03.708 2686358== Exit program on first error
      (--exit-on-first-error=yes)<br>
      2023-02-13 10:26:39.276 MSK [2686332] LOG:  server process (PID
      2686358) exited with exit code 1<br>
      2023-02-13 10:26:39.276 MSK [2686332] DETAIL:  Failed process was
      running: SELECT array_agg(DISTINCT a ORDER BY a DESC)<br>
                    FROM (VALUES (1),(1.0),(NULL)) v(a);<br>
    </font><br>
    (Without valgrind I get SIGSEGV here.)<br>
    The first bad commit is 1349d2790 again (but before 80ef92675 an
    assertion failure can be seen).<br>
    <br>
    Best regards,<br>
    Alexander<span style="font-family:monospace"><br>
    </span>
  </body>
</html>

--------------5hu4UE4Uv0fp0z7CM9Ni0e2t--