Date: Thu, 3 Mar 2022 16:11:56 +0100
From: hubert depesz lubaczewski <depesz@depesz.com>
Sender: depesz@depesz.com
To: PostgreSQL General <pgsql-general@lists.postgresql.org>
Subject: Re: Interesting fail when migrating Pg from Ubuntu Bionic to Focal
Message-ID: <20220303151156.GB26036@depesz.com>
Reply-To: depesz@depesz.com
References: <20220303150428.GA26036@depesz.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20220303150428.GA26036@depesz.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Archived-At: 
 <https://www.postgresql.org/message-id/20220303151156.GB26036%40depesz.com>
Precedence: bulk

On Thu, Mar 03, 2022 at 04:04:28PM +0100, hubert depesz lubaczewski wrote:
> and it worked, so I'm kinda at loss here.

based on some talk on IRC, I was able to get stack trace from fail:

(gdb) bt
#0  0x0000fffe4a36e4d8 in ?? ()
#1  0x0000aaaabe03ffb8 in ExecProcNode (node=0xaaaae4f87cf8) at ./build/../src/include/executor/executor.h:242
#2  ExecMergeJoin (pstate=0xaaaae4f87da8) at ./build/../src/backend/executor/nodeMergejoin.c:656
#3  0x0000aaaabe03ffb8 in ExecProcNode (node=0xaaaae4f87ae8) at ./build/../src/include/executor/executor.h:242
#4  ExecMergeJoin (pstate=0xaaaae4f876c8) at ./build/../src/backend/executor/nodeMergejoin.c:656
#5  0x0000aaaabe039b1c in ExecProcNode (node=0xaaaae4f876c8) at ./build/../src/include/executor/executor.h:242
#6  ExecHashJoinImpl (parallel=false, pstate=0xaaaae4f87408) at ./build/../src/backend/executor/nodeHashjoin.c:262
#7  ExecHashJoin (pstate=0xaaaae4f87408) at ./build/../src/backend/executor/nodeHashjoin.c:591
#8  0x0000aaaabe01ed5c in ExecProcNode (node=0xaaaae4f87408) at ./build/../src/include/executor/executor.h:242
#9  ExecutePlan (execute_once=<optimized out>, dest=0xfffe49be0828, direction=<optimized out>, numberTuples=0, sendTuples=<optimized out>, operation=CMD_SELECT, use_parallel_mode=<optimized out>, planstate=0xaaaae4f87408, estate=0xaaaae4ce4180)
    at ./build/../src/backend/executor/execMain.c:1632
#10 standard_ExecutorRun (queryDesc=0xaaaae4e9da70, direction=<optimized out>, count=0, execute_once=<optimized out>) at ./build/../src/backend/executor/execMain.c:350
#11 0x0000ffff7d1981fc in pgss_ExecutorRun (queryDesc=0xaaaae4e9da70, direction=ForwardScanDirection, count=0, execute_once=true) at ./build/../contrib/pg_stat_statements/pg_stat_statements.c:893
#12 0x0000ffff7d182688 in explain_ExecutorRun (queryDesc=0xaaaae4e9da70, direction=ForwardScanDirection, count=0, execute_once=true) at ./build/../contrib/auto_explain/auto_explain.c:320
#13 0x0000aaaabe1754c8 in PortalRunSelect (portal=portal@entry=0xaaaae4d476e0, forward=forward@entry=true, count=0, count@entry=9223372036854775807, dest=dest@entry=0xfffe49be0828) at ./build/../src/backend/tcop/pquery.c:938
#14 0x0000aaaabe176a64 in PortalRun (portal=portal@entry=0xaaaae4d476e0, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0xfffe49be0828, altdest=altdest@entry=0xfffe49be0828, 
    completionTag=completionTag@entry=0xffffc89001f8 "") at ./build/../src/backend/tcop/pquery.c:779
#15 0x0000aaaabe172a18 in exec_simple_query (
    query_string=query_string@entry=0xaaaae4c6c7b0 "SELECT\n    p.tableoid,\n    p.oid,\n    p.proname,\n    p.prolang,\n    p.pronargs,\n    p.proargtypes,\n    p.prorettype,\n    (\n        SELECT\n", ' ' <repeats 12 times>, "pg_catalog.array_agg(acl ORDER BY row_n)\n        F"...) at ./build/../src/backend/tcop/postgres.c:1215
#16 0x0000aaaabe1736ac in PostgresMain (argc=<optimized out>, argv=argv@entry=0xaaaae4cc9018, dbname=<optimized out>, username=<optimized out>) at ./build/../src/backend/tcop/postgres.c:4271
#17 0x0000aaaabe0fa768 in BackendRun (port=0xaaaae4cb88e0, port=0xaaaae4cb88e0) at ./build/../src/backend/postmaster/postmaster.c:4510
#18 BackendStartup (port=0xaaaae4cb88e0) at ./build/../src/backend/postmaster/postmaster.c:4193
#19 ServerLoop () at ./build/../src/backend/postmaster/postmaster.c:1725
#20 0x0000aaaabe0fb74c in PostmasterMain (argc=<optimized out>, argv=<optimized out>) at ./build/../src/backend/postmaster/postmaster.c:1398
#21 0x0000aaaabde8c8a8 in main (argc=5, argv=0xaaaae4c65ea0) at ./build/../src/backend/main/main.c:228

And one more thing - if it matters - it's on aarch64 architecture
(graviton ec2 in AWS)

Best regards,

depesz