Message-ID: <8e676a9a-1c61-b9b9-714b-8f4e37eaab26@gmail.com>
Date: Thu, 15 Aug 2024 18:00:00 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Subject: Re: Remove dependence on integer wrapping
Content-Language: en-US
To: Joseph Koshakow <koshy44@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Andres Freund <andres@anarazel.de>, Nathan Bossart <nathandbossart@gmail.com>
References: 
 <CAAvxfHdBPOyEGS7s+xf4iaW0-cgiq25jpYdWBqQqvLtLe_t6tw@mail.gmail.com>
 <ZmZbXK1CNMbkty0W@nathan> <618777.1717984674@sss.pgh.pa.us>
 <31ad2cd1-db94-bdb3-f91a-65ffdb4bef95@gmail.com>
 <CAAvxfHfqJca-WGvWJQ3ZAtt2po=mkQzU8r9Qjzom4YqxQi84oA@mail.gmail.com>
 <d7a0ed64-c21c-79c7-42be-c621b4609564@gmail.com>
 <CAAvxfHeWy-rQpqAkc60WmcaMefi1_tMZptrzCutW8xY5Uc1LUA@mail.gmail.com>
From: Alexander Lakhin <exclusion@gmail.com>
In-Reply-To: 
 <CAAvxfHeWy-rQpqAkc60WmcaMefi1_tMZptrzCutW8xY5Uc1LUA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/8e676a9a-1c61-b9b9-714b-8f4e37eaab26%40gmail.com>
Precedence: bulk

Hello Joe,

05.08.2024 02:55, Joseph Koshakow wrote:
>
> On Fri, Jun 14, 2024 at 8:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
> >
> >    And the most interesting case to me:
> >    SET temp_buffers TO 1000000000;
> >
> >    CREATE TEMP TABLE t(i int PRIMARY KEY);
> >    INSERT INTO t VALUES(1);
> >
> ...
> Alex, are you able to get a full stack trace for this panic? I'm unable
> to reproduce this because I don't have enough memory in my system. I've
> tried reducing `BLCKSZ` to 1024, which is the lowest value allowed per
> my understanding, and I still don't have enough memory.

Yes, please take a look at it (sorry for the late reply):

(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140438687430464) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140438687430464) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140438687430464, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007fba70025476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007fba7000b7f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x0000563945aed511 in __addvsi3 ()
#6  0x0000563945a6c106 in init_htab (hashp=0x563947700980, nelem=1000000000) at dynahash.c:720
#7  0x0000563945a6bd22 in hash_create (tabname=0x563945c591d9 "Local Buffer Lookup Table", nelem=1000000000, 
info=0x7ffd4d394620, flags=40) at dynahash.c:567
#8  0x00005639457f2760 in el () at localbuf.c:635
#9  0x00005639457f19e3 in ExtendBufferedRelLocal (bmr=..., fork=MAIN_FORKNUM, flags=8, extend_by=1, 
extend_upto=4294967295, buffers=0x7ffd4d3948e0, extended_by=0x7ffd4d3947ac) at localbuf.c:326
#10 0x00005639457e8851 in ExtendBufferedRelCommon (bmr=..., fork=MAIN_FORKNUM, strategy=0x0, flags=8, extend_by=1, 
extend_upto=4294967295, buffers=0x7ffd4d3948e0, extended_by=0x7ffd4d39488c) at bufmgr.c:2175
#11 0x00005639457e6850 in ExtendBufferedRelBy (bmr=..., fork=MAIN_FORKNUM, strategy=0x0, flags=8, extend_by=1, 
buffers=0x7ffd4d3948e0, extended_by=0x7ffd4d39488c) at bufmgr.c:923
#12 0x00005639452d8ae6 in RelationAddBlocks (relation=0x7fba650abd78, bistate=0x0, num_pages=1, use_fsm=true, 
did_unlock=0x7ffd4d394a3d) at hio.c:341
#13 0x00005639452d944a in RelationGetBufferForTuple (relation=0x7fba650abd78, len=32, otherBuffer=0, options=0, 
bistate=0x0, vmbuffer=0x7ffd4d394ac4, vmbuffer_other=0x0, num_pages=1) at hio.c:767
#14 0x00005639452be996 in heap_insert (relation=0x7fba650abd78, tup=0x5639476ecfc0, cid=0, options=0, bistate=0x0) at 
heapam.c:2019
#15 0x00005639452cee84 in heapam_tuple_insert (relation=0x7fba650abd78, slot=0x5639476ecf30, cid=0, options=0, 
bistate=0x0) at heapam_handler.c:251
#16 0x00005639455b3b07 in table_tuple_insert (rel=0x7fba650abd78, slot=0x5639476ecf30, cid=0, options=0, bistate=0x0) at 
../../../src/include/access/tableam.h:1405
#17 0x00005639455b5c60 in ExecInsert (context=0x7ffd4d394d20, resultRelInfo=0x5639476ec390, slot=0x5639476ecf30, 
canSetTag=true, inserted_tuple=0x0, insert_destrel=0x0) at nodeModifyTable.c:1139
#18 0x00005639455ba942 in ExecModifyTable (pstate=0x5639476ec180) at nodeModifyTable.c:4077
#19 0x0000563945575425 in ExecProcNodeFirst (node=0x5639476ec180) at execProcnode.c:469
#20 0x0000563945568095 in ExecProcNode (node=0x5639476ec180) at ../../../src/include/executor/executor.h:274
#21 0x000056394556af65 in ExecutePlan (estate=0x5639476ebf00, planstate=0x5639476ec180, use_parallel_mode=false, 
operation=CMD_INSERT, sendTuples=false, numberTuples=0, direction=ForwardScanDirection, dest=0x5639476f5470,
     execute_once=true) at execMain.c:1646
#22 0x00005639455687e3 in standard_ExecutorRun (queryDesc=0x5639476f3e70, direction=ForwardScanDirection, count=0, 
execute_once=true) at execMain.c:363
#23 0x00005639455685b9 in ExecutorRun (queryDesc=0x5639476f3e70, direction=ForwardScanDirection, count=0, 
execute_once=true) at execMain.c:304
#24 0x000056394584986e in ProcessQuery (plan=0x5639476f5310, sourceText=0x56394760d610 "INSERT INTO t VALUES(1);", 
params=0x0, queryEnv=0x0, dest=0x5639476f5470, qc=0x7ffd4d395180) at pquery.c:160
#25 0x000056394584b445 in PortalRunMulti (portal=0x56394768ab20, isTopLevel=true, setHoldSnapshot=false, 
dest=0x5639476f5470, altdest=0x5639476f5470, qc=0x7ffd4d395180) at pquery.c:1278
#26 0x000056394584a93c in PortalRun (portal=0x56394768ab20, count=9223372036854775807, isTopLevel=true, run_once=true, 
dest=0x5639476f5470, altdest=0x5639476f5470, qc=0x7ffd4d395180) at pquery.c:791
#27 0x0000563945842fd9 in exec_simple_query (query_string=0x56394760d610 "INSERT INTO t VALUES(1);") at postgres.c:1284
#28 0x0000563945848536 in PostgresMain (dbname=0x563947644900 "regression", username=0x5639476448e8 "law") at 
postgres.c:4766
#29 0x000056394583eb67 in BackendMain (startup_data=0x7ffd4d395404 "", startup_data_len=4) at backend_startup.c:107
#30 0x000056394574e00e in postmaster_child_launch (child_type=B_BACKEND, startup_data=0x7ffd4d395404 "", 
startup_data_len=4, client_sock=0x7ffd4d395450) at launch_backend.c:274
#31 0x0000563945753f74 in BackendStartup (client_sock=0x7ffd4d395450) at postmaster.c:3414
#32 0x00005639457515eb in ServerLoop () at postmaster.c:1648
#33 0x0000563945750eaa in PostmasterMain (argc=3, argv=0x5639476087b0) at postmaster.c:1346
#34 0x00005639455f738a in main (argc=3, argv=0x5639476087b0) at main.c:197

>
> Here's what it looks like is happening:
> ...
>
> The max value allowed for `temp_buffers` is `INT_MAX / 2` (1073741823),
> So any value of `temp_buffers` in the range (536870912, 1073741823]
> would cause this overflow. Without `-ftrapv`, `nbuckets` would wrap
> around to -2147483648, which is likely to cause all sorts of havoc, I'm
> just not sure what exactly.

Yeah, the minimum value that triggers the trap is 536870913 and the maximum
accepted is 1073741823.

Without -ftrapv, hctl->high_mask is set to 2147483647 on my machine,
when nbuckets is 1073741824, and the INSERT apparently succeeds.

>
> Also, `nbuckets = next_pow2_int(nelem);`, by itself is a bit sketchy
> considering that `nelem` is a `long` and `nbuckets` is an `int`.
> Potentially, the fix here is to just convert `nbuckets` to a `long`. I
> plan on checking if that's feasible.

Yes, it works for me; with s/int         nbuckets;/long nbuckets;/
I see no issue on 64-bit Linux.

Best regards,
Alexander