Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n8Enk-0007Q2-AO for pgsql-hackers@arkaria.postgresql.org; Fri, 14 Jan 2022 05:07:00 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1n8Eni-0007Qk-5D for pgsql-hackers@arkaria.postgresql.org; Fri, 14 Jan 2022 05:06:58 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1n8Enh-0007Qa-QN for pgsql-hackers@lists.postgresql.org; Fri, 14 Jan 2022 05:06:57 +0000 Received: from mail-lf1-x12c.google.com ([2a00:1450:4864:20::12c]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1n8Enf-0002wI-8F for pgsql-hackers@postgresql.org; Fri, 14 Jan 2022 05:06:57 +0000 Received: by mail-lf1-x12c.google.com with SMTP id d3so26546499lfv.13 for ; Thu, 13 Jan 2022 21:06:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yugabyte.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=QZdUp5novh/PhAhR9Zv3plTKAeDk+rF7YazY/CEyzro=; b=RWSavTR15F4/MojBqZIUZ5VopHAS+mHjU+gvsmKMQEvR5pgHH2I5geSEIuuyYsCtar XMK8FlqBduId/7vL2tfxpzVzyuTeyB+WnKzyR3U76PmTLuSDd2e9FhVgL1HA2LnA85K3 9aUrQntfF0T2OP83If34+TXvK48IbZ43RMOME= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QZdUp5novh/PhAhR9Zv3plTKAeDk+rF7YazY/CEyzro=; b=f7PJy/tgQ1t6EgmOpGxnlNopStgfTFprMXMURu8Vrcv8lyUQmrlL3Lrv44HaKwcrYJ 4ahR6GaU9kh2j5/A383lQk0ibjjVQD2HCH7qmJ+jetT4+V21qpGb0y3lfFDsqOOeXi6N 2vX/shAIyAdRBORG5wuIFh53XwFvHSA5pa95sRgQyEwE9rqw2NbMIt2hFkr1NKsx37io heLWo3UzSvQw7rS+JMFtDOMUmhAWdPz8dJ8XHajZ3iz47/F2n154ngWu1pSvccFyKSR4 TpnDlwXPSDvrIFYUlWfOLw8tQzo9hrkRNKnSv6a48K9UfIjVSFVlCUDMfeXnuWs5YKUn rt9Q== X-Gm-Message-State: AOAM532E4ytbPlH+ZiI9BbgOGFda6Y9Kv67JjjVYOdzcRveUk0QRs/Un J8iayS5pXJgWT3NKp4b1rWcMGYu8GW0EQAVeZF2HZA== X-Google-Smtp-Source: ABdhPJxBYSTGAURgpdGFj29EmONvByWG7xgBUay+tZMXYW5ET14gGI2sO132bVoFYfS62LPhqyMn9cv+ooKWjzG0oPU= X-Received: by 2002:a05:6512:118e:: with SMTP id g14mr5424525lfr.487.1642136814363; Thu, 13 Jan 2022 21:06:54 -0800 (PST) MIME-Version: 1.0 References: <20220109015202.GB283924@rfd.leadboat.com> <20220109031100.GC283924@rfd.leadboat.com> <1643134.1641713553@sss.pgh.pa.us> <20220109164854.GA309234@rfd.leadboat.com> <20220114040940.GA447185@rfd.leadboat.com> In-Reply-To: <20220114040940.GA447185@rfd.leadboat.com> From: Zhihong Yu Date: Thu, 13 Jan 2022 21:09:30 -0800 Message-ID: Subject: Re: null iv parameter passed to combo_init() To: Noah Misch Cc: Tom Lane , PostgreSQL-development Content-Type: multipart/alternative; boundary="0000000000007e406705d583c5d2" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000007e406705d583c5d2 Content-Type: text/plain; charset="UTF-8" On Thu, Jan 13, 2022 at 8:09 PM Noah Misch wrote: > On Sun, Jan 09, 2022 at 06:45:09PM -0800, Zhihong Yu wrote: > > On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu wrote: > > > After installing gcc-11, ./configure passed (with > 0003-memcpy-null.patch). > > > In the output of `make check-world`, I don't see `runtime error`. > > That's expected. With -fsanitize-undefined-trap-on-error, the program will > generate SIGILL when UBSan detects undefined behavior. To get "runtime > error" > messages in the postmaster log, drop -fsanitize-undefined-trap-on-error. > Both > ways of running the tests have uses. -fsanitize-undefined-trap-on-error is > better when you think the code is clean, because a zero "make check-world" > exit status confirms the code is clean. Once you know the code is unclean > in > some way, -fsanitize-undefined-trap-on-error is better for getting details. > > > > Though there was a crash (maybe specific to my machine): > > > > > > Core was generated by > > > > `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/postgres > > > --singl'. > > > Program terminated with signal SIGILL, Illegal instruction. > > > #0 0x000000000050642d in write_item.cold () > > > Missing separate debuginfos, use: debuginfo-install > > > glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64 > > > sssd-client-1.16.5-10.el7_9.10.x86_64 > > > (gdb) bt > > > #0 0x000000000050642d in write_item.cold () > > > #1 0x0000000000ba9d1b in write_relcache_init_file () > > > #2 0x0000000000bb58f7 in RelationCacheInitializePhase3 () > > > #3 0x0000000000bd5cb5 in InitPostgres () > > > #4 0x0000000000a0a9ea in PostgresMain () > > That is UBSan detecting undefined behavior. A successful patch version > will > fix write_item(), among many other places that are currently making > check-world fail. I get the same when testing your v5 under "gcc (Debian > 11.2.0-13) 11.2.0". I used the same host as buildfarm member thorntail, > and I > configured like this: > > ./configure -C --with-lz4 --prefix=$HOME/sw/nopath/pghead > --enable-tap-tests --enable-debug --enable-depend --enable-cassert > CC='ccache gcc-11 -fsanitize=undefined -fsanitize-undefined-trap-on-error' > CFLAGS='-O2 -funwind-tables' > > > Earlier I was using devtoolset-11 which had an `Illegal instruction` > error. > > > > I compiled / installed gcc-11 from source (which took whole afternoon). > > `make check-world` passed with patch v3. > > In tmp_install/log/install.log, I saw: > > > > gcc -Wall -Wmissing-prototypes -Wpointer-arith > > -Wdeclaration-after-statement -Werror=vla -Wendif-labels > > -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type > > -Wformat-security -fno-strict-aliasing -fwrapv > -fexcess-precision=standard > > -Wno-format-truncation -Wno-stringop-truncation -fsanitize=undefined > > -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND > > -I../../src/include -D_GNU_SOURCE -c -o path.o path.c > > rm -f libpgport.a > > Perhaps this self-compiled gcc-11 is defective, being unable to detect the > instances of undefined behavior that other builds detect. If so, use the > "devtoolset-11" gcc instead. You're also building without optimization; > that > might be the problem. > I tried both locally built gcc-11 and devtoolset-11 with configure command copied from above. `make world` failed in both cases with: performing post-bootstrap initialization ... sh: line 1: 24714 Illegal instruction (core dumped) ".../postgres/tmp_install/.../postgres/bin/postgres" --single -F -O -j -c search_path=pg_catalog -c exit_on_error=true -c log_checkpoints=false template1 > /dev/null child process exited with exit code 132 #0 0x000000000050a8d6 in write_item (data=, len=, fp=) at relcache.c:6471 #1 0x0000000000c33273 in write_relcache_init_file (shared=true) at relcache.c:6368 #2 0x0000000000c33c50 in RelationCacheInitializePhase3 () at relcache.c:4220 #3 0x0000000000c55825 in InitPostgres (in_dbname=, dboid=3105442800, username=, useroid=, out_dbname=0x0, override_allow_connections=) at postinit.c:1014 FYI --0000000000007e406705d583c5d2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Jan 13, 2022 at 8:09 PM Noah = Misch <noah@leadboat.com> wr= ote:
On Sun, Jan= 09, 2022 at 06:45:09PM -0800, Zhihong Yu wrote:
> On Sun, Jan 9, 2022 at 1:27 PM Zhihong Yu <zyu@yugabyte.com> wrote:
> > After installing gcc-11, ./configure passed (with 0003-memcpy-nul= l.patch).
> > In the output of `make check-world`, I don't see `runtime err= or`.

That's expected.=C2=A0 With -fsanitize-undefined-trap-on-error, the pro= gram will
generate SIGILL when UBSan detects undefined behavior.=C2=A0 To get "r= untime error"
messages in the postmaster log, drop -fsanitize-undefined-trap-on-error.=C2= =A0 Both
ways of running the tests have uses.=C2=A0 -fsanitize-undefined-trap-on-err= or is
better when you think the code is clean, because a zero "make check-wo= rld"
exit status confirms the code is clean.=C2=A0 Once you know the code is unc= lean in
some way, -fsanitize-undefined-trap-on-error is better for getting details.=

> > Though there was a crash (maybe specific to my machine):
> >
> > Core was generated by
> > `/nfusr/dev-server/zyu/postgres/tmp_install/usr/local/pgsql/bin/p= ostgres
> > --singl'.
> > Program terminated with signal SIGILL, Illegal instruction.
> > #0=C2=A0 0x000000000050642d in write_item.cold ()
> > Missing separate debuginfos, use: debuginfo-install
> > glibc-2.17-325.el7_9.x86_64 nss-pam-ldapd-0.8.13-25.el7.x86_64 > > sssd-client-1.16.5-10.el7_9.10.x86_64
> > (gdb) bt
> > #0=C2=A0 0x000000000050642d in write_item.cold ()
> > #1=C2=A0 0x0000000000ba9d1b in write_relcache_init_file ()
> > #2=C2=A0 0x0000000000bb58f7 in RelationCacheInitializePhase3 () > > #3=C2=A0 0x0000000000bd5cb5 in InitPostgres ()
> > #4=C2=A0 0x0000000000a0a9ea in PostgresMain ()

That is UBSan detecting undefined behavior.=C2=A0 A successful patch versio= n will
fix write_item(), among many other places that are currently making
check-world fail.=C2=A0 I get the same when testing your v5 under "gcc= (Debian
11.2.0-13) 11.2.0".=C2=A0 I used the same host as buildfarm member tho= rntail, and I
configured like this:

=C2=A0 ./configure -C --with-lz4 --prefix=3D$HOME/sw/nopath/pghead --enable= -tap-tests --enable-debug --enable-depend --enable-cassert CC=3D'ccache= gcc-11 -fsanitize=3Dundefined -fsanitize-undefined-trap-on-error' CFLA= GS=3D'-O2 -funwind-tables'

> Earlier I was using devtoolset-11 which had an `Illegal instruction` e= rror.
>
> I compiled / installed gcc-11 from source (which took whole afternoon)= .
> `make check-world` passed with patch v3.
> In tmp_install/log/install.log, I saw:
>
> gcc -Wall -Wmissing-prototypes -Wpointer-arith
> -Wdeclaration-after-statement -Werror=3Dvla -Wendif-labels
> -Wmissing-format-attribute -Wimplicit-fallthrough=3D3 -Wcast-function-= type
> -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=3Dst= andard
> -Wno-format-truncation -Wno-stringop-truncation -fsanitize=3Dundefined=
> -fsanitize-undefined-trap-on-error -I../../src/port -DFRONTEND
> -I../../src/include=C2=A0 -D_GNU_SOURCE=C2=A0 =C2=A0-c -o path.o path.= c
> rm -f libpgport.a

Perhaps this self-compiled gcc-11 is defective, being unable to detect the<= br> instances of undefined behavior that other builds detect.=C2=A0 If so, use = the
"devtoolset-11" gcc instead.=C2=A0 You're also building witho= ut optimization; that
might be the problem.

I tried both loca= lly built gcc-11 and devtoolset-11 with configure command copied from above= .
`make world` failed in both cases with:

per= forming post-bootstrap initialization ... sh: line 1: 24714 Illegal instruc= tion =C2=A0 =C2=A0 (core dumped) ".../postgres/tmp_install/.../postgre= s/bin/postgres" --single -F -O -j -c search_path=3Dpg_catalog -c exit_= on_error=3Dtrue -c log_checkpoints=3Dfalse template1 > /dev/null
child process exited with exit code 132=C2=A0

#0 = =C2=A00x000000000050a8d6 in write_item (data=3D<optimized out>, len= =3D<optimized out>, fp=3D<optimized out>) at relcache.c:6471#1 =C2=A00x0000000000c33273 in write_relcache_init_file (shared=3Dtrue) at= relcache.c:6368
#2 =C2=A00x0000000000c33c50 in RelationCacheInitializeP= hase3 () at relcache.c:4220
#3 =C2=A00x0000000000c55825 in InitPostgres = (in_dbname=3D<optimized out>, dboid=3D3105442800, username=3D<opti= mized out>, useroid=3D<optimized out>, out_dbname=3D0x0, override_= allow_connections=3D<optimized out>) at postinit.c:1014

FYI
--0000000000007e406705d583c5d2--