pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Replace random(), pg_erand48(), etc with a better PRNG API and a
3+ messages / 1 participants
[nested] [flat]

Commits from this thread:
  2021-11-29 11b500072e  Portability hack for pg_global_prng_state.
* pgsql: Replace random(), pg_erand48(), etc with a better PRNG API and a
@ 2021-11-29 02:33  Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 3+ messages in thread

From: Tom Lane @ 2021-11-29 02:33 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Replace random(), pg_erand48(), etc with a better PRNG API and algorithm.

Standardize on xoroshiro128** as our basic PRNG algorithm, eliminating
a bunch of platform dependencies as well as fundamentally-obsolete PRNG
code.  In addition, this API replacement will ease replacing the
algorithm again in future, should that become necessary.

xoroshiro128** is a few percent slower than the drand48 family,
but it can produce full-width 64-bit random values not only 48-bit,
and it should be much more trustworthy.  It's likely to be noticeably
faster than the platform's random(), depending on which platform you
are thinking about; and we can have non-global state vectors easily,
unlike with random().  It is not cryptographically strong, but neither
are the functions it replaces.

Fabien Coelho, reviewed by Dean Rasheed, Aleksander Alekseev, and myself

Discussion: https://postgr.es/m/alpine.DEB.2.22.394.2105241211230.165418@pseudo

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/3804539e48e794781c6145c7f988f5d507418fa8

Modified Files
--------------
configure                                          |  26 ---
configure.ac                                       |   2 -
contrib/amcheck/verify_nbtree.c                    |   5 +-
contrib/auto_explain/auto_explain.c                |   4 +-
contrib/file_fdw/file_fdw.c                        |   2 +-
contrib/postgres_fdw/postgres_fdw.c                |   2 +-
contrib/tablefunc/tablefunc.c                      |   5 +-
contrib/tsm_system_rows/tsm_system_rows.c          |  12 +-
contrib/tsm_system_time/tsm_system_time.c          |  12 +-
src/backend/access/gin/ginget.c                    |   3 +-
src/backend/access/gist/gistutil.c                 |   5 +-
src/backend/access/nbtree/nbtinsert.c              |   3 +-
src/backend/access/spgist/spgdoinsert.c            |   5 +-
src/backend/access/transam/xact.c                  |   3 +-
src/backend/commands/analyze.c                     |   7 +-
src/backend/executor/nodeSamplescan.c              |   3 +-
src/backend/lib/bloomfilter.c                      |   3 +-
src/backend/optimizer/geqo/geqo_random.c           |  23 +-
src/backend/optimizer/geqo/geqo_selection.c        |  14 +-
src/backend/postmaster/postmaster.c                |  20 +-
src/backend/storage/file/fd.c                      |   4 +-
src/backend/storage/ipc/dsm.c                      |   9 +-
src/backend/storage/lmgr/Makefile                  |   5 +-
src/backend/storage/lmgr/s_lock.c                  |   5 +-
src/backend/tcop/postgres.c                        |   7 +-
src/backend/utils/adt/float.c                      |  20 +-
src/backend/utils/misc/sampling.c                  |  52 +++--
src/bin/initdb/initdb.c                            |   8 +-
src/bin/pg_test_fsync/pg_test_fsync.c              |   5 +-
src/bin/pgbench/pgbench.c                          | 122 ++++------
src/bin/pgbench/t/001_pgbench_with_server.pl       |  23 +-
src/common/Makefile                                |   1 +
src/common/pg_prng.c                               | 247 +++++++++++++++++++++
src/include/common/pg_prng.h                       |  60 +++++
src/include/optimizer/geqo.h                       |   3 +-
src/include/optimizer/geqo_random.h                |   5 +-
src/include/pg_config.h.in                         |   6 -
src/include/pg_config_manual.h                     |  11 -
src/include/port.h                                 |  13 --
src/include/utils/sampling.h                       |  15 +-
src/port/Makefile                                  |   1 -
src/port/erand48.c                                 | 136 ------------
src/port/random.c                                  |  25 ---
src/port/srandom.c                                 |  25 ---
.../modules/test_bloomfilter/test_bloomfilter.c    |   4 +-
src/test/modules/test_integerset/test_integerset.c |   6 +-
src/test/modules/test_rbtree/test_rbtree.c         |   5 +-
src/tools/msvc/Mkvcbuild.pm                        |  12 +-
src/tools/msvc/Solution.pm                         |   2 -
src/tools/testint128.c                             |  29 +--
50 files changed, 544 insertions(+), 481 deletions(-)



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: pgsql: Replace random(), pg_erand48(), etc with a better PRNG API and a
@ 2021-11-29 03:47  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 3+ messages in thread

From: Tom Lane @ 2021-11-29 03:47 UTC (permalink / raw)
  To: Andrew Dunstan <andrew@dunslane.net>; +Cc: pgsql-committers@lists.postgresql.org

Tom Lane <tgl@sss.pgh.pa.us> writes:
> Replace random(), pg_erand48(), etc with a better PRNG API and algorithm.

Hmm, fairywren doesn't like this [1]:

gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -O2 win32ver.o pg_test_fsync.o -L../../../src/port -L../../../src/common  -Wl,--allow-multiple-definition -Wl,--disable-auto-import  -L/c/prog/3p64/lib -L/c/prog/3p64/openssl/lib -Wl,--as-needed   -lpgcommon -lpgport -lssl -lcrypto -lz -lws2_32 -lm  -lws2_32 -o pg_test_fsync.exe
C:/tools/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: pg_test_fsync.o: in function `main':
C:/tools/msys64/home/pgrunner/bf/root/HEAD/pgsql.build/../pgsql/src/bin/pg_test_fsync/pg_test_fsync.c:121: undefined reference to `__imp_pg_global_prng_state'
collect2.exe: error: ld returned 1 exit status

I think what is going on here is that the compilation of pg_test_fsync.c
sees

extern PGDLLIMPORT pg_prng_state pg_global_prng_state;

and does something that's inappropriate when the variable is actually
coming from elsewhere in the same program.  If so, we could perhaps
fix it by doing

#ifdef FRONTEND
extern pg_prng_state pg_global_prng_state;
#else
extern PGDLLIMPORT pg_prng_state pg_global_prng_state;
#endif

Thoughts?

			regards, tom lane

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-11-29%2003%3A04%3A24





^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: pgsql: Replace random(), pg_erand48(), etc with a better PRNG API and a
@ 2021-11-29 04:57  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 3+ messages in thread

From: Tom Lane @ 2021-11-29 04:57 UTC (permalink / raw)
  To: Andrew Dunstan <andrew@dunslane.net>; +Cc: pgsql-committers@lists.postgresql.org

I wrote:
> ... If so, we could perhaps
> fix it by doing

> #ifdef FRONTEND
> extern pg_prng_state pg_global_prng_state;
> #else
> extern PGDLLIMPORT pg_prng_state pg_global_prng_state;
> #endif

Ah, I now see the same hack being used in keywords.h and pg_bitutils.h,
so I'll go do that.  (Maybe we should invent a variant of PGDLLIMPORT
that can make this happen without as much code duplication?)

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2021-11-29 04:57 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-11-29 02:33 pgsql: Replace random(), pg_erand48(), etc with a better PRNG API and a Tom Lane <tgl@sss.pgh.pa.us>
2021-11-29 03:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-11-29 04:57   ` Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox