From: Ildar Musin <i.musin@postgrespro.ru>
To: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: General purpose hashing func in pgbench
Date: Tue, 9 Jan 2018 19:22:14 +0300
Message-ID: <9603bac9-ae5d-36c5-5978-a88bdb700e37@postgrespro.ru> (raw)
In-Reply-To: <alpine.DEB.2.20.1712251702170.22976@lancre>
References: <0e8bd39e-dfcd-2879-f88f-272799ad7ef2@postgrespro.ru>
<alpine.DEB.2.20.1712200807550.5165@lancre>
<699225ee-c30c-b22e-4028-415e9eab26b2@postgrespro.ru>
<alpine.DEB.2.20.1712211620040.32625@lancre>
<80e9c570-5c49-1a01-f309-3d8d0cbbae42@postgrespro.ru>
<alpine.DEB.2.20.1712240846440.22976@lancre>
<38f1503e-7b63-4612-24a9-041af25bb022@postgrespro.ru>
<alpine.DEB.2.20.1712251506160.22976@lancre>
<af1ccd41-dcd0-f56b-0a05-a558da54762b@postgrespro.ru>
<alpine.DEB.2.20.1712251702170.22976@lancre>
Hello Fabien,
25/12/2017 19:17, Fabien COELHO пишет:
>
>>> However, the key can be used if controlled so that different values do
>>> not have the same randomization in different part of the script, so as
>>> to avoid using the same patterns for different things if not desirable.
>>
>> Original murmur algorithm accepts seed as a parameter, which can be used
>> for this purpose. I used value itself as a seed in the patch, but I can
>> turn it into a function argument.
>
> Hmmm. Possibly. However it needs a default, something like
>
> x = hash(v[, key])
>
> with key some pseudo-random value?
>
Sorry for a long delay. I've added hash() function which is just an
alias for murmur2. I've also utilized variable arguments feature from
least()/greates() functions to make optional seed parameter, but I
consider this as a hack. Should we probably add some infrastructure for
optional arguments? Docs and tests are on their way.
Thanks!
--
Ildar Musin
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:pgbench_hash_v3.patch (text/plain, 3.3K) downloaddiff --git a/src/bin/pgbench/exprparse.y b/src/bin/pgbench/exprparse.yindex 26494fd..340c020 100644--- a/src/bin/pgbench/exprparse.y+++ b/src/bin/pgbench/exprparse.y@@ -131,7 +131,8 @@ make_op(yyscan_t yyscanner, const char *operator,
* List of available functions:
* - fname: function name
* - nargs: number of arguments
- * -1 is a special value for least & greatest meaning #args >= 1+ * -1 is a special value for least, greatest & hash functions+ * meaning variable arguments number
* - tag: function identifier from PgBenchFunction enum
*/
static const struct
@@ -200,6 +201,15 @@ static const struct
{
"power", 2, PGBENCH_POW
},
+ {+ "hash", -1, PGBENCH_HASH_MURMUR2+ },+ {+ "hash_murmur2", -1, PGBENCH_HASH_MURMUR2+ },+ {+ "hash_fnv1a", -1, PGBENCH_HASH_FNV1A+ },
/* keep as last array element */
{
NULL, 0, 0
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.cindex fc2c734..71e4814 100644--- a/src/bin/pgbench/pgbench.c+++ b/src/bin/pgbench/pgbench.c@@ -61,6 +61,14 @@
#define ERRCODE_UNDEFINED_TABLE "42P01"
/*
+ * Hashing constants+ */+#define FNV_PRIME 0x100000001b3+#define FNV_OFFSET_BASIS 0xcbf29ce484222325+#define MM2_MUL 0xc6a4a7935bd1e995+#define MM2_ROT 47++/*
* Multi-platform pthread implementations
*/
@@ -912,6 +920,51 @@ getZipfianRand(TState *thread, int64 min, int64 max, double s)
}
/*
+ * FNV-1a hash function+ */+static int64+getHashFnv1a(int64 val, uint64 seed)+{+ int64 result;+ int i;++ result = FNV_OFFSET_BASIS ^ seed;+ for (i = 0; i < 8; ++i)+ {+ int32 octet = val & 0xff;++ val = val >> 8;+ result = result ^ octet;+ result = result * FNV_PRIME;+ }++ return result;+}++/*+ * Murmur2 hash function+ */+static int64+getHashMurmur2(int64 val, uint64 seed)+{+ uint64 result = seed ^ (sizeof(int64) * MM2_MUL);+ uint64 k = (uint64) val;++ k *= MM2_MUL;+ k ^= k >> MM2_ROT;+ k *= MM2_MUL;++ result ^= k;+ result *= MM2_MUL;++ result ^= result >> MM2_ROT;+ result *= MM2_MUL;+ result ^= result >> MM2_ROT;++ return (int64) result;+}++/*
* Initialize the given SimpleStats struct to all zeroes
*/
static void
@@ -1868,6 +1921,37 @@ evalFunc(TState *thread, CState *st,
return true;
}
+ /* hashing */+ case PGBENCH_HASH_FNV1A:+ case PGBENCH_HASH_MURMUR2:+ {+ int64 val;+ int64 seed = 0;+ int64 result;++ Assert(nargs >= 1);++ if (!coerceToInt(&vargs[0], &val))+ return false;++ if (nargs > 2)+ {+ fprintf(stderr,+ "hash function accepts maximum of two arguments\n");+ return false;+ }++ /* read optional seed value */+ if (nargs > 1)+ if (!coerceToInt(&vargs[1], &seed))+ return false;++ result = (func == PGBENCH_HASH_FNV1A) ?+ getHashFnv1a(val, seed) : getHashMurmur2(val, seed);+ setIntValue(retval, result);+ return true;+ }+
default:
/* cannot get here */
Assert(0);
diff --git a/src/bin/pgbench/pgbench.h b/src/bin/pgbench/pgbench.hindex ce3c260..527e2a8 100644--- a/src/bin/pgbench/pgbench.h+++ b/src/bin/pgbench/pgbench.h@@ -77,7 +77,9 @@ typedef enum PgBenchFunction
PGBENCH_RANDOM_GAUSSIAN,
PGBENCH_RANDOM_EXPONENTIAL,
PGBENCH_RANDOM_ZIPFIAN,
- PGBENCH_POW+ PGBENCH_POW,+ PGBENCH_HASH_FNV1A,+ PGBENCH_HASH_MURMUR2
} PgBenchFunction;
typedef struct PgBenchExpr PgBenchExpr;
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: i.musin@postgrespro.ru, coelho@cri.ensmp.fr
Subject: Re: General purpose hashing func in pgbench
In-Reply-To: <9603bac9-ae5d-36c5-5978-a88bdb700e37@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox