Subject: Re: General purpose hashing func in pgbench
From: Ildar Musin <i.musin@postgrespro.ru>
To: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
References: <0e8bd39e-dfcd-2879-f88f-272799ad7ef2@postgrespro.ru>
 <alpine.DEB.2.20.1712200807550.5165@lancre>
 <699225ee-c30c-b22e-4028-415e9eab26b2@postgrespro.ru>
 <alpine.DEB.2.20.1712211620040.32625@lancre>
 <80e9c570-5c49-1a01-f309-3d8d0cbbae42@postgrespro.ru>
 <alpine.DEB.2.20.1712240846440.22976@lancre>
 <38f1503e-7b63-4612-24a9-041af25bb022@postgrespro.ru>
 <alpine.DEB.2.20.1712251506160.22976@lancre>
 <af1ccd41-dcd0-f56b-0a05-a558da54762b@postgrespro.ru>
 <alpine.DEB.2.20.1712251702170.22976@lancre>
 <9603bac9-ae5d-36c5-5978-a88bdb700e37@postgrespro.ru>
 <alpine.DEB.2.20.1801092000230.2326@hendaye>
 <63f5a6ac-0c5b-8d59-ca8b-fc8e11244a66@postgrespro.ru>
Message-ID: <43a8fbbb-32fa-6478-30a9-f64041adf019@postgrespro.ru>
Date: Wed, 10 Jan 2018 16:37:03 +0300
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0)
 Gecko/20100101 Thunderbird/52.5.2
MIME-Version: 1.0
In-Reply-To: <63f5a6ac-0c5b-8d59-ca8b-fc8e11244a66@postgrespro.ru>
Content-Type: multipart/mixed;
 boundary="------------AADC0154A340299199D6902A"
Content-Language: ru
Precedence: bulk

This is a multi-part message in MIME format.
--------------AADC0154A340299199D6902A
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit


10/01/2018 16:35, Ildar Musin пишет:
> 09/01/2018 23:11, Fabien COELHO пишет:
>> Hello Ildar,
>>
>>> Sorry for a long delay. I've added hash() function which is just an
>>> alias for murmur2. I've also utilized variable arguments feature from
>>> least()/greates() functions to make optional seed parameter, but I
>>> consider this as a hack.
>> Patch needs a rebase after Teodor push for a set of pgbench functions.
> Done. Congratulations on your patch finally being committed : )
>>> Should we probably add some infrastructure for optional arguments?
>> You can look at the handling of "CASE" which may or may not have an
>> "ELSE" clause.
>>
>> I'd suggest you use a new negative argument with the special meaning
>> for hash, and create the seed value when missing when building the
>> function, so as to simplify the executor code.
> Added a new nargs option -3 for hash functions and moved arguments check
> to parser. It's starting to look a bit odd and I'm thinking about
> replacing bare numbers (-1, -2, -3) with #defined macros. E.g.:
>
> #define PGBENCH_NARGS_VARIABLE (-1)
> #define PGBENCH_NARGS_CASE (-2)
> #define PGBENCH_NARGS_HASH (-3)
>> Instead of 0, I'd consider providing a random default so that the
>> hashing behavior is not the same from one run to the next. What do you
>> think?
>>
> Makes sence since each thread is also initializes its random_state with
> random numbers before start. So I added global variable 'hash_seed' and
> initialize it with random() before threads spawned.
>> Like the previous version, murmur2 with seed looks much more random
>> than fnv with seed...
>>
Sorry, here is the patch

-- 
Ildar Musin
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company 


--------------AADC0154A340299199D6902A
Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0";
 name="pgbench_hash_v4.patch"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="pgbench_hash_v4.patch"

ZGlmZiAtLWdpdCBhL3NyYy9iaW4vcGdiZW5jaC9leHBycGFyc2UueSBiL3NyYy9iaW4vcGdi
ZW5jaC9leHBycGFyc2UueQppbmRleCBlMjNjYTUxLi44NDdiMDExIDEwMDY0NAotLS0gYS9z
cmMvYmluL3BnYmVuY2gvZXhwcnBhcnNlLnkKKysrIGIvc3JjL2Jpbi9wZ2JlbmNoL2V4cHJw
YXJzZS55CkBAIC0zNDksNiArMzQ5LDE1IEBAIHN0YXRpYyBjb25zdCBzdHJ1Y3QKIAl7CiAJ
CSIhY2FzZV9lbmQiLCAtMiwgUEdCRU5DSF9DQVNFCiAJfSwKKwl7CisJCSJoYXNoIiwgLTMs
IFBHQkVOQ0hfSEFTSF9NVVJNVVIyCisJfSwKKwl7CisJCSJoYXNoX211cm11cjIiLCAtMywg
UEdCRU5DSF9IQVNIX01VUk1VUjIKKwl9LAorCXsKKwkJImhhc2hfZm52MWEiLCAtMywgUEdC
RU5DSF9IQVNIX0ZOVjFBCisJfSwKIAkvKiBrZWVwIGFzIGxhc3QgYXJyYXkgZWxlbWVudCAq
LwogCXsKIAkJTlVMTCwgMCwgMApAQCAtNDQ3LDYgKzQ1NiwxNSBAQCBtYWtlX2Z1bmMoeXlz
Y2FuX3QgeXlzY2FubmVyLCBpbnQgZm51bWJlciwgUGdCZW5jaEV4cHJMaXN0ICphcmdzKQog
CQkJZXhwcl95eWVycm9yX21vcmUoeXlzY2FubmVyLCAib2RkIGFuZCA+PSAzIG51bWJlciBv
ZiBhcmd1bWVudHMgZXhwZWN0ZWQiLAogCQkJCQkJCSAgImNhc2UgY29udHJvbCBzdHJ1Y3R1
cmUiKTsKIAl9CisJLyogc3BlY2lhbCBjYXNlOiBoYXNoIGZ1bmN0aW9ucyB3aXRoIG9wdGlv
bmFsIGFyZ3VtZW50cyAqLworCWlmIChQR0JFTkNIX0ZVTkNUSU9OU1tmbnVtYmVyXS5uYXJn
cyA9PSAtMykKKwl7CisJCWludCBsZW4gPSBlbGlzdF9sZW5ndGgoYXJncyk7CisKKwkJaWYg
KGxlbiA8IDEgfHwgbGVuID4gMikKKwkJCWV4cHJfeXllcnJvcl9tb3JlKHl5c2Nhbm5lciwg
InVuZXhwZWN0ZWQgbnVtYmVyIG9mIGFyZ3VtZW50cyIsCisJCQkJCQkJICBQR0JFTkNIX0ZV
TkNUSU9OU1tmbnVtYmVyXS5mbmFtZSk7CisJfQogCiAJZXhwci0+ZXR5cGUgPSBFTk9ERV9G
VU5DVElPTjsKIAlleHByLT51LmZ1bmN0aW9uLmZ1bmN0aW9uID0gUEdCRU5DSF9GVU5DVElP
TlNbZm51bWJlcl0udGFnOwpkaWZmIC0tZ2l0IGEvc3JjL2Jpbi9wZ2JlbmNoL3BnYmVuY2gu
YyBiL3NyYy9iaW4vcGdiZW5jaC9wZ2JlbmNoLmMKaW5kZXggMzFlYTZjYS4uNmJmOTRjYyAx
MDA2NDQKLS0tIGEvc3JjL2Jpbi9wZ2JlbmNoL3BnYmVuY2guYworKysgYi9zcmMvYmluL3Bn
YmVuY2gvcGdiZW5jaC5jCkBAIC02MSw2ICs2MSwxNCBAQAogI2RlZmluZSBFUlJDT0RFX1VO
REVGSU5FRF9UQUJMRSAgIjQyUDAxIgogCiAvKgorICogSGFzaGluZyBjb25zdGFudHMKKyAq
LworI2RlZmluZSBGTlZfUFJJTUUgMHgxMDAwMDAwMDFiMworI2RlZmluZSBGTlZfT0ZGU0VU
X0JBU0lTIDB4Y2JmMjljZTQ4NDIyMjMyNQorI2RlZmluZSBNTTJfTVVMIDB4YzZhNGE3OTM1
YmQxZTk5NQorI2RlZmluZSBNTTJfUk9UIDQ3CisKKy8qCiAgKiBNdWx0aS1wbGF0Zm9ybSBw
dGhyZWFkIGltcGxlbWVudGF0aW9ucwogICovCiAKQEAgLTQzOSw2ICs0NDcsOCBAQCBzdGF0
aWMgaW50CW51bV9zY3JpcHRzOwkJLyogbnVtYmVyIG9mIHNjcmlwdHMgaW4gc3FsX3Njcmlw
dFtdICovCiBzdGF0aWMgaW50CW51bV9jb21tYW5kcyA9IDA7CS8qIHRvdGFsIG51bWJlciBv
ZiBDb21tYW5kIHN0cnVjdHMgKi8KIHN0YXRpYyBpbnQ2NCB0b3RhbF93ZWlnaHQgPSAwOwog
CitzdGF0aWMgaW50IGhhc2hfc2VlZDsJCQkvKiBkZWZhdWx0IHNlZWQgdXNlZCBpbiBoYXNo
IGZ1bmN0aW9ucyAqLworCiBzdGF0aWMgaW50CWRlYnVnID0gMDsJCQkvKiBkZWJ1ZyBmbGFn
ICovCiAKIC8qIEJ1aWx0aW4gdGVzdCBzY3JpcHRzICovCkBAIC05MTUsNiArOTI1LDUxIEBA
IGdldFppcGZpYW5SYW5kKFRTdGF0ZSAqdGhyZWFkLCBpbnQ2NCBtaW4sIGludDY0IG1heCwg
ZG91YmxlIHMpCiB9CiAKIC8qCisgKiBGTlYtMWEgaGFzaCBmdW5jdGlvbgorICovCitzdGF0
aWMgaW50NjQKK2dldEhhc2hGbnYxYShpbnQ2NCB2YWwsIHVpbnQ2NCBzZWVkKQoreworCWlu
dDY0CXJlc3VsdDsKKwlpbnQJCWk7CisKKwlyZXN1bHQgPSBGTlZfT0ZGU0VUX0JBU0lTIF4g
c2VlZDsKKwlmb3IgKGkgPSAwOyBpIDwgODsgKytpKQorCXsKKwkJaW50MzIgb2N0ZXQgPSB2
YWwgJiAweGZmOworCisJCXZhbCA9IHZhbCA+PiA4OworCQlyZXN1bHQgPSByZXN1bHQgXiBv
Y3RldDsKKwkJcmVzdWx0ID0gcmVzdWx0ICogRk5WX1BSSU1FOworCX0KKworCXJldHVybiBy
ZXN1bHQ7Cit9CisKKy8qCisgKiBNdXJtdXIyIGhhc2ggZnVuY3Rpb24KKyAqLworc3RhdGlj
IGludDY0CitnZXRIYXNoTXVybXVyMihpbnQ2NCB2YWwsIHVpbnQ2NCBzZWVkKQoreworCXVp
bnQ2NAlyZXN1bHQgPSBzZWVkIF4gKHNpemVvZihpbnQ2NCkgKiBNTTJfTVVMKTsKKwl1aW50
NjQJayA9ICh1aW50NjQpIHZhbDsKKworCWsgKj0gTU0yX01VTDsKKwlrIF49IGsgPj4gTU0y
X1JPVDsKKwlrICo9IE1NMl9NVUw7CisKKwlyZXN1bHQgXj0gazsKKwlyZXN1bHQgKj0gTU0y
X01VTDsKKworCXJlc3VsdCBePSByZXN1bHQgPj4gTU0yX1JPVDsKKwlyZXN1bHQgKj0gTU0y
X01VTDsKKwlyZXN1bHQgXj0gcmVzdWx0ID4+IE1NMl9ST1Q7CisKKwlyZXR1cm4gKGludDY0
KSByZXN1bHQ7Cit9CisKKy8qCiAgKiBJbml0aWFsaXplIHRoZSBnaXZlbiBTaW1wbGVTdGF0
cyBzdHJ1Y3QgdG8gYWxsIHplcm9lcwogICovCiBzdGF0aWMgdm9pZApAQCAtMjIwOSw2ICsy
MjY0LDM0IEBAIGV2YWxTdGFuZGFyZEZ1bmMoCiAJCQkJcmV0dXJuIHRydWU7CiAJCQl9CiAK
KwkJCS8qIGhhc2hpbmcgKi8KKwkJY2FzZSBQR0JFTkNIX0hBU0hfRk5WMUE6CisJCWNhc2Ug
UEdCRU5DSF9IQVNIX01VUk1VUjI6CisJCQl7CisJCQkJaW50NjQJdmFsOworCQkJCWludDY0
CXNlZWQ7CisJCQkJaW50NjQJcmVzdWx0OworCisJCQkJQXNzZXJ0KG5hcmdzID49IDEpOwor
CisJCQkJaWYgKCFjb2VyY2VUb0ludCgmdmFyZ3NbMF0sICZ2YWwpKQorCQkJCQlyZXR1cm4g
ZmFsc2U7CisKKwkJCQkvKiByZWFkIG9wdGlvbmFsIHNlZWQgdmFsdWUgKi8KKwkJCQlpZiAo
bmFyZ3MgPiAxKQorCQkJCXsKKwkJCQkJaWYgKCFjb2VyY2VUb0ludCgmdmFyZ3NbMV0sICZz
ZWVkKSkKKwkJCQkJCXJldHVybiBmYWxzZTsKKwkJCQl9CisJCQkJZWxzZQorCQkJCQlzZWVk
ID0gaGFzaF9zZWVkOworCisJCQkJcmVzdWx0ID0gKGZ1bmMgPT0gUEdCRU5DSF9IQVNIX0ZO
VjFBKSA/CisJCQkJCWdldEhhc2hGbnYxYSh2YWwsIHNlZWQpIDogZ2V0SGFzaE11cm11cjIo
dmFsLCBzZWVkKTsKKwkJCQlzZXRJbnRWYWx1ZShyZXR2YWwsIHJlc3VsdCk7CisJCQkJcmV0
dXJuIHRydWU7CisJCQl9CisKIAkJZGVmYXVsdDoKIAkJCS8qIGNhbm5vdCBnZXQgaGVyZSAq
LwogCQkJQXNzZXJ0KDApOwpAQCAtNTA1NCw2ICs1MTM3LDkgQEAgbWFpbihpbnQgYXJnYywg
Y2hhciAqKmFyZ3YpCiAJSU5TVFJfVElNRV9TRVRfQ1VSUkVOVChzdGFydF90aW1lKTsKIAlz
cmFuZG9tKCh1bnNpZ25lZCBpbnQpIElOU1RSX1RJTUVfR0VUX01JQ1JPU0VDKHN0YXJ0X3Rp
bWUpKTsKIAorCS8qIHNldCBkZWZhdWx0IHNlZWQgZm9yIGhhc2ggZnVuY3Rpb25zICovCisJ
aGFzaF9zZWVkID0gcmFuZG9tKCk7CisKIAkvKiBzZXQgdXAgdGhyZWFkIGRhdGEgc3RydWN0
dXJlcyAqLwogCXRocmVhZHMgPSAoVFN0YXRlICopIHBnX21hbGxvYyhzaXplb2YoVFN0YXRl
KSAqIG50aHJlYWRzKTsKIAluY2xpZW50c19kZWFsdCA9IDA7CmRpZmYgLS1naXQgYS9zcmMv
YmluL3BnYmVuY2gvcGdiZW5jaC5oIGIvc3JjL2Jpbi9wZ2JlbmNoL3BnYmVuY2guaAppbmRl
eCAwNzA1Y2NkLi42OTgzODY1IDEwMDY0NAotLS0gYS9zcmMvYmluL3BnYmVuY2gvcGdiZW5j
aC5oCisrKyBiL3NyYy9iaW4vcGdiZW5jaC9wZ2JlbmNoLmgKQEAgLTk3LDcgKzk3LDkgQEAg
dHlwZWRlZiBlbnVtIFBnQmVuY2hGdW5jdGlvbgogCVBHQkVOQ0hfTEUsCiAJUEdCRU5DSF9M
VCwKIAlQR0JFTkNIX0lTLAotCVBHQkVOQ0hfQ0FTRQorCVBHQkVOQ0hfQ0FTRSwKKwlQR0JF
TkNIX0hBU0hfRk5WMUEsCisJUEdCRU5DSF9IQVNIX01VUk1VUjIKIH0gUGdCZW5jaEZ1bmN0
aW9uOwogCiB0eXBlZGVmIHN0cnVjdCBQZ0JlbmNoRXhwciBQZ0JlbmNoRXhwcjsK
--------------AADC0154A340299199D6902A--