From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
To: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: CPU costs of random_zipfian in pgbench
Date: Sun, 17 Feb 2019 23:08:31 +0100
Message-ID: <779980b4-a4a6-3bf9-7ecf-56cc9ce6f5be@2ndquadrant.com> (raw)
In-Reply-To: <6065.1550419767@sss.pgh.pa.us>
References: <b5e172e9-ad22-48a3-86a3-589afa20e8f7@2ndquadrant.com>
<alpine.DEB.2.21.1902170907110.8818@lancre>
<6065.1550419767@sss.pgh.pa.us>
On 2/17/19 5:09 PM, Tom Lane wrote:
> Fabien COELHO <coelho@cri.ensmp.fr> writes:
>>> I'm trying to use random_zipfian() for benchmarking of skewed data sets,
>>> and I ran head-first into an issue with rather excessive CPU costs.
>
>> If you want skewed but not especially zipfian, use exponential which is
>> quite cheap. Also zipfian with a > 1.0 parameter does not have to compute
>> the harmonic number, so it depends in the parameter.
>
> Maybe we should drop support for parameter values < 1.0, then. The idea
> that pgbench is doing something so expensive as to require caching seems
> flat-out insane from here.
Maybe.
It's not quite clear to me why we support the two modes at all? We use
one algorithm for values < 1.0 and another one for values > 1.0, what's
the difference there? Are those distributions materially different?
Also, I wonder if just dropping support for parameters < 1.0 would be
enough, because the docs say:
The function's performance is poor for parameter values close and
above 1.0 and on a small range.
which seems to suggest it might be slow even for values > 1.0 in some
cases. Not sure.
> That cannot be seen as anything but a foot-gun
> for unwary users. Under what circumstances would an informed user use
> that random distribution rather than another far-cheaper-to-compute one?
>
>> ... This is why I submitted a pseudo-random permutation
>> function, which alas does not get much momentum from committers.
>
> TBH, I think pgbench is now much too complex; it does not need more
> features, especially not ones that need large caveats in the docs.
> (What exactly is the point of having zipfian at all?)
>
I wonder about the growing complexity of pgbench too ...
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, tgl@sss.pgh.pa.us, coelho@cri.ensmp.fr
Subject: Re: CPU costs of random_zipfian in pgbench
In-Reply-To: <779980b4-a4a6-3bf9-7ecf-56cc9ce6f5be@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox