From: David Fetter <david@fetter.org>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Fabien COELHO <coelho@cri.ensmp.fr>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: CPU costs of random_zipfian in pgbench
Date: Tue, 19 Feb 2019 19:03:03 +0100
Message-ID: <20190219180303.GZ10435@fetter.org> (raw)
In-Reply-To: <0666abc4-1a41-7703-8fd8-244ac10243b4@2ndquadrant.com>
References: <b5e172e9-ad22-48a3-86a3-589afa20e8f7@2ndquadrant.com>
<alpine.DEB.2.21.1902170907110.8818@lancre>
<6065.1550419767@sss.pgh.pa.us>
<20190217173344.GY10435@fetter.org>
<0666abc4-1a41-7703-8fd8-244ac10243b4@2ndquadrant.com>
On Sun, Feb 17, 2019 at 11:02:37PM +0100, Tomas Vondra wrote:
> On 2/17/19 6:33 PM, David Fetter wrote:
> > On Sun, Feb 17, 2019 at 11:09:27AM -0500, Tom Lane wrote:
> >> Fabien COELHO <coelho@cri.ensmp.fr> writes:
> >>>> I'm trying to use random_zipfian() for benchmarking of skewed data sets,
> >>>> and I ran head-first into an issue with rather excessive CPU costs.
> >>
> >>> If you want skewed but not especially zipfian, use exponential which is
> >>> quite cheap. Also zipfian with a > 1.0 parameter does not have to compute
> >>> the harmonic number, so it depends in the parameter.
> >>
> >> Maybe we should drop support for parameter values < 1.0, then. The idea
> >> that pgbench is doing something so expensive as to require caching seems
> >> flat-out insane from here. That cannot be seen as anything but a foot-gun
> >> for unwary users. Under what circumstances would an informed user use
> >> that random distribution rather than another far-cheaper-to-compute one?
> >>
> >>> ... This is why I submitted a pseudo-random permutation
> >>> function, which alas does not get much momentum from committers.
> >>
> >> TBH, I think pgbench is now much too complex; it does not need more
> >> features, especially not ones that need large caveats in the docs.
> >> (What exactly is the point of having zipfian at all?)
> >
> > Taking a statistical perspective, Zipfian distributions violate some
> > assumptions we make by assuming uniform distributions. This matters
> > because Zipf-distributed data sets are quite common in real life.
> >
>
> I don't think there's any disagreement about the value of non-uniform
> distributions. The question is whether it has to be a zipfian one, when
> the best algorithm we know about is this expensive in some cases? Or
> would an exponential distribution be enough?
I suppose to people who care about the difference between Zipf and
exponential would appreciate having the former around to test.
Whether pgbench should support this is a different question, and it's
sounding a little like the answer to that one is "no."
Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: david@fetter.org, tomas.vondra@2ndquadrant.com, tgl@sss.pgh.pa.us, coelho@cri.ensmp.fr
Subject: Re: CPU costs of random_zipfian in pgbench
In-Reply-To: <20190219180303.GZ10435@fetter.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox