Date: Sun, 17 Feb 2019 18:33:45 +0100
From: David Fetter <david@fetter.org>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Fabien COELHO <coelho@cri.ensmp.fr>,
	Tomas Vondra <tomas.vondra@2ndquadrant.com>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: CPU costs of random_zipfian in pgbench
Message-ID: <20190217173344.GY10435@fetter.org>
References: <b5e172e9-ad22-48a3-86a3-589afa20e8f7@2ndquadrant.com>
 <alpine.DEB.2.21.1902170907110.8818@lancre>
 <6065.1550419767@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <6065.1550419767@sss.pgh.pa.us>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

On Sun, Feb 17, 2019 at 11:09:27AM -0500, Tom Lane wrote:
> Fabien COELHO <coelho@cri.ensmp.fr> writes:
> >> I'm trying to use random_zipfian() for benchmarking of skewed data sets, 
> >> and I ran head-first into an issue with rather excessive CPU costs. 
> 
> > If you want skewed but not especially zipfian, use exponential which is 
> > quite cheap. Also zipfian with a > 1.0 parameter does not have to compute 
> > the harmonic number, so it depends in the parameter.
> 
> Maybe we should drop support for parameter values < 1.0, then.  The idea
> that pgbench is doing something so expensive as to require caching seems
> flat-out insane from here.  That cannot be seen as anything but a foot-gun
> for unwary users.  Under what circumstances would an informed user use
> that random distribution rather than another far-cheaper-to-compute one?
> 
> > ... This is why I submitted a pseudo-random permutation 
> > function, which alas does not get much momentum from committers.
> 
> TBH, I think pgbench is now much too complex; it does not need more
> features, especially not ones that need large caveats in the docs.
> (What exactly is the point of having zipfian at all?)

Taking a statistical perspective, Zipfian distributions violate some
assumptions we make by assuming uniform distributions. This matters
because Zipf-distributed data sets are quite common in real life.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate