Date: Thu, 3 Sep 2015 00:38:12 +0200
From: Andres Freund <andres@anarazel.de>
To: Greg Stark <stark@mit.edu>
Cc: Bruce Momjian <bruce@momjian.us>, Jeff Janes <jeff.janes@gmail.com>,
 "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>,
 Merlin Moncure <mmoncure@gmail.com>,
 Julien Rouhaud <julien.rouhaud@dalibo.com>
Subject: Re: Allow a per-tablespace effective_io_concurrency setting
Message-ID: <20150902223812.GE8555@awork2.anarazel.de>
References: 
 <CAHyXU0yaUG9R_E5=1gdXhD-MpWR=Gr=4=EHFD_fRid2+SCQrqA@mail.gmail.com>
 <CAHyXU0yiVvfQAnR9cyH=HWh1WbLRsioe=mzRJTHwtr=2azsTdQ@mail.gmail.com>
 <20141206130828.GA1314@momjian.us>
 <CAHyXU0wgpE2E3B+rmZ959tJT_adPFfPvHNqeA9K9mkJRAT9HXw@mail.gmail.com>
 <20141209204337.GC24488@momjian.us>
 <CAMkU=1w0nXDcwhyQurh+XCdQ2AyeSLBcMGqrNAYsckLbTqq3mw@mail.gmail.com>
 <55AA2469.20306@dalibo.com> <55AA27C3.7060009@dalibo.com>
 <20150902135351.GA5286@alap3.anarazel.de>
 <CAM-w4HNFSi9HAvRXrvPhYf7awhzZq1ojOtcbK064VxC5U5t1jA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAM-w4HNFSi9HAvRXrvPhYf7awhzZq1ojOtcbK064VxC5U5t1jA@mail.gmail.com>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 2015-09-02 19:49:13 +0100, Greg Stark wrote:
> I can take the blame for this formula.
> 
> It's called the "Coupon Collector Problem". If you hit get a random
> coupon from a set of n possible coupons, how many random coupons would
> you have to collect before you expect to have at least one of each.

My point is that that's just the entirely wrong way to model
prefetching. Prefetching can be massively beneficial even if you only
have a single platter! Even if there were no queues on the hardware or
OS level! Concurrency isn't the right way to look at prefetching.

You need to prefetch so far ahead that you'll never block on reading
heap pages - and that's only the case if processing the next N heap
blocks takes longer than the prefetch of the N+1 th page. That doesn't
mean there continously have to be N+1 prefetches in progress - in fact
that actually often will only be the case for the first few, after that
you hopefully are bottlnecked on CPU.

If you additionally take into account hardware realities where you have
multiple platters, multiple spindles, command queueing etc, that's even
more true. A single rotation of a single platter with command queuing
can often read several non consecutive blocks if they're on a similar

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers