MIME-Version: 1.0
References: <54899ec9-5043-6f2a-01a8-0773f549b0a5@portavita.eu>
In-Reply-To: <54899ec9-5043-6f2a-01a8-0773f549b0a5@portavita.eu>
From: Rick Otten <rottenwindfish@gmail.com>
Date: Mon, 22 Jul 2019 08:06:09 -0400
Message-ID: 
 <CAMAYy4JRGCY+ReT4KuLDn5O2ssxxJndJCH-e-Mw34-DOc+MAZw@mail.gmail.com>
Subject: Re: benchmarking effective_io_concurrency
To: Fabio Pardi <f.pardi@portavita.eu>
Cc: pgsql-performance@lists.postgresql.org
Content-Type: multipart/alternative; boundary="000000000000729525058e43e7ac"
Precedence: bulk

--000000000000729525058e43e7ac
Content-Type: text/plain; charset="UTF-8"

On Mon, Jul 22, 2019 at 2:42 AM Fabio Pardi <f.pardi@portavita.eu> wrote:

> Hello,
>
>
> I recently spent a bit of time benchmarking effective_io_concurrency on
> Postgres.
>
> I would like to share my findings with you:
>
>
> https://portavita.github.io/2019-07-19-PostgreSQL_effective_io_concurrency_benchmarked/
>
> Comments are welcome.
>
> regards,
>
> fabio pardi
>

You didn't mention what type of disk storage you are using, or if that
matters.  The number of cores in your database could also matter.

Does the max_parallel_workers setting have any influence on how
effective_io_concurrency works?

Based on your data, one should set effective_io_concurrency at the highest
possible setting with no ill effects with the possible exception that your
disk will get busier.  Somehow I suspect that as you scale the number of
concurrent disk i/o tasks, other things may start to suffer.  For example
does CPU wait time start to increase as more and more threads are consumed
waiting for i/o instead of doing other processing?  Do you run into lock
contention on the i/o subsystem?  (Back in the day, lock contention for
/dev/tcp was a major bottleneck for scaling busy webservers vertically.  I
have no idea if modern linux kernels could run into the same issue waiting
for locks for /dev/sd0.  Surely if anything was going to push that issue,
it would be setting effective_io_concurrency really high and then demanding
a lot of concurrent disk accesses.)

--000000000000729525058e43e7ac
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Mon, Jul 22, 2019 at 2:42 AM Fabio=
 Pardi &lt;<a href=3D"mailto:f.pardi@portavita.eu">f.pardi@portavita.eu</a>=
&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hell=
o,<br>
<br>
<br>
I recently spent a bit of time benchmarking effective_io_concurrency on Pos=
tgres.<br>
<br>
I would like to share my findings with you:<br>
<br>
<a href=3D"https://portavita.github.io/2019-07-19-PostgreSQL_effective_io_c=
oncurrency_benchmarked/" rel=3D"noreferrer" target=3D"_blank">https://porta=
vita.github.io/2019-07-19-PostgreSQL_effective_io_concurrency_benchmarked/<=
/a><br>
<br>
Comments are welcome.<br>
<br>
regards,<br>
<br>
fabio pardi<br></blockquote><div><br></div><div>You didn&#39;t mention what=
 type of disk storage you are using, or if that matters.=C2=A0 The number o=
f cores in your database could also matter.</div><div><br></div><div>Does t=
he max_parallel_workers setting have any influence on how effective_io_conc=
urrency works?</div><div><br></div><div>Based on your data, one should set =
effective_io_concurrency at the highest possible setting with no ill effect=
s with the possible exception that your disk will get busier.=C2=A0 Somehow=
 I suspect that as you scale the number of concurrent disk i/o tasks, other=
 things may start to suffer.=C2=A0 For example does CPU wait time start to =
increase as more and more threads are consumed waiting for i/o instead of d=
oing other processing?=C2=A0 Do you run into lock contention on the i/o sub=
system?=C2=A0 (Back in the day, lock contention for /dev/tcp was a major bo=
ttleneck for scaling busy webservers vertically.=C2=A0 I have no idea if mo=
dern linux kernels could run into the same issue waiting for locks for /dev=
/sd0.=C2=A0 Surely if anything was going to push that issue, it would be se=
tting effective_io_concurrency really high and then demanding a lot of conc=
urrent disk accesses.)</div><div><br></div><div><br></div><div>=C2=A0</div>=
</div></div>

--000000000000729525058e43e7ac--