MIME-Version: 1.0
In-Reply-To: 
 <CADFyZw6McexkAV=p2POXrmHBDgNS7dw-izUvFDmTWyoXeTkp0g@mail.gmail.com>
References: 
 <CADFyZw7aGoD0AaStxdyHByR5Qta=M5wx0v=iptKLhPUp+EOKvA@mail.gmail.com>
 <CAMAYy4LiT925ZqouMs6Vs8fz=P53ev_SLH3qK7_7EtUWwUFMBg@mail.gmail.com>
 <CADFyZw6McexkAV=p2POXrmHBDgNS7dw-izUvFDmTWyoXeTkp0g@mail.gmail.com>
From: Rick Otten <rottenwindfish@gmail.com>
Date: Wed, 12 Jul 2017 10:10:27 -0400
Message-ID: 
 <CAMAYy4Lyjt3iSUi9wQtasX+q_SZoH8GCHvJeYmiSj1owWrzUSQ@mail.gmail.com>
Subject: Re: Very poor read performance, query independent
To: Charles Nadeau <charles.nadeau@gmail.com>
Cc: "pgsql-performa." <pgsql-performance@postgresql.org>
Content-Type: multipart/alternative; boundary="001a11441568ca46c505541f60d6"
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

--001a11441568ca46c505541f60d6
Content-Type: text/plain; charset="UTF-8"

On Wed, Jul 12, 2017 at 9:38 AM, Charles Nadeau <charles.nadeau@gmail.com>
wrote:

> Rick,
>
> Should the number of page should always be correlated to the VmPeak of the
> postmaster or could it be set to reflect shared_buffer or another setting?
> Thanks!
>
>
The documentation implies that you may need to adjust its size when you
change shared_buffer settings.

I usually check it every now and then (I haven't build a formal monitor
yet.) to see if all of the huge pages are free/used and if it looks like
they are all getting consumed - consider bumping it higher.  If there are
lots free, you are probably fine.

cat /proc/meminfo | grep -i "^huge"

--

Also regarding my note on effective_io_concurrency, which I'm not sure you
tried tweaking yet.

With file system and hardware caching between you and your spindles, your
best setting for effective_io_concurrency may be much higher than the
actual number of spindles.   It is worth experimenting with.   If you can,
try several values.  You can use pg_bench to put consistent workloads on
your database for measurement purposes.


Charles
>
> On Mon, Jul 10, 2017 at 5:25 PM, Rick Otten <rottenwindfish@gmail.com>
> wrote:
>
>> Although probably not the root cause, at the least I would set up
>> hugepages  ( https://www.postgresql.org/docs/9.6/static/kernel-resourc
>> es.html#LINUX-HUGE-PAGES ), and bump effective_io_concurrency up quite a
>> bit as well (256 ?).
>>
>>

--001a11441568ca46c505541f60d6
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><div class=3D"gmail_quo=
te">On Wed, Jul 12, 2017 at 9:38 AM, Charles Nadeau <span dir=3D"ltr">&lt;<=
a href=3D"mailto:charles.nadeau@gmail.com" target=3D"_blank">charles.nadeau=
@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding=
-left:1ex"><div dir=3D"ltr"><div>Rick,</div><div><br></div><div>Should the =
number of page should always be correlated to the VmPeak of the postmaster =
or could it be set to reflect shared_buffer or another setting?</div><div>T=
hanks!</div><div><br></div></div></blockquote><div><br></div><div>The docum=
entation implies that you may need to adjust its size when you change share=
d_buffer settings.=C2=A0</div><div><br></div><div>I usually check it every =
now and then (I haven&#39;t build a formal monitor yet.) to see if all of t=
he huge pages are free/used and if it looks like they are all getting consu=
med - consider bumping it higher.=C2=A0 If there are lots free, you are pro=
bably fine.</div><div><br></div><div>cat /proc/meminfo | grep -i &quot;^hug=
e&quot;<br></div><div><br></div><div>--</div><div><br></div><div>Also regar=
ding my note on effective_io_concurrency, which I&#39;m not sure you tried =
tweaking yet.</div><div><br></div><div>With file system and hardware cachin=
g between you and your spindles, your best setting for effective_io_concurr=
ency may be much higher than the actual number of spindles. =C2=A0 It is wo=
rth experimenting with. =C2=A0 If you can, try several values.=C2=A0 You ca=
n use pg_bench to put consistent workloads on your database for measurement=
 purposes.</div><div><br></div><div><br></div><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20=
4);padding-left:1ex"><div dir=3D"ltr"><div></div><div>Charles</div></div><d=
iv class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Mon, Jul 10, 201=
7 at 5:25 PM, Rick Otten <span dir=3D"ltr">&lt;<a href=3D"mailto:rottenwind=
fish@gmail.com" target=3D"_blank">rottenwindfish@gmail.com</a>&gt;</span> w=
rote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"=
>Although probably not the root cause, at the least I would set up hugepage=
s =C2=A0(=C2=A0<a href=3D"https://www.postgresql.org/docs/9.6/static/kernel=
-resources.html#LINUX-HUGE-PAGES" target=3D"_blank">https://www.postgresql.=
org/<wbr>docs/9.6/static/kernel-resourc<wbr>es.html#LINUX-HUGE-PAGES</a> ),=
 and bump effective_io_concurrency up quite a bit as well (256 ?).<div><br>=
</div></div></blockquote></div></div></blockquote></div></div></div>

--001a11441568ca46c505541f60d6--