From: Justin Pryzby <pryzby@telsasoft.com>
To: Krzysztof Olszewski <kolszew73@gmail.com>
Cc: Pavel Stehule <pavel.stehule@gmail.com>
Cc: Pgsql Performance <pgsql-performance@lists.postgresql.org>
Subject: Re: Postgresql server gets stuck at low load
Date: Tue, 9 Jun 2020 07:09:59 -0500
Message-ID: <20200609120959.GC22473@telsasoft.com> (raw)
In-Reply-To: <CAHihO3yZ2698bBgt2EMocMhy=SRAUMjw7KZ2u_36EaKB4Mcp_Q@mail.gmail.com>
References: <CAHihO3wAmb_b=uMT86PhJYk3-C-4_pNBqJ-TUehZVk82ifFFDw@mail.gmail.com>
<CAFj8pRDERh_L7qHzRyG4QG4dRTfYZBcYOOTO0-0nTRBHTK+PVA@mail.gmail.com>
<CAHihO3yZ2698bBgt2EMocMhy=SRAUMjw7KZ2u_36EaKB4Mcp_Q@mail.gmail.com>
On Tue, Jun 09, 2020 at 01:54:21PM +0200, Krzysztof Olszewski wrote:
> I had hugepage's off and on, problems still occurs,
> thanx for "perf top" suggestion,
> pt., 5 cze 2020 o 13:38 Pavel Stehule <pavel.stehule@gmail.com> napisał(a):
> > pá 5. 6. 2020 v 12:07 odesílatel Krzysztof Olszewski <kolszew73@gmail.com> napsal:
> >
> >> I have problem with one of my Postgres production server. Server works
> >> fine almost always, but sometimes without any increase of transactions or
> >> statements amount, machine gets stuck. Cores goes up to 100%, load up to
> >> 160%. When it happens then there are problems with connect to database and
> >> even it will succeed, simple queries works several seconds instead of
> >> milliseconds.Problem sometimes stops after a period a time (e.g. 35 min),
> >> sometimes we must restart Postgres, Linux, or even KVM (which exists as
> >> virtualization host).
> >>
> >> My hardware
> >> 56 cores (Intel Core Processor (Skylake, IBRS))
> >> 400 GB RAM
> >> RAID10 with about 40k IOPS
> >>
> >> shared_buffers 64000 MB
> >>
> >> In correlation with stucks i see informations in kernel log about
> >> NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s! [postmaster:33935]
> >
> > https://www.postgresql.org/message-id/CAHyXU0yAsVxoab2PcyoCuPjqymtnaE93v7bN4ctv2aNi92fefA%40mail.gma...
> >
> > Can be interesting a reply on Merlin's question from mail/.
> >
> > cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
> > cat /sys/kernel/mm/redhat_transparent_hugepage/defrag
try this:
echo 2 |sudo /sys/kernel/mm/ksm/run
https://www.postgresql.org/message-id/20170718180152.GE17566%40telsasoft.com
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: pryzby@telsasoft.com, kolszew73@gmail.com, pavel.stehule@gmail.com, pgsql-performance@lists.postgresql.org
Subject: Re: Postgresql server gets stuck at low load
In-Reply-To: <20200609120959.GC22473@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox