From: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
To: Jeff Janes <jeff.janes@gmail.com>
Cc: Armand du Plessis <adp@bank.io>
Cc: pgsql-performance <pgsql-performance@postgresql.org>
Subject: Re: Problems with pg_locks explosion
Date: Tue, 02 Apr 2013 19:25:00 +1300
Message-ID: <515A79BC.9020006@catalyst.net.nz> (raw)
In-Reply-To: <CAMkU=1zFuG2PDzcZ4xwX20aNBF9NXzXNnz8c7Gsmrs3V1jXMWQ@mail.gmail.com>
References: <CANf99sX6dK9cG9PDhXARq70QybO3Tn7EkRwt_YPxp8Lo-AFhmQ@mail.gmail.com>
<515A2223.5080204@catalyst.net.nz>
<CAMkU=1zFuG2PDzcZ4xwX20aNBF9NXzXNnz8c7Gsmrs3V1jXMWQ@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
On 02/04/13 19:08, Jeff Janes wrote:
> On Monday, April 1, 2013, Mark Kirkwood wrote:
>
>
> Your provisioned volumes are much better than the default AWS ones,
> but are still not hugely fast (i.e 1000 IOPS is about 8 MB/s worth
> of Postgres 8k buffers). So you may need to look at adding more
> volumes into the array, or adding some separate ones and putting
> pg_xlog directory on 'em.
>
> However before making changes I would recommend using iostat or sar
> to monitor how volumes are handling the load (I usually choose a 1
> sec granularity and look for 100% util and high - server hundred ms
> - awaits). Also iotop could be enlightening.
>
>
> Hi Mark,
>
> Do you have experience using these tools with AWS? When using non-DAS
> in other contexts, I've noticed that these tools often give deranged
> results, because the kernel doesn't correctly know what time to
> attribute to "network" and what to attribute to "disk". But I haven't
> looked into it on AWS EBS, maybe they do a better job there.
> Thanks for any insight,
>
Hi Jeff,
That is a very good point. I did notice a reasonable amount of network
traffic on the graphs posted previously, along with a suspiciously low
amount of IO for md127 (which I assume is the raid0 array)...and
wondered if iostat was not seeing IO fully, however it slipped my mind
(I am on leave with kittens - so claim that for the purrrfect excuse)!
However I don't recall there being a problem with the io tools for
standard EBS volumes - but I haven't benchmarked AWS for a over a year,
so things could be different now - and I have no experience with these
new provisioned volumes.
Armand - it might be instructive to do some benchmarking (with another
host and volume set) where you do something like:
$ dd if=/dev/zero of=file bs=8k count=1000000
and see if iostat and friends actually show you doing IO as expected!
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mark.kirkwood@catalyst.net.nz, jeff.janes@gmail.com, adp@bank.io
Subject: Re: Problems with pg_locks explosion
In-Reply-To: <515A79BC.9020006@catalyst.net.nz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox