From: Kevin Grittner <kgrittn@ymail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Dave Owens <dave@teamunify.com>
Cc: Merlin Moncure <mmoncure@gmail.com>
Cc: Matheus de Oliveira <matioli.matheus@gmail.com>
Cc: postgres performance list <pgsql-performance@postgresql.org>
Subject: Re: query against pg_locks leads to large memory alloc
Date: Tue, 19 Aug 2014 04:38:52 -0700
Message-ID: <1408448332.31825.YahooMailNeo@web122302.mail.ne1.yahoo.com> (raw)
In-Reply-To: <12486.1408404273@sss.pgh.pa.us>
References: <CA+OQrzhdtpC-9enDpY_PTue3_eh9xxmP-z51fJHFqezvhJ-9Nw@mail.gmail.com>
<CAJghg4+y5NcvjfJO+Ynad-5NcuHHOUOgFpUEP=fLA3WTgGAoCw@mail.gmail.com>
<CAHyXU0w+haLQ2UdU_XjyiEPnC_Xp24QM4QjNYXFXQhtkvdSsgA@mail.gmail.com>
<CA+OQrzhBziFavzH==iqrJegR5Us6q7EvbopkQAWJV9f-UYVthQ@mail.gmail.com>
<1408399307.93048.YahooMailNeo@web122304.mail.ne1.yahoo.com>
<12486.1408404273@sss.pgh.pa.us>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Kevin Grittner <kgrittn@ymail.com> writes:
>> Dave Owens <dave@teamunify.com> wrote:
>>> max_connections = 450 ...we have found that we run out of shared
>>> memory when max_pred_locks_per_transaction is less than 30k.
>
>> It gathers the information in memory to return for all those locks
>> (I think both the normal heavyweight locks and the predicate locks
>> do that). 450 * 30000 is 13.5 million predicate locks you could
>> have, so they don't need a very big structure per lock to start
>> adding up. I guess we should refactor that to use a tuplestore, so
>> it can spill to disk when it gets to be more than work_mem.
>
> Seems to me the bigger issue is why does he need such a huge
> max_pred_locks_per_transaction setting? It's hard to believe that
> performance wouldn't tank with 10 million predicate locks active.
> Whether you can do "select * from pg_locks" seems pretty far down
> the list of concerns about this setting.
It would be interesting to know more about the workload which is
capable of that, but it would be a lot easier to analyze what's
going on if we could look at where those locks are being used (in
summary, of course -- nobody can make sense of 10 million detail
lines). About all I can think to ask at this point is: how many
total tables and indexes are there in all databases in this cluster
(counting each partition of a partitioned table as a separate
table)? With the promotion of finer-grained locks to courser ones
this should be pretty hard to hit without a very large number of
tables.
--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: kgrittn@ymail.com, tgl@sss.pgh.pa.us, dave@teamunify.com, mmoncure@gmail.com, matioli.matheus@gmail.com
Subject: Re: query against pg_locks leads to large memory alloc
In-Reply-To: <1408448332.31825.YahooMailNeo@web122302.mail.ne1.yahoo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox