From: Tom Lane <tgl@sss.pgh.pa.us>
To: Peter Geoghegan <pg@bowt.ie>
cc: baron@xaprb.com, PostgreSQL-documentation <pgsql-docs@postgresql.org>
Subject: Re: pg_stat_statements max size clarification
In-reply-to: 
 <CAH2-Wzkwz3sKJnuoii0bwYgxvnopFwybte1982Y8BAdwd0m8og@mail.gmail.com>
References: <20170716003147.9230.86974@wrigleys.postgresql.org>
 <CAH2-Wzkwz3sKJnuoii0bwYgxvnopFwybte1982Y8BAdwd0m8og@mail.gmail.com>
Comments: In-reply-to Peter Geoghegan <pg@bowt.ie>
	message dated "Sat, 15 Jul 2017 18:59:01 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7690.1500171444.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Sat, 15 Jul 2017 22:17:24 -0400
Message-ID: <7691.1500171444@sss.pgh.pa.us>
Precedence: bulk
Sender: pgsql-docs-owner@postgresql.org

Peter Geoghegan <pg@bowt.ie> writes:
> On Sat, Jul 15, 2017 at 5:31 PM,  <baron@xaprb.com> wrote:
>> I wonder if &quot;least-executed&quot; is correct. I&#39;m not an expert=
 and haven&#39;t
>> convinced myself of this by examining the code, but I think after N dist=
inct
>> queryid&#39;s have been seen, then any additional ones are ignored. But =
that may
>> not be &quot;least-executed&quot; at all. It&#39;s &quot;most-recent&quo=
t; instead. I think we need
>> a new phrase here.

> It's most executed since tracking for the entry began, with a special
> heuristic for queries that take a long time to execute, and might
> therefore consistently be evicted before execution finishes and costs
> are tallied (see "sticky entries" stuff for full details). Most
> executed means the total number of calls, which may not be the best
> thing to evict on the basis of, but certainly isn't too bad.

> The way it actually works is that either 5% of all entries or 10
> entries are evicted (whichever amount is greatest) once
> pg_stat_statements.max entries are reached. You're right that this
> means that the most marginal of entries cannot be usefully tracked,
> but I doubt that that's much of a problem in practice. It's the usual
> "recency versus frequency" cache eviction problem, but for query cost
> tracking purposes if 5,000 entries or 10,000 entries is truly
> insufficient, then pg_stat_statements probably isn't the right tool.

The short answer, really, is that the algorithm is too complicated to be
worth explaining in the documentation --- and it's subject to change,
anyway.  But "least-executed" is a reasonable short description, since
frequency of use is a major factor in the decisions.  Certainly
"most-recent" is flat out wrong.

I am not sure whether this complaint is actually meant as a bug report
that the algorithm didn't seem to work well on the OP's use case.  If so,
we'd need a lot more details to have any hope of improving it (and the
documentation comments aren't the right submission forum, either).

			regards, tom lane


--=20
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs