Feedback-ID: id4a34324:Fastmail
Date: Wed, 20 Jul 2022 21:08:49 +0200
From: Andres Freund <andres@anarazel.de>
To: Greg Stark <stark@mit.edu>, Tom Lane <tgl@sss.pgh.pa.us>
CC: Melanie Plageman <melanieplageman@gmail.com>,
 Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
 Justin Pryzby <pryzby@telsasoft.com>, Thomas Munro <thomas.munro@gmail.com>,
 "David G. Johnston" <david.g.johnston@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: shared-memory based stats collector - v70
User-Agent: K-9 Mail for Android
In-Reply-To: 
 <CAM-w4HOxaO6iJ94bGt1A65PRgX_b4gGSmb4mdzakZ1QhvGW=jA@mail.gmail.com>
References: <20220317073652.kz3uuh6le6vbfod4@alap3.anarazel.de>
 <CAAKRu_YsFzW3jsQUD7MVUY8c+87OiKW_Y93TsPAfLr3zi_jX9w@mail.gmail.com>
 <20220325.142256.1649716600998659084.horikyota.ntt@gmail.com>
 <20220325.172418.1055357550509939415.horikyota.ntt@gmail.com>
 <20220402082102.ojdd2e6cobsmljvh@alap3.anarazel.de>
 <20220404041516.cctrvpadhuriawlq@alap3.anarazel.de>
 <20220405030506.lfdhbu5zf4tzdpux@alap3.anarazel.de>
 <20220406030008.2qxipjxo776dwnqs@alap3.anarazel.de>
 <CAM-w4HN=4ss5dS2DH6Cof=zocJXeGOjRNKxEGgUZdJPMndyNzA@mail.gmail.com>
 <CAAKRu_Z18jYbarKA4GZMOO00KtUqKtBqgQ8SgB=wSvSqKq3iiA@mail.gmail.com>
 <1736122.1658333315@sss.pgh.pa.us>
 <CAM-w4HOxaO6iJ94bGt1A65PRgX_b4gGSmb4mdzakZ1QhvGW=jA@mail.gmail.com>
Message-ID: <83D73EC0-E3ED-4C98-A528-5E122278D553@anarazel.de>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/83D73EC0-E3ED-4C98-A528-5E122278D553%40anarazel.de>
Precedence: bulk

Hi,=20

On July 20, 2022 8:41:53 PM GMT+02:00, Greg Stark <stark@mit=2Eedu> wrote:
>On Wed, 20 Jul 2022 at 12:08, Tom Lane <tgl@sss=2Epgh=2Epa=2Eus> wrote:
>>
>> AFAIR, the previous stats collector implementation had no such provisio=
n
>> either: it'd just keep adding hashtable entries as it received info abo=
ut
>> new objects=2E  The only thing that's changed is that now those entries=
 are
>> in shared memory instead of process-local memory=2E  We'd be well advis=
ed to
>> be sure that memory can be swapped out under pressure, but otherwise I'=
m
>> not seeing that things have gotten worse=2E
>
>Just to be clear I'm not looking for ways things have gotten worse=2E
>Just trying to understand what I'm reading and I guess I came in with
>assumptions that led me astray=2E
>
>But=2E=2E=2E adding entries as it received info about new objects isn't t=
he
>same as having info on everything=2E I didn't really understand how the
>old system worked but if you had a very large schema but each session
>only worked with a small subset did the local stats data ever absorb
>info on the objects it never touched?

Each backend only had stats for things it touched=2E But the stats collect=
or read all files at startup into hash tables and absorbed all generated st=
ats into those as well=2E


>All that said -- having all objects loaded in shared memory makes my
>work way easier=2E

What are your trying to do?=20

>It actually seems feasible to dump out all the
>objects from shared memory and including objects from other databases
>and if I don't need a consistent snapshot it even seems like it would
>be possible to do that without having a copy of more than one stats
>entry at a time in local memory=2E I hope that doesn't cause huge
>contention on the shared hash table to be doing that regularly=2E

The stats accessors now default to not creating a full snapshot of stats d=
ata at first access (but that's configurable)=2E So yes, that behavior is p=
ossible=2E E=2Eg=2E autovac now uses a single object access like you descri=
be=2E

Andres


--=20
Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E