From: Tom Lane <tgl@sss.pgh.pa.us>
To: Melanie Plageman <melanieplageman@gmail.com>
cc: Greg Stark <stark@mit.edu>, Andres Freund <andres@anarazel.de>,
        Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
        Justin Pryzby <pryzby@telsasoft.com>,
        Thomas Munro <thomas.munro@gmail.com>,
        "David G. Johnston" <david.g.johnston@gmail.com>,
        PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: shared-memory based stats collector - v70
In-reply-to: 
 <CAAKRu_Z18jYbarKA4GZMOO00KtUqKtBqgQ8SgB=wSvSqKq3iiA@mail.gmail.com>
References: <20220317073652.kz3uuh6le6vbfod4@alap3.anarazel.de>
 <CAAKRu_YsFzW3jsQUD7MVUY8c+87OiKW_Y93TsPAfLr3zi_jX9w@mail.gmail.com>
 <20220325.142256.1649716600998659084.horikyota.ntt@gmail.com>
 <20220325.172418.1055357550509939415.horikyota.ntt@gmail.com>
 <20220402082102.ojdd2e6cobsmljvh@alap3.anarazel.de>
 <20220404041516.cctrvpadhuriawlq@alap3.anarazel.de>
 <20220405030506.lfdhbu5zf4tzdpux@alap3.anarazel.de>
 <20220406030008.2qxipjxo776dwnqs@alap3.anarazel.de>
 <CAM-w4HN=4ss5dS2DH6Cof=zocJXeGOjRNKxEGgUZdJPMndyNzA@mail.gmail.com>
 <CAAKRu_Z18jYbarKA4GZMOO00KtUqKtBqgQ8SgB=wSvSqKq3iiA@mail.gmail.com>
Comments: In-reply-to Melanie Plageman <melanieplageman@gmail.com>
	message dated "Wed, 20 Jul 2022 11:55:51 -0400"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1736121.1658333315.1@sss.pgh.pa.us>
Date: Wed, 20 Jul 2022 12:08:35 -0400
Message-ID: <1736122.1658333315@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/1736122.1658333315%40sss.pgh.pa.us>
Precedence: bulk

Melanie Plageman <melanieplageman@gmail.com> writes:
> On Wed, Jul 20, 2022 at 11:35 AM Greg Stark <stark@mit.edu> wrote:
>> It seems like if we really think the total number of database objects
>> is reasonably limited to scales that fit in RAM there would be a much
>> simpler database design that would just store the catalog tables in
>> simple in-memory data structures and map them all on startup without
>> doing all the work Postgres does to make relational storage scale.

> I think efforts to do such a thing have gotten caught up in solving
> issues around visibility and managing the relationship between local and
> global caches [1]. It doesn't seem like the primary technical concern
> was memory usage.

AFAIR, the previous stats collector implementation had no such provision
either: it'd just keep adding hashtable entries as it received info about
new objects.  The only thing that's changed is that now those entries are
in shared memory instead of process-local memory.  We'd be well advised to
be sure that memory can be swapped out under pressure, but otherwise I'm
not seeing that things have gotten worse.

			regards, tom lane