From: Tom Lane <tgl@sss.pgh.pa.us>
To: Jeff Janes <jeff.janes@gmail.com>
Cc: Greg Spiegelberg <gspiegelberg@gmail.com>
Cc: Álvaro Hernández Tortosa <aht@8kdata.com>
Cc: pgsql-performa. <pgsql-performance@postgresql.org>
Subject: Re: Millions of tables
Date: Mon, 26 Sep 2016 12:52:39 -0400
Message-ID: <22569.1474908759@sss.pgh.pa.us> (raw)
In-Reply-To: <CAMkU=1y-YUNFpwmbW7jnfma7Rf6VtKTiuHv4gcYxyEF7NJ33Gw@mail.gmail.com>
References: <CAEtnbpVXi0ESh16KAeYkFOiFPhTrJ2JXps9PypuAGCYidWtcnA@mail.gmail.com>
<7413c596-f2e0-77eb-ad6c-0ce456fc2665@8kdata.com>
<CAEtnbpVddc_3o+vTrMhGEPzeoV4H1ngYEXXU9sUaFD0VgCM=Jg@mail.gmail.com>
<CAMkU=1y-YUNFpwmbW7jnfma7Rf6VtKTiuHv4gcYxyEF7NJ33Gw@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
Jeff Janes <jeff.janes@gmail.com> writes:
> A problem is that those statistics are stored in one file (per database; it
> used to be one file per cluster). With 8 million tables, that is going to
> be a pretty big file. But the code pretty much assumes the file is going
> to be pretty small, and so it has no compunction about commanding that it
> be read and written, in its entirety, quite often.
I don't know that anyone ever believed it would be small. But at the
time the pgstats code was written, there was no good alternative to
passing the data through files. (And I'm not sure we envisioned
applications that would be demanding fresh data constantly, anyway.)
Now that the DSM stuff exists and has been more or less shaken out,
I wonder how practical it'd be to use a DSM segment to make the stats
collector's data available to backends. You'd need a workaround for
the fact that not all the DSM implementations support resize (although
given the lack of callers of dsm_resize, one could be forgiven for
wondering whether any of that code has been tested at all). But you
could imagine abandoning one DSM segment and creating a new one of
double the size anytime the hash tables got too big.
regards, tom lane
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: tgl@sss.pgh.pa.us, jeff.janes@gmail.com, gspiegelberg@gmail.com, aht@8kdata.com
Subject: Re: Millions of tables
In-Reply-To: <22569.1474908759@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox