Date: Tue, 31 Aug 2021 08:07:27 +0200
From: hubert depesz lubaczewski <depesz@depesz.com>
Sender: depesz@depesz.com
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Laurenz Albe <laurenz.albe@cybertec.at>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: Can we get rid of repeated queries from pg_dump?
Message-ID: <20210831060727.GA32253@depesz.com>
Reply-To: depesz@depesz.com
References: <20210826084430.GA26282@depesz.com>
 <648517.1629986527@sss.pgh.pa.us>
 <20210826140827.GA15868@depesz.com>
 <649446.1629987629@sss.pgh.pa.us>
 <20210826160644.GA3120@depesz.com>
 <e6a7503afd115d111fbb96c5787906d73cfe2605.camel@cybertec.at>
 <896732.1630099403@sss.pgh.pa.us>
 <1484548.1630368660@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <1484548.1630368660@sss.pgh.pa.us>
User-Agent: Mutt/1.10.1 (2018-07-13)
Archived-At: 
 <https://www.postgresql.org/message-id/20210831060727.GA32253%40depesz.com>
Precedence: bulk

On Mon, Aug 30, 2021 at 08:11:00PM -0400, Tom Lane wrote:
> [ redirecting to -hackers ]
> 
> I wrote:
> > I experimented with the attached, very quick-n-dirty patch to collect
> > format_type results during the initial scan of pg_type, instead.  On the
> > regression database in HEAD, it reduces the number of queries pg_dump
> > issues from 3260 to 2905; but I'm having a hard time detecting any net
> > performance change.
> 
> I decided that that patch wasn't too safe, because it applies
> format_type() to pg_type rows that we have no reason to trust the
> longevity of.  I think it could fall over if some concurrent process
> were busy dropping a temp table, for example.
> 
> So here's a version that just does plain caching of the results
> of retail getFormattedTypeName() calls.  This visibly adds no
> queries that were not done before, so it should be safe enough.
> And there can't be any cases that it makes slower, either.

Hi,
tested it in my case, and it reduced query count to 4261.

Which is great.

But, I also looked closer into the pg_proc queries and extensions.
And - most functions come from relatively standard extensions:
- postgis     1246 functions
- btree_gist  179 functions
- btree_gin   87 functions
- hstore      58 functions

My point in here is that potential optimizations regarding queries for
pg_proc might speed up dumps for more people - as they might use things
like postgis, but never realized that it can be much faster.

Best regards,

depesz