Date: Wed, 28 Oct 2020 19:55:38 +0100
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: "David G. Johnston" <david.g.johnston@gmail.com>
Cc: =?utf-8?B?RmFicsOtemlv?= Mello <fabriziomello@gmail.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Add important info about ANALYZE after create Functional Index
Message-ID: <20201028185538.fzux2lhim472h46o@development>
References: 
 <CAFcNs+qpFPmiHd1oTXvcPdvAHicJDA9qBUSujgAhUMJyUMb+SA@mail.gmail.com>
 <CAKFQuwb9GFe4oM3k9bsn7GOk5_jfow-9SpWocAG4xAYVp9nLjg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: 
 <CAKFQuwb9GFe4oM3k9bsn7GOk5_jfow-9SpWocAG4xAYVp9nLjg@mail.gmail.com>
Precedence: bulk

On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote:
>On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello <
>fabriziomello@gmail.com> wrote:
>
>> Hi all,
>>
>> As you all already know Postgres supports functions in index expressions
>> (marked as immutable ofc) and for this special index the ANALYZE command
>> creates some statistics (new pg_statistic entry) about it.
>>
>> The problem is just after creating a new index or rebuilding concurrently
>> (using the new REINDEX .. CONCURRENTLY or the old manner creating new one
>> and then swapping) we need to run ANALYZE to update statistics but we don't
>> mention it in any part of our documentation.
>>
>> Last weekend Gitlab went down because the lack of an ANALYZE after
>> rebuilding concurrently a functional index and they followed the
>> recommendation we have into our documentation [1] about how to rebuild it
>> concurrently, but we don't warn users about the ANALYZE after.
>>
>> Would be nice if add some information about it into our docs but not sure
>> where. I'm thinking about:
>> - doc/src/sgml/ref/create_index.sgml
>> - doc/src/sgml/maintenance.sgml (routine-reindex)
>>
>> Thoughts?
>>
>> [1]
>> https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2885#note_436310499
>>
>
>It would seem preferable to call the lack of auto-analyzing after these
>operations a bug and back-patch a fix that injects an analyze side-effect
>just before their completion.  It doesn't have to be smart either,
>analyzing things even if the created (or newly validated) index doesn't
>have statistics of its own isn't a problem in my book.
>

I agree the lack of stats may be quite annoying and cause issues, but my
guess is the chances of backpatching such change are about 0.000001%. We
have a usable 'workaround' for this - manual analyze.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services