Feedback-ID: id4a34324:Fastmail
Date: Mon, 23 Jan 2023 16:46:12 -0800
From: Andres Freund <andres@anarazel.de>
To: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Thomas Munro <thomas.munro@gmail.com>,
	Alvaro Herrera <alvherre@2ndquadrant.com>,
	Ibrar Ahmed <ibrar.ahmad@gmail.com>,
	Amit Khandekar <amitdkhan.pg@gmail.com>, fabriziomello@gmail.com,
	tushar <tushar.ahuja@enterprisedb.com>,
	Rahila Syed <rahila.syed@2ndquadrant.com>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: Minimal logical decoding on standbys
Message-ID: <20230124004612.xmryok6tcyk6i4pc@awork3.anarazel.de>
References: <CA+TgmoaOsoQuXEo6gMq0GP8SyvCaS=ptYqdYc0uzDMoeShf_nw@mail.gmail.com>
 <CA+TgmoYTTsxP8y6uknZvCBNCRq+1FJ4zGbX8Px1TGW459fGsaQ@mail.gmail.com>
 <1f2ad9fd-53ff-74cc-a602-72d45423bcab@gmail.com>
 <a78de011-81cf-729c-e110-e6d636fdca4d@gmail.com>
 <05c0c4f3-82eb-bc6d-c876-5a265c05927b@gmail.com>
 <CA+TgmoY8LAaTwAhS+YNQNrM+3K8eqxGvPT3gbJaHER86XShvFw@mail.gmail.com>
 <20230106034036.2m4qnn7ep7b5ipet@awork3.anarazel.de>
 <490206c5-bcde-e558-628a-dc67dae4d439@gmail.com>
 <20230119024617.jzsgpcrgjdc6ysge@awork3.anarazel.de>
 <a343b646-c632-4b9a-0e01-b661075d1d8b@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <a343b646-c632-4b9a-0e01-b661075d1d8b@gmail.com>
Archived-At: <https://www.postgresql.org/message-id/20230124004612.xmryok6tcyk6i4pc%40awork3.anarazel.de>
Precedence: bulk

Hi,

On 2023-01-19 10:43:27 +0100, Drouvot, Bertrand wrote:
> > > With a reload in place in my testing, now I notice that the catalog_xmin
> > > is updated on the primary physical slot after logical slots invalidation
> > > when reloading hot_standby_feedback from "off" to "on".
> > > 
> > > This is not the case after a re-start (aka catalog_xmin is NULL).
> > > 
> > > I think a re-start and reload should produce identical behavior on
> > > the primary physical slot. If so, I'm tempted to think that the catalog_xmin
> > > should be updated in case of a re-start too (even if all the logical slots are invalidated)
> > > because the slots are not dropped yet. What do you think?
> > 
> > I can't quite follow the steps leading up to the difference. Could you list
> > them in a bit more detail?
> > 
> > 
> 
> Sure, so with:
> 
> 1) hot_standby_feedback set to off on the standby
> 2) create 2 logical replication slots on the standby and activate one
> 3) Invalidate the logical slots on the standby with VACUUM FULL on the primary
> 4) change hot_standby_feedback to on on the standby
> 
> If:
> 
> 5) pg_reload_conf() on the standby, then on the primary we get a catalog_xmin
> for the physical slot that the standby is attached to:
> 
> postgres=# select slot_type,xmin,catalog_xmin  from pg_replication_slots ;
>  slot_type | xmin | catalog_xmin
> -----------+------+--------------
>  physical  |  822 |          748
> (1 row)

How long did you wait for this to change? I don't think there's anything right
now that'd force a new hot-standby-feedback message to be sent to the primary,
after slots got invalidated.

I suspect that if you terminated the walsender connection on the primary,
you'd not see it anymore either?

If that isn't it, something is broken in InvalidateObsolete...


> No, but a question still remains to me:
> 
> Given the fact that the row removal case is already done
> in the next test (aka Scenario 2), If we want to replace the "vacuum full" test
> on the database (done in Scenario 1) with a cheaper one at the table level,
> what could it be to guarantee an invalidation?
> 
> Same as scenario 2 but with "vacuum full pg_class" would not really add value
> to the tests, right?

A database wide VACUUM FULL is also just a row removal test, no? I think it
makes sense to test that both VACUUM and VACUUM FULL both trigger conflicts,
because they internally use *very* different mechanisms.  It'd probably be
good to test at least conflicts triggered due to row removal via on-access
pruning as well. And perhaps also for btree killtuples.  I think those are the
common cases for catalog tables.

Greetings,

Andres Freund