Feedback-ID: id4a34324:Fastmail
Date: Tue, 12 Jul 2022 10:18:10 -0700
From: Andres Freund <andres@anarazel.de>
To: Melanie Plageman <melanieplageman@gmail.com>
Cc: Justin Pryzby <pryzby@telsasoft.com>,
	Alvaro Herrera <alvherre@alvh.no-ip.org>,
	Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
	Magnus Hagander <magnus@hagander.net>,
	Pg Hackers <pgsql-hackers@postgresql.org>,
	Lukas Fittl <lukas@fittl.com>,
	Thomas Munro <thomas.munro@gmail.com>
Subject: Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and
 more?)
Message-ID: <20220712171810.7f2ppc45torgaavu@awork3.anarazel.de>
References: 
 <CAAKRu_bwAmyQ=3SfxG3Jb8rRUBjoMwJtT99D6gG5fqqLmVpCQA@mail.gmail.com>
 <20211216201837.fm73l7zixfjknttd@alap3.anarazel.de>
 <CAAKRu_ZKgdacVqLjjfpp6zAnd_Ks9vTqU3UOh-JkPWvWMJ6KEw@mail.gmail.com>
 <CAAKRu_Y2t8uOhMBU461foxJd_C0gTyND-wZzvE8d-FLFtJOksw@mail.gmail.com>
 <CAAKRu_YbiJxFiEUi_d_UW_DWc=rnP4vJmBNgUHmXPr5awfQPfA@mail.gmail.com>
 <CAAKRu_ZJkk1KUZS-KAaa3EZ2e1o2YD7w8-NF-po5dTHM7cF=Rg@mail.gmail.com>
 <20220322001505.fgysjvihvagduete@alap3.anarazel.de>
 <CAAKRu_Y_B9pLm1-b6A5CfRhuWEMMy2iMy1zNG0_3GhejD-ZsMg@mail.gmail.com>
 <20220706192047.sikr2vuwvpslqdrw@alap3.anarazel.de>
 <CAAKRu_YKc9e_=CZ4RB97Krx9TQb-zfNMoqdP45AYs5uZFqXeSA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAAKRu_YKc9e_=CZ4RB97Krx9TQb-zfNMoqdP45AYs5uZFqXeSA@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/20220712171810.7f2ppc45torgaavu%40awork3.anarazel.de>
Precedence: bulk

Hi,

On 2022-07-11 22:22:28 -0400, Melanie Plageman wrote:
> Yes, per an off list suggestion by you, I have changed the tests to use a
> sum of writes. I've also added a test for IOPATH_LOCAL and fixed some of
> the missing calls to count IO Operations for IOPATH_LOCAL and
> IOPATH_STRATEGY.
> 
> I struggled to come up with a way to test writes for a particular
> type of backend are counted correctly since a dirty buffer could be
> written out by another type of backend before the target BackendType has
> a chance to write it out.

I guess temp file writes would be reliably done by one backend... Don't have a
good idea otherwise.


> I also struggled to come up with a way to test IO operations for
> background workers. I'm not sure of a way to deterministically have a
> background worker do a particular kind of IO in a test scenario.

I think it's perfectly fine to not test that - for it to be broken we'd have
to somehow screw up setting the backend type. Everything else is the same as
other types of backends anyway.

If you *do* want to test it, you probably could use
SET parallel_leader_participation = false;
SET force_parallel_mode = 'regress';
SELECT something_triggering_io;


> I'm not sure how to cause a strategy "extend" for testing.

COPY into a table should work. But might be unattractive due to the size of of
the COPY ringbuffer.


> > Would be nice to have something testing that the ringbuffer stats stuff
> > does something sensible - that feels not entirely trivial.
> >
> >
> I've added a test to test that reused strategy buffers are counted as
> allocs. I would like to add a test which checks that if a buffer in the
> ring is pinned and thus not reused, that it is not counted as a strategy
> alloc, but I found it challenging without a way to pause vacuuming, pin
> a buffer, then resume vacuuming.

Yea, that's probably too hard to make reliable to be worth it.

Greetings,

Andres Freund