Date: Thu, 11 Aug 2022 16:09:21 -0700
From: Nathan Bossart <nathandbossart@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Simon Riggs <simon.riggs@enterprisedb.com>,
	Robert Haas <robertmhaas@gmail.com>,
	"Bossart, Nathan" <bossartn@amazon.com>,
	Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>,
	Maxim Orlov <orlovmg@gmail.com>, Amul Sul <sulamul@gmail.com>,
	Bruce Momjian <bruce@momjian.us>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: O(n) tasks cause lengthy startups and checkpoints
Message-ID: <20220811230921.GA1707962@nathanxps13>
References: <20220218205111.GA3447220@nathanxps13>
 <CANbhV-H9=DOBdpKE77XB_hiPDKVre-+1vuLvKWTke07Y3K1P5w@mail.gmail.com>
 <CA+TgmoYkDY7r8PwAGhD3tMqVAeKBfY-_7miJKcj-ZzPWN0UYzQ@mail.gmail.com>
 <20220623171552.GA96638@nathanxps13>
 <CANbhV-GjpUi43Kyts+ib+j2=X_+1_AVh+CSdj-ow=M5aZqvfuQ@mail.gmail.com>
 <20220702220554.GA1044460@nathanxps13>
 <20220702225456.zit5kjdtdfqmjujt@alap3.anarazel.de>
 <20220703170754.GA1048244@nathanxps13>
 <20220703172732.wembjsb55xl63vuw@awork3.anarazel.de>
 <20220706165110.GB2135371@nathanxps13>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="X1bOJ3K7DJ5YkBrT"
Content-Disposition: inline
In-Reply-To: <20220706165110.GB2135371@nathanxps13>
Archived-At: 
 <https://www.postgresql.org/message-id/20220811230921.GA1707962%40nathanxps13>
Precedence: bulk


--X1bOJ3K7DJ5YkBrT
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Wed, Jul 06, 2022 at 09:51:10AM -0700, Nathan Bossart wrote:
> Here's a new revision where I've attempted to address all the feedback I've
> received thus far.  Notably, the custodian now uses a queue for registering
> tasks and determining which tasks to execute.  Other changes include
> splitting the temporary file functions apart to avoid consecutive boolean
> flags, using a timestamp instead of an integer for the staging name for
> temporary directories, moving temporary directories to a dedicated
> directory so that the custodian doesn't need to scan relation files,
> ERROR-ing when something goes wrong when cleaning up temporary files,
> executing requested tasks immediately in single-user mode, and more.

Here is a rebased patch set for cfbot.  There are no other differences
between v7 and v8.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

--X1bOJ3K7DJ5YkBrT
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
	filename="v8-0001-Introduce-custodian.patch"