MIME-Version: 1.0
References: 
 <CALj2ACWh+hgE4U2W_zKUVLjLBaonyMCOKDEB6oFLP91y3AccWw@mail.gmail.com>
In-Reply-To: 
 <CALj2ACWh+hgE4U2W_zKUVLjLBaonyMCOKDEB6oFLP91y3AccWw@mail.gmail.com>
From: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Date: Mon, 2 May 2022 18:26:33 +0530
Message-ID: 
 <CAExHW5v1Byk5PBB8o7A5iHgj9FcbzQDZQBCv-CoK89cV3dEbrg@mail.gmail.com>
Subject: Re: Progress report removal of temp files and temp relation files
 using ereport_startup_progress
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: 
 <https://www.postgresql.org/message-id/CAExHW5v1Byk5PBB8o7A5iHgj9FcbzQDZQBCv-CoK89cV3dEbrg%40mail.gmail.com>
Precedence: bulk

Hi Bharath,


On Sat, Apr 30, 2022 at 11:08 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
>
> Hi,
>
> At times, there can be many temp files (under pgsql_tmp) and temp
> relation files (under removal which after crash may take longer during
> which users have no clue about what's going on in the server before it
> comes up online.
>
> Here's a proposal to use ereport_startup_progress to report the
> progress of the file removal.
>
> Thoughts?

The patch looks good to me.

With this patch, the user would at least know which directory is being
scanned and how much time has elapsed. It would be better to know how
much work is remaining. I could not find a way to estimate the number
of files in the directory so that we can extrapolate elapsed time and
estimate the remaining time. Well, we could loop the output of
opendir() twice, first to estimate and then for the actual work. This
might actually work, if the time to delete all the files is very high
compared to the time it takes to scan all the files/directories.

Another possibility is to scan the sorted output of opendir() thus
using the current file name to estimate remaining files in a very
crude and inaccurate way. That doesn't look attractive either. I can't
think of any better way to estimate the remaining time.

But at least with this patch, a user knows which files have been
deleted, guessing how far, in the directory structure, the process has
reached. S/he can then take a look at the remaining contents of the
directory to estimate how much it should wait.

-- 
Best Wishes,
Ashutosh Bapat