MIME-Version: 1.0
References: 
 <CALj2ACWh+hgE4U2W_zKUVLjLBaonyMCOKDEB6oFLP91y3AccWw@mail.gmail.com>
 <CAExHW5v1Byk5PBB8o7A5iHgj9FcbzQDZQBCv-CoK89cV3dEbrg@mail.gmail.com>
 <CALj2ACW-ELOF5JT2zPavs95wbZ0BrLPrqvSZ7Ac+pjxCkmXtEQ@mail.gmail.com>
In-Reply-To: 
 <CALj2ACW-ELOF5JT2zPavs95wbZ0BrLPrqvSZ7Ac+pjxCkmXtEQ@mail.gmail.com>
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Tue, 2 Aug 2022 11:52:02 +0530
Message-ID: 
 <CALj2ACWeUFhhnDJKm6R5YxCsF4K7aB2pmRMvqP0BVTxdyce3EA@mail.gmail.com>
Subject: Re: Progress report removal of temp files and temp relation files
 using ereport_startup_progress
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: multipart/mixed; boundary="0000000000001f01f505e53c2366"
Archived-At: 
 <https://www.postgresql.org/message-id/CALj2ACWeUFhhnDJKm6R5YxCsF4K7aB2pmRMvqP0BVTxdyce3EA%40mail.gmail.com>
Precedence: bulk

--0000000000001f01f505e53c2366
Content-Type: text/plain; charset="UTF-8"

On Thu, May 5, 2022 at 12:11 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
>
> On Mon, May 2, 2022 at 6:26 PM Ashutosh Bapat
> <ashutosh.bapat.oss@gmail.com> wrote:
> >
> > Hi Bharath,
> >
> >
> > On Sat, Apr 30, 2022 at 11:08 AM Bharath Rupireddy
> > <bharath.rupireddyforpostgres@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > At times, there can be many temp files (under pgsql_tmp) and temp
> > > relation files (under removal which after crash may take longer during
> > > which users have no clue about what's going on in the server before it
> > > comes up online.
> > >
> > > Here's a proposal to use ereport_startup_progress to report the
> > > progress of the file removal.
> > >
> > > Thoughts?
> >
> > The patch looks good to me.
> >
> > With this patch, the user would at least know which directory is being
> > scanned and how much time has elapsed.
>
> There's a problem with the patch, the timeout mechanism isn't being
> used by the postmaster process. Postmaster doesn't
> InitializeTimeouts() and doesn't register STARTUP_PROGRESS_TIMEOUT, I
> tried to make postmaster do that (attached a v2 patch) but make check
> fails.
>
> Now, I'm thinking if it's a good idea to let postmaster use timeouts at all?

Here's the v3 patch, which adds progress reports for temp file removal
under the pgsql_tmp directory and temporary relation files under the
pg_tblspc directory, regression tests pass with it.

Regards,
Bharath Rupireddy.

--0000000000001f01f505e53c2366
Content-Type: application/x-patch;
	name="v3-0001-Progress-report-removal-of-temp-files-and-temp-re.patch"
Content-Disposition: attachment;
	filename="v3-0001-Progress-report-removal-of-temp-files-and-temp-re.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_l6bqifj50>
X-Attachment-Id: f_l6bqifj50

RnJvbSBjMzE0OTIxOTFmZmNkNjIyMmQ3ZTQyOTE3Y2JkOTI4MzUxNjIzNzgwIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBCaGFyYXRoIFJ1cGlyZWRkeSA8YmhhcmF0aC5ydXBpcmVkZHlm
b3Jwb3N0Z3Jlc0BnbWFpbC5jb20+CkRhdGU6IFR1ZSwgMiBBdWcgMjAyMiAwNToxMDowNiArMDAw
MApTdWJqZWN0OiBbUEFUQ0ggdjNdIFByb2dyZXNzIHJlcG9ydCByZW1vdmFsIG9mIHRlbXAgZmls
ZXMgYW5kIHRlbXAgcmVsYXRpb24KIGZpbGVzCgpBdCB0aW1lcywgdGhlcmUgY2FuIGJlIG1hbnkg
dGVtcCBmaWxlcyAodW5kZXIgcGdzcWxfdG1wKSBhbmQgdGVtcApyZWxhdGlvbiBmaWxlcyAodW5k
ZXIgIHJlbW92YWwgd2hpY2ggYWZ0ZXIgY3Jhc2ggbWF5IHRha2UgbG9uZ2VyCmR1cmluZyB3aGlj
aCB1c2VycyBoYXZlIG5vIGNsdWUgYWJvdXQgd2hhdCdzIGdvaW5nIG9uIGluIHRoZQpzZXJ2ZXIg
YmVmb3JlIGl0IGNvbWVzIHVwIG9ubGluZS4gVGhpcyBwYXRjaCB1c2VzCmVyZXBvcnRfc3RhcnR1
cF9wcm9ncmVzcyB0byByZXBvcnQgdGhlIHByb2dyZXNzIG9mIHRoZSBmaWxlCnJlbW92YWwuCi0t
LQogc3JjL2JhY2tlbmQvcG9zdG1hc3Rlci9wb3N0bWFzdGVyLmMgfCAxMCArKysrKysrKystCiBz
cmMvYmFja2VuZC9zdG9yYWdlL2ZpbGUvZmQuYyAgICAgICB8IDEzICsrKysrKysrKysrKysKIHNy
Yy9iYWNrZW5kL3V0aWxzL21pc2MvdGltZW91dC5jICAgIHwgMTEgKysrKysrKysrLS0KIDMgZmls
ZXMgY2hhbmdlZCwgMzEgaW5zZXJ0aW9ucygrKSwgMyBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQg
YS9zcmMvYmFja2VuZC9wb3N0bWFzdGVyL3Bvc3RtYXN0ZXIuYyBiL3NyYy9iYWNrZW5kL3Bvc3Rt
YXN0ZXIvcG9zdG1hc3Rlci5jCmluZGV4IGU1NDFiMTZiZGIuLjQwYjUzMjc2YmUgMTAwNjQ0Ci0t
LSBhL3NyYy9iYWNrZW5kL3Bvc3RtYXN0ZXIvcG9zdG1hc3Rlci5jCisrKyBiL3NyYy9iYWNrZW5k
L3Bvc3RtYXN0ZXIvcG9zdG1hc3Rlci5jCkBAIC0xMTYsNiArMTE2LDcgQEAKICNpbmNsdWRlICJw
b3N0bWFzdGVyL2ludGVycnVwdC5oIgogI2luY2x1ZGUgInBvc3RtYXN0ZXIvcGdhcmNoLmgiCiAj
aW5jbHVkZSAicG9zdG1hc3Rlci9wb3N0bWFzdGVyLmgiCisjaW5jbHVkZSAicG9zdG1hc3Rlci9z
dGFydHVwLmgiCiAjaW5jbHVkZSAicG9zdG1hc3Rlci9zeXNsb2dnZXIuaCIKICNpbmNsdWRlICJy
ZXBsaWNhdGlvbi9sb2dpY2FsbGF1bmNoZXIuaCIKICNpbmNsdWRlICJyZXBsaWNhdGlvbi93YWxz
ZW5kZXIuaCIKQEAgLTY1Myw3ICs2NTQsNiBAQCBQb3N0bWFzdGVyTWFpbihpbnQgYXJnYywgY2hh
ciAqYXJndltdKQogCXBxc2lnbmFsX3BtKFNJR0lOVCwgcG1kaWUpOyAvKiBzZW5kIFNJR1RFUk0g
YW5kIHNodXQgZG93biAqLwogCXBxc2lnbmFsX3BtKFNJR1FVSVQsIHBtZGllKTsJLyogc2VuZCBT
SUdRVUlUIGFuZCBkaWUgKi8KIAlwcXNpZ25hbF9wbShTSUdURVJNLCBwbWRpZSk7CS8qIHdhaXQg
Zm9yIGNoaWxkcmVuIGFuZCBzaHV0IGRvd24gKi8KLQlwcXNpZ25hbF9wbShTSUdBTFJNLCBTSUdf
SUdOKTsJLyogaWdub3JlZCAqLwogCXBxc2lnbmFsX3BtKFNJR1BJUEUsIFNJR19JR04pOwkvKiBp
Z25vcmVkICovCiAJcHFzaWduYWxfcG0oU0lHVVNSMSwgc2lndXNyMV9oYW5kbGVyKTsJLyogbWVz
c2FnZSBmcm9tIGNoaWxkIHByb2Nlc3MgKi8KIAlwcXNpZ25hbF9wbShTSUdVU1IyLCBkdW1teV9o
YW5kbGVyKTsJLyogdW51c2VkLCByZXNlcnZlIGZvciBjaGlsZHJlbiAqLwpAQCAtNjg4LDYgKzY4
OCwxMSBAQCBQb3N0bWFzdGVyTWFpbihpbnQgYXJnYywgY2hhciAqYXJndltdKQogCXBxc2lnbmFs
X3BtKFNJR1hGU1osIFNJR19JR04pOwkvKiBpZ25vcmVkICovCiAjZW5kaWYKIAorCUluaXRpYWxp
emVUaW1lb3V0cygpOwkvKiBlc3RhYmxpc2hlcyBTSUdBTFJNIGhhbmRsZXIgKi8KKworCVJlZ2lz
dGVyVGltZW91dChTVEFSVFVQX1BST0dSRVNTX1RJTUVPVVQsCisJCQkJCXN0YXJ0dXBfcHJvZ3Jl
c3NfdGltZW91dF9oYW5kbGVyKTsKKwogCS8qCiAJICogT3B0aW9ucyBzZXR1cAogCSAqLwpAQCAt
MTExOSw2ICsxMTI0LDkgQEAgUG9zdG1hc3Rlck1haW4oaW50IGFyZ2MsIGNoYXIgKmFyZ3ZbXSkK
IAkvKiBXcml0ZSBvdXQgbm9uZGVmYXVsdCBHVUMgc2V0dGluZ3MgZm9yIGNoaWxkIHByb2Nlc3Nl
cyB0byB1c2UgKi8KIAl3cml0ZV9ub25kZWZhdWx0X3ZhcmlhYmxlcyhQR0NfUE9TVE1BU1RFUik7
CiAKKwkvKiBQcmVwYXJlIHRvIHJlcG9ydCBwcm9ncmVzcyBvZiB0aGUgdGVtcG9yYXJ5IGZpbGVz
IHJlbW92YWwgcGhhc2UgKi8KKwliZWdpbl9zdGFydHVwX3Byb2dyZXNzX3BoYXNlKCk7CisKIAkv
KgogCSAqIENsZWFuIG91dCB0aGUgdGVtcCBkaXJlY3RvcnkgdXNlZCB0byB0cmFuc21pdCBwYXJh
bWV0ZXJzIHRvIGNoaWxkCiAJICogcHJvY2Vzc2VzIChzZWUgaW50ZXJuYWxfZm9ya2V4ZWMsIGJl
bG93KS4gIFdlIG11c3QgZG8gdGhpcyBiZWZvcmUKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3N0
b3JhZ2UvZmlsZS9mZC5jIGIvc3JjL2JhY2tlbmQvc3RvcmFnZS9maWxlL2ZkLmMKaW5kZXggZjkw
NGY2MGMwOC4uMzdjMTNhNjBiYSAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvc3RvcmFnZS9maWxl
L2ZkLmMKKysrIGIvc3JjL2JhY2tlbmQvc3RvcmFnZS9maWxlL2ZkLmMKQEAgLTEwMiw2ICsxMDIs
NyBAQAogI2luY2x1ZGUgInN0b3JhZ2UvaXBjLmgiCiAjaW5jbHVkZSAidXRpbHMvZ3VjLmgiCiAj
aW5jbHVkZSAidXRpbHMvcmVzb3duZXJfcHJpdmF0ZS5oIgorI2luY2x1ZGUgInV0aWxzL3RpbWVv
dXQuaCIKIAogLyogRGVmaW5lIFBHX0ZMVVNIX0RBVEFfV09SS1MgaWYgd2UgaGF2ZSBhbiBpbXBs
ZW1lbnRhdGlvbiBmb3IgcGdfZmx1c2hfZGF0YSAqLwogI2lmIGRlZmluZWQoSEFWRV9TWU5DX0ZJ
TEVfUkFOR0UpCkBAIC0zMDkzLDYgKzMwOTQsMTIgQEAgUmVtb3ZlUGdUZW1wRmlsZXModm9pZCkK
IAlESVIJCSAgICpzcGNfZGlyOwogCXN0cnVjdCBkaXJlbnQgKnNwY19kZTsKIAorCS8qCisJICog
UHJlcGFyZSB0byByZXBvcnQgcHJvZ3Jlc3Mgb2YgdGhlIHRlbXBvcmFyeSBhbmQgdGVtcG9yYXJ5
IHJlbGF0aW9uIGZpbGVzCisJICogcmVtb3ZhbCBwaGFzZS4KKwkgKi8KKwliZWdpbl9zdGFydHVw
X3Byb2dyZXNzX3BoYXNlKCk7CisKIAkvKgogCSAqIEZpcnN0IHByb2Nlc3MgdGVtcCBmaWxlcyBp
biBwZ19kZWZhdWx0ICgkUEdEQVRBL2Jhc2UpCiAJICovCkBAIC0zMTY2LDYgKzMxNzMsOSBAQCBS
ZW1vdmVQZ1RlbXBGaWxlc0luRGlyKGNvbnN0IGNoYXIgKnRtcGRpcm5hbWUsIGJvb2wgbWlzc2lu
Z19vaywgYm9vbCB1bmxpbmtfYWxsKQogCQlzbnByaW50ZihybV9wYXRoLCBzaXplb2Yocm1fcGF0
aCksICIlcy8lcyIsCiAJCQkJIHRtcGRpcm5hbWUsIHRlbXBfZGUtPmRfbmFtZSk7CiAKKwkJZXJl
cG9ydF9zdGFydHVwX3Byb2dyZXNzKCJyZW1vdmluZyB0ZW1wb3JhcnkgZmlsZXMgdW5kZXIgcGdz
cWxfdG1wIGRpcmVjdG9yeSwgZWxhcHNlZCB0aW1lOiAlbGQuJTAyZCBzLCBjdXJyZW50IGZpbGU6
ICVzIiwKKwkJCQkJCQkJIHJtX3BhdGgpOworCiAJCWlmICh1bmxpbmtfYWxsIHx8CiAJCQlzdHJu
Y21wKHRlbXBfZGUtPmRfbmFtZSwKIAkJCQkJUEdfVEVNUF9GSUxFX1BSRUZJWCwKQEAgLTMyNTYs
NiArMzI2Niw5IEBAIFJlbW92ZVBnVGVtcFJlbGF0aW9uRmlsZXNJbkRic3BhY2UoY29uc3QgY2hh
ciAqZGJzcGFjZWRpcm5hbWUpCiAJCXNucHJpbnRmKHJtX3BhdGgsIHNpemVvZihybV9wYXRoKSwg
IiVzLyVzIiwKIAkJCQkgZGJzcGFjZWRpcm5hbWUsIGRlLT5kX25hbWUpOwogCisJCWVyZXBvcnRf
c3RhcnR1cF9wcm9ncmVzcygicmVtb3ZpbmcgdGVtcG9yYXJ5IHJlbGF0aW9uIGZpbGVzIHVuZGVy
IHBnX3RibHNwYyBkaXJlY3RvcnksIGVsYXBzZWQgdGltZTogJWxkLiUwMmQgcywgY3VycmVudCBm
aWxlOiAlcyIsCisJCQkJCQkJCSBybV9wYXRoKTsKKwogCQlpZiAodW5saW5rKHJtX3BhdGgpIDwg
MCkKIAkJCWVyZXBvcnQoTE9HLAogCQkJCQkoZXJyY29kZV9mb3JfZmlsZV9hY2Nlc3MoKSwKZGlm
ZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvdGltZW91dC5jIGIvc3JjL2JhY2tlbmQv
dXRpbHMvbWlzYy90aW1lb3V0LmMKaW5kZXggNmY1ZTA4YmMzMC4uY2M0YjlkNmFkNyAxMDA2NDQK
LS0tIGEvc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy90aW1lb3V0LmMKKysrIGIvc3JjL2JhY2tlbmQv
dXRpbHMvbWlzYy90aW1lb3V0LmMKQEAgLTE2LDYgKzE2LDcgQEAKIAogI2luY2x1ZGUgPHN5cy90
aW1lLmg+CiAKKyNpbmNsdWRlICJsaWJwcS9wcXNpZ25hbC5oIgogI2luY2x1ZGUgIm1pc2NhZG1p
bi5oIgogI2luY2x1ZGUgInN0b3JhZ2UvcHJvYy5oIgogI2luY2x1ZGUgInV0aWxzL3RpbWVvdXQu
aCIKQEAgLTM3NSw4ICszNzYsMTEgQEAgaGFuZGxlX3NpZ19hbGFybShTSUdOQUxfQVJHUykKIAkv
KgogCSAqIFNJR0FMUk0gaXMgYWx3YXlzIGNhdXNlIGZvciB3YWtpbmcgYW55dGhpbmcgd2FpdGlu
ZyBvbiB0aGUgcHJvY2VzcwogCSAqIGxhdGNoLgorCSAqCisJICogUG9zdG1hc3RlciBoYXMgbm8g
bGF0Y2ggYXNzb2NpYXRlZCB3aXRoIGl0LgogCSAqLwotCVNldExhdGNoKE15TGF0Y2gpOworCWlm
IChNeUxhdGNoKQorCQlTZXRMYXRjaChNeUxhdGNoKTsKIAogCS8qCiAJICogQWx3YXlzIHJlc2V0
IHNpZ25hbF9wZW5kaW5nLCBldmVuIGlmICFhbGFybV9lbmFibGVkLCBzaW5jZSBpbmRlZWQgbm8K
QEAgLTQ5NCw3ICs0OTgsMTAgQEAgSW5pdGlhbGl6ZVRpbWVvdXRzKHZvaWQpCiAJYWxsX3RpbWVv
dXRzX2luaXRpYWxpemVkID0gdHJ1ZTsKIAogCS8qIE5vdyBlc3RhYmxpc2ggdGhlIHNpZ25hbCBo
YW5kbGVyICovCi0JcHFzaWduYWwoU0lHQUxSTSwgaGFuZGxlX3NpZ19hbGFybSk7CisJaWYgKElz
UG9zdG1hc3RlckVudmlyb25tZW50KQorCQlwcXNpZ25hbF9wbShTSUdBTFJNLCBoYW5kbGVfc2ln
X2FsYXJtKTsKKwllbHNlCisJCXBxc2lnbmFsKFNJR0FMUk0sIGhhbmRsZV9zaWdfYWxhcm0pOwog
fQogCiAvKgotLSAKMi4zNC4xCgo=
--0000000000001f01f505e53c2366--