Message-ID: <a15f904a70924ffa4ca25c3c744cff31e0e6e143.camel@gmail.com>
Subject: Progress report of CREATE INDEX for nested partitioned tables
From: Ilya Gladyshev <ilya.v.gladyshev@gmail.com>
To: pgsql-hackers@lists.postgresql.org
Date: Sat, 10 Dec 2022 12:18:32 +0400
Content-Type: multipart/mixed; boundary="=-JOwAY565k3uxsdaS0EJl"
User-Agent: Evolution 3.38.3-1 
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/a15f904a70924ffa4ca25c3c744cff31e0e6e143.camel%40gmail.com>
Precedence: bulk


--=-JOwAY565k3uxsdaS0EJl
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Hi,

I have noticed that progress reporting for CREATE INDEX of partitioned
tables seems to be working poorly for nested partitioned tables. In
particular, it overwrites total and done partitions count when it
recurses down to child partitioned tables and it only reports top-level
partitions. So it's not hard to see something like this during CREATE
INDEX now:

postgres=# select partitions_total, partitions_done from
pg_stat_progress_create_index ;
 partitions_total | partitions_done 
------------------+-----------------
                1 |               2
(1 row)


I changed current behaviour to report the total number of partitions in
the inheritance tree and fixed recursion in the attached patch. I used
a static variable to keep the counter to avoid ABI breakage of
DefineIndex, so that we could backpatch this to previous versions.

Thanks,
Ilya Gladyshev

--=-JOwAY565k3uxsdaS0EJl
Content-Disposition: attachment;
	filename="0001-report-top-parent-progress-for-CREATE-INDEX.patch"
Content-Type: text/x-patch;
	name="0001-report-top-parent-progress-for-CREATE-INDEX.patch";
	charset="UTF-8"
Content-Transfer-Encoding: base64

RnJvbSAzNDJjYWEzZDRjZTI3M2IxNGE2OTk4YzIwYzMyYjg2MmQyZDRjODkwIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBJbHlhIEdsYWR5c2hldiA8aWx5YS52LmdsYWR5c2hldkBnbWFp
bC5jb20+CkRhdGU6IEZyaSwgOSBEZWMgMjAyMiAyMzoxNzoyOSArMDQwMApTdWJqZWN0OiBbUEFU
Q0hdIHJlcG9ydCB0b3AgcGFyZW50IHByb2dyZXNzIGZvciBDUkVBVEUgSU5ERVgKCi0tLQogc3Jj
L2JhY2tlbmQvY29tbWFuZHMvaW5kZXhjbWRzLmMgfCAzNiArKysrKysrKysrKysrKysrKysrKysr
KysrKy0tLS0tLQogMSBmaWxlIGNoYW5nZWQsIDMwIGluc2VydGlvbnMoKyksIDYgZGVsZXRpb25z
KC0pCgpkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvY29tbWFuZHMvaW5kZXhjbWRzLmMgYi9zcmMv
YmFja2VuZC9jb21tYW5kcy9pbmRleGNtZHMuYwppbmRleCBiNWI4NjBjM2FiLi44MDU1N2RhZDhk
IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9jb21tYW5kcy9pbmRleGNtZHMuYworKysgYi9zcmMv
YmFja2VuZC9jb21tYW5kcy9pbmRleGNtZHMuYwpAQCAtMTI5LDYgKzEyOSwxMiBAQCB0eXBlZGVm
IHN0cnVjdCBSZWluZGV4RXJyb3JJbmZvCiAJY2hhcgkJcmVsa2luZDsKIH0gUmVpbmRleEVycm9y
SW5mbzsKIAorLyoKKyAqIFVzZWQgdG8gcmVwb3J0IHRoZSBudW1iZXIgb2YgcGFydGl0aW9ucywK
KyAqIHRoYXQgd2VyZSBwcm9jZXNzZWQgZHVyaW5nIGluZGV4IGNyZWF0aW9uLgorICovCitzdGF0
aWMgaW50IGNyZWF0ZV9pbmRleF9wYXJ0c19kb25lOworCiAvKgogICogQ2hlY2tJbmRleENvbXBh
dGlibGUKICAqCQlEZXRlcm1pbmUgd2hldGhlciBhbiBleGlzdGluZyBpbmRleCBkZWZpbml0aW9u
IGlzIGNvbXBhdGlibGUgd2l0aCBhCkBAIC0xMjE4LDggKzEyMjQsMTggQEAgRGVmaW5lSW5kZXgo
T2lkIHJlbGF0aW9uSWQsCiAJCQlSZWxhdGlvbglwYXJlbnRJbmRleDsKIAkJCVR1cGxlRGVzYwlw
YXJlbnREZXNjOwogCi0JCQlwZ3N0YXRfcHJvZ3Jlc3NfdXBkYXRlX3BhcmFtKFBST0dSRVNTX0NS
RUFURUlEWF9QQVJUSVRJT05TX1RPVEFMLAotCQkJCQkJCQkJCSBucGFydHMpOworCQkJaWYgKCFP
aWRJc1ZhbGlkKHBhcmVudEluZGV4SWQpKQorCQkJeworCQkJCUxpc3QgKmluaHM7CisKKwkJCQkv
KiBSZXNldCBjb3VudGVyIG9mIGRvbmUgcGFydGl0aW9ucyB3aGVuIHByb2Nlc3Npbmcgcm9vdCBp
bmRleCAqLworCQkJCWNyZWF0ZV9pbmRleF9wYXJ0c19kb25lID0gMDsKKwkJCQlpbmhzID0gZmlu
ZF9hbGxfaW5oZXJpdG9ycyhyZWxhdGlvbklkLCBOb0xvY2ssIE5VTEwpOworCQkJCS8qIGluaGVy
aXRhbmNlIHRyZWUgc2l6ZSB3aXRob3V0IHJvb3QgaXRzZWxmICovCisJCQkJcGdzdGF0X3Byb2dy
ZXNzX3VwZGF0ZV9wYXJhbShQUk9HUkVTU19DUkVBVEVJRFhfUEFSVElUSU9OU19UT1RBTCwKKwkJ
CQkJCQkJCQkJIGxpc3RfbGVuZ3RoKGluaHMpIC0gMSk7CisJCQkJbGlzdF9mcmVlKGluaHMpOwor
CQkJfQogCiAJCQkvKiBNYWtlIGEgbG9jYWwgY29weSBvZiBwYXJ0ZGVzYy0+b2lkc1tdLCBqdXN0
IGZvciBzYWZldHkgKi8KIAkJCW1lbWNweShwYXJ0X29pZHMsIHBhcnRkZXNjLT5vaWRzLCBzaXpl
b2YoT2lkKSAqIG5wYXJ0cyk7CkBAIC0xNDMxLDggKzE0NDcsNiBAQCBEZWZpbmVJbmRleChPaWQg
cmVsYXRpb25JZCwKIAkJCQkJCQkJCQkgICBjaGlsZF9zYXZlX3NlY19jb250ZXh0KTsKIAkJCQl9
CiAKLQkJCQlwZ3N0YXRfcHJvZ3Jlc3NfdXBkYXRlX3BhcmFtKFBST0dSRVNTX0NSRUFURUlEWF9Q
QVJUSVRJT05TX0RPTkUsCi0JCQkJCQkJCQkJCSBpICsgMSk7CiAJCQkJZnJlZV9hdHRybWFwKGF0
dG1hcCk7CiAJCQl9CiAKQEAgLTE0NjUsMTMgKzE0NzksMTcgQEAgRGVmaW5lSW5kZXgoT2lkIHJl
bGF0aW9uSWQsCiAKIAkJLyoKIAkJICogSW5kZXhlcyBvbiBwYXJ0aXRpb25lZCB0YWJsZXMgYXJl
IG5vdCB0aGVtc2VsdmVzIGJ1aWx0LCBzbyB3ZSdyZQotCQkgKiBkb25lIGhlcmUuCisJCSAqIGRv
bmUgaGVyZS4gSWYgaXQgaXMgYSBjaGlsZCBwYXJ0aXRpb25lZCB0YWJsZSwgaW5jcmVtZW50IGRv
bmUgcGFydHMgY291bnRlci4KIAkJICovCiAJCUF0RU9YYWN0X0dVQyhmYWxzZSwgcm9vdF9zYXZl
X25lc3RsZXZlbCk7CiAJCVNldFVzZXJJZEFuZFNlY0NvbnRleHQocm9vdF9zYXZlX3VzZXJpZCwg
cm9vdF9zYXZlX3NlY19jb250ZXh0KTsKIAkJdGFibGVfY2xvc2UocmVsLCBOb0xvY2spOwogCQlp
ZiAoIU9pZElzVmFsaWQocGFyZW50SW5kZXhJZCkpCiAJCQlwZ3N0YXRfcHJvZ3Jlc3NfZW5kX2Nv
bW1hbmQoKTsKKwkJZWxzZQorCQkJcGdzdGF0X3Byb2dyZXNzX3VwZGF0ZV9wYXJhbShQUk9HUkVT
U19DUkVBVEVJRFhfUEFSVElUSU9OU19ET05FLAorCQkJCQkJCQkJCSArK2NyZWF0ZV9pbmRleF9w
YXJ0c19kb25lKTsKKwogCQlyZXR1cm4gYWRkcmVzczsKIAl9CiAKQEAgLTE0ODMsOSArMTUwMSwx
NSBAQCBEZWZpbmVJbmRleChPaWQgcmVsYXRpb25JZCwKIAkJLyogQ2xvc2UgdGhlIGhlYXAgYW5k
IHdlJ3JlIGRvbmUsIGluIHRoZSBub24tY29uY3VycmVudCBjYXNlICovCiAJCXRhYmxlX2Nsb3Nl
KHJlbCwgTm9Mb2NrKTsKIAotCQkvKiBJZiB0aGlzIGlzIHRoZSB0b3AtbGV2ZWwgaW5kZXgsIHdl
J3JlIGRvbmUuICovCisJCS8qCisJCSAqIElmIHRoaXMgaXMgdGhlIHRvcC1sZXZlbCBpbmRleCwg
d2UncmUgZG9uZSwgb3RoZXJ3aXNlLCBpbmNyZW1lbnQKKwkJICogZG9uZSBwYXJ0aXRpb24gY291
bnRlci4KKwkJICovCiAJCWlmICghT2lkSXNWYWxpZChwYXJlbnRJbmRleElkKSkKIAkJCXBnc3Rh
dF9wcm9ncmVzc19lbmRfY29tbWFuZCgpOworCQllbHNlCisJCQlwZ3N0YXRfcHJvZ3Jlc3NfdXBk
YXRlX3BhcmFtKFBST0dSRVNTX0NSRUFURUlEWF9QQVJUSVRJT05TX0RPTkUsCisJCQkJCQkJCQkJ
ICsrY3JlYXRlX2luZGV4X3BhcnRzX2RvbmUpOwogCiAJCXJldHVybiBhZGRyZXNzOwogCX0KLS0g
CjIuMzAuMgoK


--=-JOwAY565k3uxsdaS0EJl--