Message-ID: <4e9bfa724b301b55a2a242ebebcddea62b2e1292.camel@j-davis.com>
Subject: Re: HashAgg degenerate case
From: Jeff Davis <pgsql@j-davis.com>
To: David Rowley <dgrowleyml@gmail.com>
Cc: pgsql-bugs@postgresql.org
Date: Thu, 07 Nov 2024 20:26:09 -0800
In-Reply-To: 
 <CAApHDvr3byTEhJ+yZb91t3OqAHtsyADUea2Xe2zCnw+G4k0RpA@mail.gmail.com>
References: <a050b6067334cf513bf34f9c7bc925d651160f3a.camel@j-davis.com>
	 <CAApHDvr3byTEhJ+yZb91t3OqAHtsyADUea2Xe2zCnw+G4k0RpA@mail.gmail.com>
Content-Type: multipart/mixed; boundary="=-gyTe6PJa/U9Sm9/MEXVs"
User-Agent: Evolution 3.44.4-0ubuntu2 
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/4e9bfa724b301b55a2a242ebebcddea62b2e1292.camel%40j-davis.com>
Precedence: bulk

--=-gyTe6PJa/U9Sm9/MEXVs
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, 2024-11-06 at 15:50 +1300, David Rowley wrote:
> I don't think it could be that exactly though as that could lead to
> JIT compilation over and over again from the following chain of
> function calls: build_hash_tables() -> build_hash_table() ->
> BuildTupleHashTableExt() -> ExecBuildGroupingEqual() ->
> ExecReadyExpr() -> jit_compile_expr()

New patch attached. It extends ResetTupleHashTable() so that the caller
can force setting a new nbuckets, which recreates the tuplehash_hash.

agg_refill_hash_table() now uses that to recalculate and set the
nbuckets for the current grouping set's hashtab, and frees the hashtab
for all other grouping sets (it "frees" them by setting to a minimal
size bucket array).

Regards,
	Jeff Davis


--=-gyTe6PJa/U9Sm9/MEXVs
Content-Disposition: attachment;
	filename="v2-0001-HashAgg-free-hash-table-memory-each-iteration.patch"
Content-Type: text/x-patch;
	name="v2-0001-HashAgg-free-hash-table-memory-each-iteration.patch";
	charset="UTF-8"
Content-Transfer-Encoding: base64

RnJvbSBlYWMxY2UyYTNiNDllMGZhNTRkOTc2OTVhNTk5YzY2NTM4NDA3ZjdhIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBKZWZmIERhdmlzIDxqZWZmQGotZGF2aXMuY29tPgpEYXRlOiBU
aHUsIDcgTm92IDIwMjQgMjA6MTU6NDQgLTA4MDAKU3ViamVjdDogW1BBVENIIHYyXSBIYXNoQWdn
OiBmcmVlIGhhc2ggdGFibGUgbWVtb3J5IGVhY2ggaXRlcmF0aW9uLgoKLS0tCiBzcmMvYmFja2Vu
ZC9leGVjdXRvci9leGVjR3JvdXBpbmcuYyAgICAgICB8IDIyICsrKysrKysrKysrKysrLS0tCiBz
cmMvYmFja2VuZC9leGVjdXRvci9ub2RlQWdnLmMgICAgICAgICAgICB8IDMwICsrKysrKysrKysr
KysrKysrKysrKy0tCiBzcmMvYmFja2VuZC9leGVjdXRvci9ub2RlUmVjdXJzaXZldW5pb24uYyB8
ICAyICstCiBzcmMvYmFja2VuZC9leGVjdXRvci9ub2RlU2V0T3AuYyAgICAgICAgICB8ICAyICst
CiBzcmMvYmFja2VuZC9leGVjdXRvci9ub2RlU3VicGxhbi5jICAgICAgICB8ICA0ICstLQogc3Jj
L2luY2x1ZGUvZXhlY3V0b3IvZXhlY3V0b3IuaCAgICAgICAgICAgfCAgMiArLQogNiBmaWxlcyBj
aGFuZ2VkLCA1MSBpbnNlcnRpb25zKCspLCAxMSBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9z
cmMvYmFja2VuZC9leGVjdXRvci9leGVjR3JvdXBpbmcuYyBiL3NyYy9iYWNrZW5kL2V4ZWN1dG9y
L2V4ZWNHcm91cGluZy5jCmluZGV4IDc3NGU0ZGU4ODIuLmRkNWMzZTg1MmEgMTAwNjQ0Ci0tLSBh
L3NyYy9iYWNrZW5kL2V4ZWN1dG9yL2V4ZWNHcm91cGluZy5jCisrKyBiL3NyYy9iYWNrZW5kL2V4
ZWN1dG9yL2V4ZWNHcm91cGluZy5jCkBAIC0yNzksMTMgKzI3OSwyNyBAQCBCdWlsZFR1cGxlSGFz
aFRhYmxlKFBsYW5TdGF0ZSAqcGFyZW50LAogCiAvKgogICogUmVzZXQgY29udGVudHMgb2YgdGhl
IGhhc2h0YWJsZSB0byBiZSBlbXB0eSwgcHJlc2VydmluZyBhbGwgdGhlIG5vbi1jb250ZW50Ci0g
KiBzdGF0ZS4gTm90ZSB0aGF0IHRoZSB0YWJsZWN4dCBwYXNzZWQgdG8gQnVpbGRUdXBsZUhhc2hU
YWJsZUV4dCgpIHNob3VsZAotICogYWxzbyBiZSByZXNldCwgb3RoZXJ3aXNlIHRoZXJlIHdpbGwg
YmUgbGVha3MuCisgKiBzdGF0ZS4gSWYgbmJ1Y2tldHMgaXMgLTEsIG9yIGlmIHRoZSBidWNrZXQg
YXJyYXkgaXMgYWxyZWFkeSB0aGUgcmlnaHQgc2l6ZSwKKyAqIHRoZW4gcHJlc2VydmUgaXQ7IG90
aGVyd2lzZSByZWJ1aWxkIGl0LiBOb3RlIHRoYXQgdGhlIHRhYmxlY3h0IHBhc3NlZCB0bworICog
QnVpbGRUdXBsZUhhc2hUYWJsZUV4dCgpIHNob3VsZCBhbHNvIGJlIHJlc2V0LCBvdGhlcndpc2Ug
dGhlcmUgd2lsbCBiZQorICogbGVha3MuCiAgKi8KIHZvaWQKLVJlc2V0VHVwbGVIYXNoVGFibGUo
VHVwbGVIYXNoVGFibGUgaGFzaHRhYmxlKQorUmVzZXRUdXBsZUhhc2hUYWJsZShUdXBsZUhhc2hU
YWJsZSBoYXNodGFibGUsIGxvbmcgbmJ1Y2tldHMpCiB7Ci0JdHVwbGVoYXNoX3Jlc2V0KGhhc2h0
YWJsZS0+aGFzaHRhYik7CisJdHVwbGVoYXNoX2hhc2ggKmhhc2h0YWIgPSBoYXNodGFibGUtPmhh
c2h0YWI7CisKKwlpZiAobmJ1Y2tldHMgPCAwIHx8IG5idWNrZXRzID09IGhhc2h0YWJsZS0+aGFz
aHRhYi0+c2l6ZSkKKwl7CisJCXR1cGxlaGFzaF9yZXNldChoYXNodGFiKTsKKwl9CisJZWxzZQor
CXsKKwkJTWVtb3J5Q29udGV4dCBtZXRhY3h0ID0gaGFzaHRhYi0+Y3R4OworCisJCXR1cGxlaGFz
aF9kZXN0cm95KGhhc2h0YWIpOworCQloYXNodGFibGUtPmhhc2h0YWIgPSB0dXBsZWhhc2hfY3Jl
YXRlKG1ldGFjeHQsIG5idWNrZXRzLCBoYXNodGFibGUpOworCX0KIH0KIAogLyoKZGlmZiAtLWdp
dCBhL3NyYy9iYWNrZW5kL2V4ZWN1dG9yL25vZGVBZ2cuYyBiL3NyYy9iYWNrZW5kL2V4ZWN1dG9y
L25vZGVBZ2cuYwppbmRleCA1M2VhZDc3ZWNlLi40MDkwMGE4OWJkIDEwMDY0NAotLS0gYS9zcmMv
YmFja2VuZC9leGVjdXRvci9ub2RlQWdnLmMKKysrIGIvc3JjL2JhY2tlbmQvZXhlY3V0b3Ivbm9k
ZUFnZy5jCkBAIC0xNDc3LDcgKzE0NzcsNyBAQCBidWlsZF9oYXNoX3RhYmxlcyhBZ2dTdGF0ZSAq
YWdnc3RhdGUpCiAKIAkJaWYgKHBlcmhhc2gtPmhhc2h0YWJsZSAhPSBOVUxMKQogCQl7Ci0JCQlS
ZXNldFR1cGxlSGFzaFRhYmxlKHBlcmhhc2gtPmhhc2h0YWJsZSk7CisJCQlSZXNldFR1cGxlSGFz
aFRhYmxlKHBlcmhhc2gtPmhhc2h0YWJsZSwgLTEpOwogCQkJY29udGludWU7CiAJCX0KIApAQCAt
MjYyMyw3ICsyNjIzLDMzIEBAIGFnZ19yZWZpbGxfaGFzaF90YWJsZShBZ2dTdGF0ZSAqYWdnc3Rh
dGUpCiAJLyogZnJlZSBtZW1vcnkgYW5kIHJlc2V0IGhhc2ggdGFibGVzICovCiAJUmVTY2FuRXhw
ckNvbnRleHQoYWdnc3RhdGUtPmhhc2hjb250ZXh0KTsKIAlmb3IgKGludCBzZXRubyA9IDA7IHNl
dG5vIDwgYWdnc3RhdGUtPm51bV9oYXNoZXM7IHNldG5vKyspCi0JCVJlc2V0VHVwbGVIYXNoVGFi
bGUoYWdnc3RhdGUtPnBlcmhhc2hbc2V0bm9dLmhhc2h0YWJsZSk7CisJeworCQlUdXBsZUhhc2hU
YWJsZSBoYXNodGFibGUgPSBhZ2dzdGF0ZS0+cGVyaGFzaFtzZXRub10uaGFzaHRhYmxlOworCisJ
CWlmIChzZXRubyA9PSBiYXRjaC0+c2V0bm8pCisJCXsKKwkJCS8qCisJCQkgKiBSZWNhbGN1bGF0
ZSBuYnVja2V0cyBmb3IgdGhpcyBiYXRjaC4gVGhpcyBpcyBpbXBvcnRhbnQgd2hlbiBhCisJCQkg
KiBwcmV2aW91cyBpdGVyYXRpb24gY2F1c2VkIHRoZSBudW1iZXIgb2YgYnVja2V0cyB0byBncm93
IHRvIGFuCisJCQkgKiB1bmV4cGVjdGVkbHkgbGFyZ2UgdmFsdWUsIGNyb3dkaW5nIG91dCBzcGFj
ZSBmb3IgdGhlIHR1cGxlcy4KKwkJCSAqLworCQkJbG9uZwkJbmJ1Y2tldHM7CisKKwkJCW5idWNr
ZXRzID0gaGFzaF9jaG9vc2VfbnVtX2J1Y2tldHMoYWdnc3RhdGUtPmhhc2hlbnRyeXNpemUsCisJ
CQkJCQkJCQkJCSAgIGJhdGNoLT5pbnB1dF9jYXJkLAorCQkJCQkJCQkJCQkgICBhZ2dzdGF0ZS0+
aGFzaF9tZW1fbGltaXQpOworCisJCQlSZXNldFR1cGxlSGFzaFRhYmxlKGhhc2h0YWJsZSwgbmJ1
Y2tldHMpOworCQl9CisJCWVsc2UKKwkJeworCQkJLyoKKwkJCSAqIEZyZWUgbWVtb3J5IGJ1dCBs
ZWF2ZSBtZXRhZGF0YSBpbnRhY3QgYnkgc2V0dGluZyBuYnVja2V0cyB0bworCQkJICogbWluaW1h
bCB2YWx1ZS4KKwkJCSAqLworCQkJUmVzZXRUdXBsZUhhc2hUYWJsZShoYXNodGFibGUsIDEpOwor
CQl9CisJfQogCiAJYWdnc3RhdGUtPmhhc2hfbmdyb3Vwc19jdXJyZW50ID0gMDsKIApkaWZmIC0t
Z2l0IGEvc3JjL2JhY2tlbmQvZXhlY3V0b3Ivbm9kZVJlY3Vyc2l2ZXVuaW9uLmMgYi9zcmMvYmFj
a2VuZC9leGVjdXRvci9ub2RlUmVjdXJzaXZldW5pb24uYwppbmRleCAyMmU3YjgzYjJlLi4wYTY3
MDgzMDc5IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9leGVjdXRvci9ub2RlUmVjdXJzaXZldW5p
b24uYworKysgYi9zcmMvYmFja2VuZC9leGVjdXRvci9ub2RlUmVjdXJzaXZldW5pb24uYwpAQCAt
MzI4LDcgKzMyOCw3IEBAIEV4ZWNSZVNjYW5SZWN1cnNpdmVVbmlvbihSZWN1cnNpdmVVbmlvblN0
YXRlICpub2RlKQogCiAJLyogRW1wdHkgaGFzaHRhYmxlIGlmIG5lZWRlZCAqLwogCWlmIChwbGFu
LT5udW1Db2xzID4gMCkKLQkJUmVzZXRUdXBsZUhhc2hUYWJsZShub2RlLT5oYXNodGFibGUpOwor
CQlSZXNldFR1cGxlSGFzaFRhYmxlKG5vZGUtPmhhc2h0YWJsZSwgLTEpOwogCiAJLyogcmVzZXQg
cHJvY2Vzc2luZyBzdGF0ZSAqLwogCW5vZGUtPnJlY3Vyc2luZyA9IGZhbHNlOwpkaWZmIC0tZ2l0
IGEvc3JjL2JhY2tlbmQvZXhlY3V0b3Ivbm9kZVNldE9wLmMgYi9zcmMvYmFja2VuZC9leGVjdXRv
ci9ub2RlU2V0T3AuYwppbmRleCBhOGFjNjhiNDgyLi5lOWEwYmRkMzZkIDEwMDY0NAotLS0gYS9z
cmMvYmFja2VuZC9leGVjdXRvci9ub2RlU2V0T3AuYworKysgYi9zcmMvYmFja2VuZC9leGVjdXRv
ci9ub2RlU2V0T3AuYwpAQCAtNjM2LDcgKzYzNiw3IEBAIEV4ZWNSZVNjYW5TZXRPcChTZXRPcFN0
YXRlICpub2RlKQogCS8qIEFuZCByZWJ1aWxkIGVtcHR5IGhhc2h0YWJsZSBpZiBuZWVkZWQgKi8K
IAlpZiAoKChTZXRPcCAqKSBub2RlLT5wcy5wbGFuKS0+c3RyYXRlZ3kgPT0gU0VUT1BfSEFTSEVE
KQogCXsKLQkJUmVzZXRUdXBsZUhhc2hUYWJsZShub2RlLT5oYXNodGFibGUpOworCQlSZXNldFR1
cGxlSGFzaFRhYmxlKG5vZGUtPmhhc2h0YWJsZSwgLTEpOwogCQlub2RlLT50YWJsZV9maWxsZWQg
PSBmYWxzZTsKIAl9CiAKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2V4ZWN1dG9yL25vZGVTdWJw
bGFuLmMgYi9zcmMvYmFja2VuZC9leGVjdXRvci9ub2RlU3VicGxhbi5jCmluZGV4IDIzNjIyMmQ3
MmEuLjc5MmMzZTI0OTQgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL2V4ZWN1dG9yL25vZGVTdWJw
bGFuLmMKKysrIGIvc3JjL2JhY2tlbmQvZXhlY3V0b3Ivbm9kZVN1YnBsYW4uYwpAQCAtNTI5LDcg
KzUyOSw3IEBAIGJ1aWxkU3ViUGxhbkhhc2goU3ViUGxhblN0YXRlICpub2RlLCBFeHByQ29udGV4
dCAqZWNvbnRleHQpCiAJCW5idWNrZXRzID0gMTsKIAogCWlmIChub2RlLT5oYXNodGFibGUpCi0J
CVJlc2V0VHVwbGVIYXNoVGFibGUobm9kZS0+aGFzaHRhYmxlKTsKKwkJUmVzZXRUdXBsZUhhc2hU
YWJsZShub2RlLT5oYXNodGFibGUsIC0xKTsKIAllbHNlCiAJCW5vZGUtPmhhc2h0YWJsZSA9IEJ1
aWxkVHVwbGVIYXNoVGFibGVFeHQobm9kZS0+cGFyZW50LAogCQkJCQkJCQkJCQkJIG5vZGUtPmRl
c2NSaWdodCwKQEAgLTU1Nyw3ICs1NTcsNyBAQCBidWlsZFN1YlBsYW5IYXNoKFN1YlBsYW5TdGF0
ZSAqbm9kZSwgRXhwckNvbnRleHQgKmVjb250ZXh0KQogCQl9CiAKIAkJaWYgKG5vZGUtPmhhc2hu
dWxscykKLQkJCVJlc2V0VHVwbGVIYXNoVGFibGUobm9kZS0+aGFzaG51bGxzKTsKKwkJCVJlc2V0
VHVwbGVIYXNoVGFibGUobm9kZS0+aGFzaG51bGxzLCAtMSk7CiAJCWVsc2UKIAkJCW5vZGUtPmhh
c2hudWxscyA9IEJ1aWxkVHVwbGVIYXNoVGFibGVFeHQobm9kZS0+cGFyZW50LAogCQkJCQkJCQkJ
CQkJCSBub2RlLT5kZXNjUmlnaHQsCmRpZmYgLS1naXQgYS9zcmMvaW5jbHVkZS9leGVjdXRvci9l
eGVjdXRvci5oIGIvc3JjL2luY2x1ZGUvZXhlY3V0b3IvZXhlY3V0b3IuaAppbmRleCA2OWMzZWJm
ZjAwLi5mMjEwODYxZWM4IDEwMDY0NAotLS0gYS9zcmMvaW5jbHVkZS9leGVjdXRvci9leGVjdXRv
ci5oCisrKyBiL3NyYy9pbmNsdWRlL2V4ZWN1dG9yL2V4ZWN1dG9yLmgKQEAgLTE2MSw3ICsxNjEs
NyBAQCBleHRlcm4gVHVwbGVIYXNoRW50cnkgRmluZFR1cGxlSGFzaEVudHJ5KFR1cGxlSGFzaFRh
YmxlIGhhc2h0YWJsZSwKIAkJCQkJCQkJCQkgVHVwbGVUYWJsZVNsb3QgKnNsb3QsCiAJCQkJCQkJ
CQkJIEV4cHJTdGF0ZSAqZXFjb21wLAogCQkJCQkJCQkJCSBGbWdySW5mbyAqaGFzaGZ1bmN0aW9u
cyk7Ci1leHRlcm4gdm9pZCBSZXNldFR1cGxlSGFzaFRhYmxlKFR1cGxlSGFzaFRhYmxlIGhhc2h0
YWJsZSk7CitleHRlcm4gdm9pZCBSZXNldFR1cGxlSGFzaFRhYmxlKFR1cGxlSGFzaFRhYmxlIGhh
c2h0YWJsZSwgbG9uZyBuYnVja2V0cyk7CiAKIC8qCiAgKiBwcm90b3R5cGVzIGZyb20gZnVuY3Rp
b25zIGluIGV4ZWNKdW5rLmMKLS0gCjIuMzQuMQoK


--=-gyTe6PJa/U9Sm9/MEXVs--