Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oD6w7-000119-S8 for pgsql-hackers@arkaria.postgresql.org; Sun, 17 Jul 2022 16:16:04 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1oD6w6-0001yd-OA for pgsql-hackers@arkaria.postgresql.org; Sun, 17 Jul 2022 16:16:02 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oD6w6-0001r9-3G for pgsql-hackers@lists.postgresql.org; Sun, 17 Jul 2022 16:16:02 +0000 Received: from mout-b-105.mailbox.org ([195.10.208.50]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oD6w2-0002PY-KG for pgsql-hackers@lists.postgresql.org; Sun, 17 Jul 2022 16:16:00 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-105.mailbox.org (Postfix) with ESMTPS id 4Lm9F60LXKz9sbh; Sun, 17 Jul 2022 18:15:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aboutsource.net; s=MBO0001; t=1658074554; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=G7d7csNvHf6HAIEDx8CGi4S60R48ZnWoSE5zIs5Uo/Y=; b=rT6gjSVlFHNq7co88SDCc+OYRhLn7nfsiySCHJB9KsOpqVr0BZOm/2x9DzXxkLWgqETzQ6 WlBUze78xZNw0osMM/fi7YTPqpTO0a9SI2nosDkjFehT45j2qR4FMqnwJzBbPHBXt3eCpK 6NX5kJ+twV40oibEYS8HEA+hjgGxN7TDpMux3qCKhoSJlJ2pJbW9Uv0BOR8Bp8UY41uhaZ fuVoiRyaqMuTWMh6CVzBPa87Xbgrjb6s6um5JaAtf3enM1OU1H4eK2O48mcSCtI9eerUKq v5X7z7Bl0DdIIqvXV8DCZVqKjo9ydtkQRJ71kEw5oq9vkFwnpP/Vezkb+mWwhQ== Content-Type: multipart/mixed; boundary="------------3vFP0glmrXXmlPYrw6PnzXth" Message-ID: <5ef15c6f-e108-53af-3d82-587fac394f1a@aboutsource.net> Date: Sun, 17 Jul 2022 18:15:51 +0200 MIME-Version: 1.0 Content-Language: de-DE To: Thomas Munro , Tom Lane Cc: PostgreSQL Hackers References: <9d160a44-7675-51e8-60cf-6d64b76db831@aboutsource.net> <4073858.1658027909@sss.pgh.pa.us> <4077925.1658029046@sss.pgh.pa.us> From: Martin Kalcher Subject: Re: Proposal to introduce a shuffle function to intarray extension In-Reply-To: X-Rspamd-Queue-Id: 4Lm9F60LXKz9sbh List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------3vFP0glmrXXmlPYrw6PnzXth Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Am 17.07.22 um 08:00 schrieb Thomas Munro: >> Actually ... is there a reason to bother with an intarray version >> at all, rather than going straight for an in-core anyarray function? >> It's not obvious to me that an int4-only version would have >> major performance advantages. > > Yeah, that seems like a good direction. If there is a performance > advantage to specialising, then perhaps we only have to specialise on > size, not type. Perhaps there could be a general function that > internally looks out for typbyval && typlen == 4, and dispatches to a > specialised 4-byte, and likewise for 8, if it can, and that'd already > be enough to cover int, bigint, float etc, without needing > specialisations for each type. I played around with the idea of an anyarray shuffle(). The hard part was to deal with arrays with variable length elements, as they can not be swapped easily in place. I solved it by creating an intermediate array of references to the elements. I'll attach a patch with the proof of concept. Unfortunatly it is already about 5 times slower than the specialised version and i am not sure if it is worth going down that road. Martin --------------3vFP0glmrXXmlPYrw6PnzXth Content-Type: text/x-patch; charset=UTF-8; name="0001-introduce-array_shuffle.patch" Content-Disposition: attachment; filename="0001-introduce-array_shuffle.patch" Content-Transfer-Encoding: base64 RnJvbSA4NWNiOTBjNzk0NTkxMzJjNDIyMGI4ZDc4NTg2ZGZmZTZlNTkwYmE5IE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBNYXJ0aW4gS2FsY2hlciA8bWFydGluLmthbGNoZXJA YWJvdXRzb3VyY2UubmV0PgpEYXRlOiBTdW4sIDE3IEp1bCAyMDIyIDE4OjA2OjA0ICswMjAw ClN1YmplY3Q6IFtQQVRDSF0gaW50cm9kdWNlIGFycmF5X3NodWZmbGUoKQoKLS0tCiBzcmMv YmFja2VuZC91dGlscy9hZHQvYXJyYXlmdW5jcy5jIHwgNzMgKysrKysrKysrKysrKysrKysr KysrKysrKysrKysrCiBzcmMvaW5jbHVkZS9jYXRhbG9nL3BnX3Byb2MuZGF0ICAgIHwgIDMg KysKIDIgZmlsZXMgY2hhbmdlZCwgNzYgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdpdCBhL3Ny Yy9iYWNrZW5kL3V0aWxzL2FkdC9hcnJheWZ1bmNzLmMgYi9zcmMvYmFja2VuZC91dGlscy9h ZHQvYXJyYXlmdW5jcy5jCmluZGV4IGZiMTY3ZjIyNmEuLmI2NTc2Y2UxNzggMTAwNjQ0Ci0t LSBhL3NyYy9iYWNrZW5kL3V0aWxzL2FkdC9hcnJheWZ1bmNzLmMKKysrIGIvc3JjL2JhY2tl bmQvdXRpbHMvYWR0L2FycmF5ZnVuY3MuYwpAQCAtNjg3MiwzICs2ODcyLDc2IEBAIHRyaW1f YXJyYXkoUEdfRlVOQ1RJT05fQVJHUykKIAogCVBHX1JFVFVSTl9EQVRVTShyZXN1bHQpOwog fQorCitEYXR1bQorYXJyYXlfc2h1ZmZsZShQR19GVU5DVElPTl9BUkdTKQoreworCUFycmF5 VHlwZSAgKmFycmF5LAorCQkJICAgKnJlc3VsdDsKKwljaGFyCSAgICphZGF0LAorCQkJICAg KnJkYXQ7CisJaW50CQkgICBuaXRlbXMsCisJCQkgICBpOworCU9pZCAJICAgZWxlbXR5cGU7 CisJVHlwZUNhY2hlRW50cnkgdHlwZW50cnk7CisJc3RydWN0IHsgY2hhciAqZGF0OyBzaXpl X3Qgc2l6ZTsgfSAqcmVmczsKKworCWFycmF5ID0gUEdfR0VUQVJHX0FSUkFZVFlQRV9QKDAp OworCisJaWYgKEFSUl9IQVNOVUxMKGFycmF5KSkKKwkJZXJlcG9ydChFUlJPUiwKKwkJCQko ZXJyY29kZShFUlJDT0RFX0ZFQVRVUkVfTk9UX1NVUFBPUlRFRCksCisJCQkJIGVycm1zZygi c2h1ZmZsaW5nIGFycmF5cyB3aXRoIG51bGwgZWxlbWVudHMgaXMgbm90IHN1cHBvcnRlZCIp KSk7CisKKwlpZiAoQVJSX05ESU0oYXJyYXkpID4gMSkKKwkJZXJlcG9ydChFUlJPUiwKKwkJ CQkoZXJyY29kZShFUlJDT0RFX0ZFQVRVUkVfTk9UX1NVUFBPUlRFRCksCisJCQkJIGVycm1z Zygic2h1ZmZsaW5nIG9mIG11bHRpZGltZW5zaW9uYWwgYXJyYXlzIGlzIG5vdCBzdXBwb3J0 ZWQiKSkpOworCisJZWxlbXR5cGUgPSBBUlJfRUxFTVRZUEUoYXJyYXkpOworCXR5cGVudHJ5 ID0gKmxvb2t1cF90eXBlX2NhY2hlKGVsZW10eXBlLCAwKTsKKwluaXRlbXMgPSBBcnJheUdl dE5JdGVtcyhBUlJfTkRJTShhcnJheSksIEFSUl9ESU1TKGFycmF5KSk7CisJcmVmcyA9IHBh bGxvYyhuaXRlbXMgKiAoc2l6ZW9mKGNoYXIgKikgKyBzaXplb2Yoc2l6ZV90KSkpOworCWFk YXQgPSBBUlJfREFUQV9QVFIoYXJyYXkpOworCisJLyoKKwkgKiBUbyBhbGxvdyBzaHVmZmxp bmcgb2YgYXJyYXlzIHdpdGggdmFyaWFibGUgbGVuZ3RoIGVsZW1lbnRzLCB3ZSBhcmUgZ29p bmcKKwkgKiB0byBzaHVmZmxlIHJlZmVyZW5jZXMgdG8gdGhlIGVsZW1lbnRzLCBiZWNhdXNl IHN3YXBwaW5nIHZhcmlhYmxlIGxlbmd0aAorCSAqIGVsZW1lbnRzIGluIGFuIGFycmF5IGlz IGNvbXBsaWNhdGVkLiBJbiBhIHNlY29uZCBzdGVwIHdlIHdpbGwgaXRlcmF0ZSB0aGUKKwkg KiBzaHVmZmxlZCByZWZlcmVuY2VzIGFuZCBjb3B5IHRoZSBlbGVtZW50cyB0byB0aGUgZGVz dGluYXRpb24gYXJyYXkuCisJICogV2UgYXJlIHVzaW5nIHRoZSAiaW5zaWRlLW91dCIgdmFy aWFudCBvZiB0aGUgRmlzaGVyLVlhdGVzIGFsZ29yaXRobSB0bworCSAqIHByb2R1Y2UgYSBz aHVmZmxlZCBhcnJheSBvZiByZWZlcmVuY2VzOiBBcHBlbmQgZWFjaCByZWZlcmVuY2UgdGhh biBzd2FwIGl0CisJICogd2l0aCBhIHJhZG9tbHktY2hvc2VuIHJlZmVuY2UuCisJICovCisJ Zm9yIChpID0gMDsgaSA8IG5pdGVtczsgaSsrKQorCXsKKwkJaW50IGogPSByYW5kb20oKSAl IChpICsgMSk7CisJCWNoYXIgKm5leHQgPSBhdHRfYWRkbGVuZ3RoX3BvaW50ZXIoYWRhdCwg dHlwZW50cnkudHlwbGVuLCBhZGF0KTsKKwkJbmV4dCA9IChjaGFyICopIGF0dF9hbGlnbl9u b21pbmFsKG5leHQsIHR5cGVudHJ5LnR5cGFsaWduKTsKKwkJcmVmc1tpXSA9IHJlZnNbal07 CisJCXJlZnNbal0uZGF0ID0gYWRhdDsKKwkJcmVmc1tqXS5zaXplID0gbmV4dCAtIGFkYXQ7 CisJCWFkYXQgPSBuZXh0OworCX0KKworCXJlc3VsdCA9IGNyZWF0ZV9hcnJheV9lbnZlbG9w ZShBUlJfTkRJTShhcnJheSksCisJCQkJCQkJCSAgIEFSUl9ESU1TKGFycmF5KSwKKwkJCQkJ CQkJICAgQVJSX0xCT1VORChhcnJheSksCisJCQkJCQkJCSAgIEFSUl9TSVpFKGFycmF5KSwK KwkJCQkJCQkJICAgQVJSX0VMRU1UWVBFKGFycmF5KSwKKwkJCQkJCQkJICAgYXJyYXktPmRh dGFvZmZzZXQpOworCisJcmRhdCA9IEFSUl9EQVRBX1BUUihyZXN1bHQpOworCisJLyogQ29s bGVjdCBhbGwgcmVmZXJlbmNlcyBhbmQgY29weSBlbGVtZW50cyB0byB0aGUgZGVzdGluYXRp b24gYXJyYXkgKi8KKwlmb3IgKGkgPSAwOyBpIDwgbml0ZW1zOyBpKyspCisJeworCQltZW1j cHkocmRhdCwgcmVmc1tpXS5kYXQsIHJlZnNbaV0uc2l6ZSk7CisJCXJkYXQgKz0gcmVmc1tp XS5zaXplOworCX0KKworCXBmcmVlKHJlZnMpOworCVBHX0ZSRUVfSUZfQ09QWShhcnJheSwg MCk7CisKKwlQR19SRVRVUk5fQVJSQVlUWVBFX1AocmVzdWx0KTsKK30KZGlmZiAtLWdpdCBh L3NyYy9pbmNsdWRlL2NhdGFsb2cvcGdfcHJvYy5kYXQgYi9zcmMvaW5jbHVkZS9jYXRhbG9n L3BnX3Byb2MuZGF0CmluZGV4IDJlNDFmNGQ5ZTguLjU2YWZmNTUxZDMgMTAwNjQ0Ci0tLSBh L3NyYy9pbmNsdWRlL2NhdGFsb2cvcGdfcHJvYy5kYXQKKysrIGIvc3JjL2luY2x1ZGUvY2F0 YWxvZy9wZ19wcm9jLmRhdApAQCAtMTY4MSw2ICsxNjgxLDkgQEAKICAgcHJvbmFtZSA9PiAn YXJyYXljb250am9pbnNlbCcsIHByb3ZvbGF0aWxlID0+ICdzJywgcHJvcmV0dHlwZSA9PiAn ZmxvYXQ4JywKICAgcHJvYXJndHlwZXMgPT4gJ2ludGVybmFsIG9pZCBpbnRlcm5hbCBpbnQy IGludGVybmFsJywKICAgcHJvc3JjID0+ICdhcnJheWNvbnRqb2luc2VsJyB9LAoreyBvaWQg PT4gJzc3NzcnLCBkZXNjciA9PiAnc2h1ZmZsZSBhcnJheScsCisgIHByb25hbWUgPT4gJ2Fy cmF5X3NodWZmbGUnLCBwcm9pc3N0cmljdCA9PiAnZicsIHByb3JldHR5cGUgPT4gJ2FueWFy cmF5JywKKyAgcHJvYXJndHlwZXMgPT4gJ2FueWFycmF5JywgcHJvc3JjID0+ICdhcnJheV9z aHVmZmxlJyB9LAogCiB7IG9pZCA9PiAnNzY0JywgZGVzY3IgPT4gJ2xhcmdlIG9iamVjdCBp bXBvcnQnLAogICBwcm9uYW1lID0+ICdsb19pbXBvcnQnLCBwcm92b2xhdGlsZSA9PiAndics IHByb3BhcmFsbGVsID0+ICd1JywKLS0gCjIuMzcuMQoK --------------3vFP0glmrXXmlPYrw6PnzXth--