Feedback-ID: ie0a040ee:Fastmail
Content-Type: multipart/mixed; boundary="------------dGHl6l6nybtnp7mOt09YOfAA"
Message-ID: <d42db427-0b1e-41c8-b3fd-db7f85aec70a@eisentraut.org>
Date: Mon, 29 Jun 2026 10:52:39 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: vectorized CRC on ARM64
To: John Naylor <johncnaylorls@gmail.com>,
 Nathan Bossart <nathandbossart@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: 
 <CANWCAZaKhE+RD5KKouUFoxx1EbUNrNhcduM1VQ=DkSDadNEFng@mail.gmail.com>
 <CANWCAZYAKWTFW_1byMq44t57urk6YKoDh1z1Zp6fk5-4dOPy-w@mail.gmail.com>
 <CANWCAZZj35DGtjSX3umT1XHM49T4-v5ysc20TXPLcfn1oHGHfA@mail.gmail.com>
 <acwQpqLKzH0Ft-I3@nathan>
 <CANWCAZYhe8jtH4QTy-VUqZcTHw6OjfBzO6vYLm-JU9ZcYwXq_g@mail.gmail.com>
 <ac04uCXJOPEAwE1Z@nathan>
 <CANWCAZaRuJC6vb+VKqimeEPLV-d_fUk8AO1-jL1Pwx4STM4J+A@mail.gmail.com>
 <ac6Q9JduymhuxcNt@nathan> <ac6WfKIKshE3AmWr@nathan>
 <CANWCAZa-4WN8_bPV=GKZii2b5kdwffdWreuykP6ogT_dRYhPdQ@mail.gmail.com>
Content-Language: en-US
From: Peter Eisentraut <peter@eisentraut.org>
In-Reply-To: 
 <CANWCAZa-4WN8_bPV=GKZii2b5kdwffdWreuykP6ogT_dRYhPdQ@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/d42db427-0b1e-41c8-b3fd-db7f85aec70a%40eisentraut.org>
Precedence: bulk

This is a multi-part message in MIME format.
--------------dGHl6l6nybtnp7mOt09YOfAA
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

It looks like this patch was developed concurrently with the switch from 
pg_attribute_aligned to C11 standard alignas, and it ended up being 
committed with the "old" style.  I propose the attached patch to update 
that.  It's only a stylistic change, but good for consistency and as an 
example for future code.


On 03.04.26 10:22, John Naylor wrote:
> On Thu, Apr 2, 2026 at 11:17 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
>>
>> On Thu, Apr 02, 2026 at 10:53:24AM -0500, Nathan Bossart wrote:
>>> I think the new pg_comp_crc32_choose() is infinitely recursing on macOS
>>> because USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK is not defined but
>>> pg_crc32c_armv8_available() returns false.  If I trace through that
>>> function, I see that it's going straight to the
>>>
>>>        #else
>>>                return false;
>>>        #endif
>>>
>>> at the end.  And sure enough, both HAVE_ELF_AUX_INFO and HAVE_GETAUXVAL
> 
> Ah of course.
> 
>>> aren't defined in pg_config.h.  I think we might need to use sysctlbyname()
>>> to determine PMULL support on macOS, but at this stage of the development
>>> cycle, I would probably lean towards just compiling in the sb8
>>> implementation.
>>
>> Hm.  On second thought, that probably regresses macOS builds because it was
>> presumably using the armv8 path without runtime checks before...
> 
> I went with the following for v5, and it passes MacOS on my Github CI:
> 
> +  /* set fallbacks */
> +#ifdef USE_ARMV8_CRC32C
> +  /* On e.g. MacOS, our runtime feature detection doesn't work */
> +  pg_comp_crc32c = pg_comp_crc32c_armv8;
> +#else
> +  pg_comp_crc32c = pg_comp_crc32c_sb8;
> +#endif
> + [...crc and pmull checks]
> 
> That should keep scalar hardware support working, but now it'll only
> use direct calls for constant inputs.
> 
> I also did some benchmarking on an ARM Neoverse N1 / gcc 8.3
> (attached). There the vector loop still works well all the way down to
> the minimum input size of 64 bytes, and on long inputs it's almost
> twice as fast as scalar. For reproduceability, I slightly modified the
> benchmark we used last year, to make sure the input is aligned
> (attached but not for CI). In the end, I want to add a length check so
> that inputs smaller than 80 bytes go straight to the scalar path.
> Above 80, after alignment adjustments in the preamble, that still
> guarantees at least one loop iteration in the vector path.
> 
> --
> John Naylor
> Amazon Web Services

--------------dGHl6l6nybtnp7mOt09YOfAA
Content-Type: text/plain; charset=UTF-8;
 name="0001-Use-C11-alignas-instead-of-pg_attribute_aligned.patch"
Content-Disposition: attachment;
 filename*0="0001-Use-C11-alignas-instead-of-pg_attribute_aligned.patch"
Content-Transfer-Encoding: base64

RnJvbSBkZTJjNjQ5ODU2NjJlMGE3ODhkYTVhNGVkOWY0ZTczNThlZDU0NGY0IE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBQZXRlciBFaXNlbnRyYXV0IDxwZXRlckBlaXNlbnRy
YXV0Lm9yZz4KRGF0ZTogTW9uLCAyOSBKdW4gMjAyNiAxMDo0Njo1MSArMDIwMApTdWJqZWN0
OiBbUEFUQ0hdIFVzZSBDMTEgYWxpZ25hcyBpbnN0ZWFkIG9mIHBnX2F0dHJpYnV0ZV9hbGln
bmVkCgpSZXBsYWNlIHBnX2F0dHJpYnV0ZV9hbGlnbmVkIHdpdGggQzExIGFsaWduYXMsIGZv
ciBjb25zaXN0ZW5jeSB3aXRoCmN1cnJlbnQgY29udmVudGlvbnMuCgpEaXNjdXNzaW9uOiBo
dHRwczovL3Bvc3Rnci5lcy9tL0NBTldDQVphS2hFK1JENUtLb3VVRm94eDFFYlVOck5oY2R1
TTFWUT1Ea1NEYWRORUZuZ0BtYWlsLmdtYWlsLmNvbQotLS0KIHNyYy9wb3J0L3BnX2NyYzMy
Y19hcm12OC5jIHwgNiArKystLS0KIDEgZmlsZSBjaGFuZ2VkLCAzIGluc2VydGlvbnMoKyks
IDMgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL3BvcnQvcGdfY3JjMzJjX2FybXY4
LmMgYi9zcmMvcG9ydC9wZ19jcmMzMmNfYXJtdjguYwppbmRleCA4Y2Y4MzhhNTEwYS4uOGZh
ZDJmODNjMjQgMTAwNjQ0Ci0tLSBhL3NyYy9wb3J0L3BnX2NyYzMyY19hcm12OC5jCisrKyBi
L3NyYy9wb3J0L3BnX2NyYzMyY19hcm12OC5jCkBAIC0xNjYsNyArMTY2LDcgQEAgcGdfY29t
cF9jcmMzMmNfcG11bGwocGdfY3JjMzJjIGNyYywgY29uc3Qgdm9pZCAqZGF0YSwgc2l6ZV90
IGxlbikKIAkJdWludDY0eDJfdAlrOwogCiAJCXsKLQkJCXN0YXRpYyBjb25zdCB1aW50NjRf
dCBwZ19hdHRyaWJ1dGVfYWxpZ25lZCgxNikga19bXSA9IHsweDc0MGVlZjAyLCAweDllNGFk
ZGY4fTsKKwkJCXN0YXRpYyBjb25zdCBhbGlnbmFzKDE2KSB1aW50NjRfdCBrX1tdID0gezB4
NzQwZWVmMDIsIDB4OWU0YWRkZjh9OwogCiAJCQlrID0gdmxkMXFfdTY0KGtfKTsKIAkJfQpA
QCAtMTkyLDE0ICsxOTIsMTQgQEAgcGdfY29tcF9jcmMzMmNfcG11bGwocGdfY3JjMzJjIGNy
YywgY29uc3Qgdm9pZCAqZGF0YSwgc2l6ZV90IGxlbikKIAogCQkvKiBSZWR1Y2UgeDAgLi4u
IHgzIHRvIGp1c3QgeDAuICovCiAJCXsKLQkJCXN0YXRpYyBjb25zdCB1aW50NjRfdCBwZ19h
dHRyaWJ1dGVfYWxpZ25lZCgxNikga19bXSA9IHsweGYyMGMwZGZlLCAweDQ5M2M3ZDI3fTsK
KwkJCXN0YXRpYyBjb25zdCBhbGlnbmFzKDE2KSB1aW50NjRfdCBrX1tdID0gezB4ZjIwYzBk
ZmUsIDB4NDkzYzdkMjd9OwogCiAJCQlrID0gdmxkMXFfdTY0KGtfKTsKIAkJfQogCQl5MCA9
IGNsbXVsX2xvX2UoeDAsIGssIHgxKSwgeDAgPSBjbG11bF9oaV9lKHgwLCBrLCB5MCk7CiAJ
CXkyID0gY2xtdWxfbG9fZSh4MiwgaywgeDMpLCB4MiA9IGNsbXVsX2hpX2UoeDIsIGssIHky
KTsKIAkJewotCQkJc3RhdGljIGNvbnN0IHVpbnQ2NF90IHBnX2F0dHJpYnV0ZV9hbGlnbmVk
KDE2KSBrX1tdID0gezB4M2RhNmQwY2IsIDB4YmE0ZmMyOGV9OworCQkJc3RhdGljIGNvbnN0
IGFsaWduYXMoMTYpIHVpbnQ2NF90IGtfW10gPSB7MHgzZGE2ZDBjYiwgMHhiYTRmYzI4ZX07
CiAKIAkJCWsgPSB2bGQxcV91NjQoa18pOwogCQl9Ci0tIAoyLjU0LjAKCg==

--------------dGHl6l6nybtnp7mOt09YOfAA--