Subject: Re: Proposal for enabling auto-vectorization for checksum calculations

From: Matthew Sterrett <[email-masked]>
Date: Wed, 7 May 2025 16:57:06 -0700

Hello! I'm still trying to figure out those CI failures, I just wanted
to update things.

---

From: Stepan Neretin <[email-masked]>
Date: Sat, 10 May 2025 18:01:06 +0700

On Thu, May 8, 2025 at 6:57 AM Matthew Sterrett <[email-masked]>
wrote:

> From my testing, with this patch repeatedly disabling/enabling
> checksums is about 12.4% on an approximately 15 GB database.
>
> By the way, I'd love it if anyone could help me figure out how to
> replicate a CI failure in the Cirrus CI.
> I haven't been able to figure out how to test CI runs locally, does
> anyone know a good method to do that?
>
>
>
Hi Matthew,

Thanks for the patch!

I ran some timing tests:

(without avx2)

Time: 4034.351 ms
SELECT drive_pg_checksum(512);

(with avx2)

Time: 3559.076 ms
SELECT drive_pg_checksum(512);

Also attached two patches that should fix the CI issues.

Best,

Stepan Neretin

---

From: Stepan Neretin <[email-masked]>
Date: Sat, 10 May 2025 18:01:54 +0700

On Sat, May 10, 2025 at 6:01 PM Stepan Neretin <[email-masked]> wrote:

>
>
> On Thu, May 8, 2025 at 6:57 AM Matthew Sterrett <
> [email-masked]> wrote:
>
Oops, forgot to attach patches :)

Best,

Stepan Neretin

---

From: Matthew Sterrett <[email-masked]>
Date: Mon, 19 May 2025 16:54:09 -0700

Hello! Thanks for helping me with this.
I'm still trying to figure out what is going on with the Bookworm test
failures. I'm pretty sure this patchset should resolve all the issues
with the macOS build, but I don't think it will help the linux
failures unfortunately.

On Sat, May 10, 2025 at 4:02 AM Stepan Neretin <[email-masked]> wrote:
>> On Thu, May 8, 2025 at 6:57 AM Matthew Sterrett <[email-masked]> wrote:
>>>

---

From: Nazir Bilal Yavuz <[email-masked]>
Date: Tue, 20 May 2025 17:42:27 +0300

Hi,

> with the macOS build, but I don't think it will help the linux

You can see the failure at the artifacts ->
'log/tmp_install/log/install.log' file on the CI web page [1].

If you want to replicate that on your local:

$ ./configure --with-llvm CLANG="ccache clang-16"
$ make -s -j8 world-bin
$ make -j8 check-world

should be enough. I was able to replicate it with these commands. I
hope these help.

[1] https://cirrus-ci.com/task/4834162550505472

--
Regards,
Nazir Bilal Yavuz
Microsoft

---

From: Matthew Sterrett <[email-masked]>
Date: Thu, 22 May 2025 14:54:28 -0700

> should be enough. I was able to replicate it with these commands. I
Thanks so much for helping me figure this out!

Okay, I've determined that versions of LLVM/Clang before 19 crash when
compiling this patch for some reason; it seems that both make
check-world and make install will crash with the affected LLVM
versions.
Unfortunately, what matters seems to be the version of the linker/LTO
optimizer, which I don't think we can check at compile time.
I added a check for Clang>=19 which works at preventing the crash on my system.
I think it's possible some unusual combination of clang/LLVM might
still crash during the build, but I think this is a reasonable
solution

---

From: John Naylor <[email-masked]>
Date: Mon, 2 Jun 2025 19:11:45 +0700

> still crash during the build, but I think this is a reasonable

I don't know if this is related to the crashes, but it doesn't seem
like a good idea to #include the function pointer stuff everywhere,
that should probably go into src/port like the others.

--
John Naylor
Amazon Web Services