public inbox for [email protected]help / color / mirror / Atom feed
Broken build on macOS (Universal / Intel): cpuid instruction not available 24+ messages / 8 participants [nested] [flat]
* Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 11:41 Jakob Egger <[email protected]> 0 siblings, 2 replies; 24+ messages in thread From: Jakob Egger @ 2026-05-07 11:41 UTC (permalink / raw) To: pgsql-hackers; +Cc: [email protected]; Tobias Bussmann <[email protected]> Hello! We have been investigating recent build failures of the master branch. Currently both Intel and Universal builds are broken on macOS. We tested building on macOS 26.4.1 with Xcode 26.2 and on macOS 14.5. Universal builds ============ Universal builds are broken since this commit: 16743db: Centralize detection of x86 CPU features To reproduce: export CFLAGS="-arch arm64 -arch x86_64" ./configure --without-icu make This results in an error "cpuid instruction not available" Universal builds used to work; they are broken since commit 16743db. Intel Builds ======== Intel-only builds (using Rosetta) are also broken in master since the following commit: 5e13b0f: Use AVX2 for calculating page checksums where available To reproduce: arch -x86_64 zsh ./configure --without-icu make This results in an error: checksum.c:57:6: error: call to undeclared function 'x86_feature_available' Best regards, Jakob Egger and Tobias Bussmann ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 12:42 John Naylor <[email protected]> parent: Jakob Egger <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: John Naylor @ 2026-05-07 12:42 UTC (permalink / raw) To: Jakob Egger <[email protected]>; +Cc: pgsql-hackers; [email protected]; Tobias Bussmann <[email protected]> On Thu, May 7, 2026 at 6:41 PM Jakob Egger <[email protected]> wrote: > Universal builds > ============ > This results in an error "cpuid instruction not available" Hmm, I imagine that may not work on normal Intel builds either, but maybe it didn't get that far. > Intel Builds > ======== > > Intel-only builds (using Rosetta) are also broken in master since the following commit: > 5e13b0f: Use AVX2 for calculating page checksums where available > This results in an error: > checksum.c:57:6: error: call to undeclared function 'x86_feature_available' That's strange. I have an Intel MacBook laying around -- I'll see what I can do. -- John Naylor Amazon Web Services ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 12:50 Daniel Gustafsson <[email protected]> parent: John Naylor <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Daniel Gustafsson @ 2026-05-07 12:50 UTC (permalink / raw) To: John Naylor <[email protected]>; +Cc: Jakob Egger <[email protected]>; pgsql-hackers; [email protected]; Tobias Bussmann <[email protected]> > On 7 May 2026, at 14:42, John Naylor <[email protected]> wrote: >> This results in an error: >> checksum.c:57:6: error: call to undeclared function 'x86_feature_available' > > That's strange. I have an Intel MacBook laying around -- I'll see what I can do. I use an Intel MacBook Pro as my daily driver (currently running macOS 14.7) and have not had any issues. -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 13:59 Tom Lane <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 2 replies; 24+ messages in thread From: Tom Lane @ 2026-05-07 13:59 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; +Cc: John Naylor <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; [email protected]; Tobias Bussmann <[email protected]> Daniel Gustafsson <[email protected]> writes: >> On 7 May 2026, at 14:42, John Naylor <[email protected]> wrote: >>> This results in an error: >>> checksum.c:57:6: error: call to undeclared function 'x86_feature_available' >> That's strange. I have an Intel MacBook laying around -- I'll see what I can do. > I use an Intel MacBook Pro as my daily driver (currently running macOS 14.7) > and have not had any issues. BF member longfin is an Intel Mac Mini and has not shown issues (although I think it's one macOS rev behind). I can believe that universal builds are busted, because we don't test that. But claiming that a plain Intel build is broken is contrary to available evidence. regards, tom lane ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 14:38 Jakob Egger <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Jakob Egger @ 2026-05-07 14:38 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; John Naylor <[email protected]>; pgsql-hackers; [email protected]; Tobias Bussmann <[email protected]> > Am 07.05.2026 um 15:59 schrieb Tom Lane <[email protected]>: > > Daniel Gustafsson <[email protected]> writes: >>> On 7 May 2026, at 14:42, John Naylor <[email protected]> wrote: >>>> This results in an error: >>>> checksum.c:57:6: error: call to undeclared function 'x86_feature_available' > >>> That's strange. I have an Intel MacBook laying around -- I'll see what I can do. > >> I use an Intel MacBook Pro as my daily driver (currently running macOS 14.7) >> and have not had any issues. > > BF member longfin is an Intel Mac Mini and has not shown issues > (although I think it's one macOS rev behind). I can believe that > universal builds are busted, because we don't test that. But > claiming that a plain Intel build is broken is contrary to > available evidence. I've set up a fresh VM with macOS 26.4.1 and command line tools. Universal builds are still broken, but Intel builds (with Rosetta) work in the new VM. So the intel issue might be something unique to my machine. I''ll try to figure it out. Jakob ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 14:57 Tobias Bussmann <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Tobias Bussmann @ 2026-05-07 14:57 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; John Naylor <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; [email protected] > Am 07.05.2026 um 15:59 schrieb Tom Lane <[email protected]>: > > I can believe that > universal builds are busted, because we don't test that. I've already noticed that there isn't much variety in the buildfarm animals available for Darwin. Given the availability of a fairly beefy macOS VM server here, I've considered hosting some VMs to provide coverage of things like universal builds and cross-compilation using different versions of the Apple toolkits for the build farm. This issue provides the motivation to take it further. regards Tobias ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 15:19 Tom Lane <[email protected]> parent: Tobias Bussmann <[email protected]> 0 siblings, 0 replies; 24+ messages in thread From: Tom Lane @ 2026-05-07 15:19 UTC (permalink / raw) To: Tobias Bussmann <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; John Naylor <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; [email protected] Tobias Bussmann <[email protected]> writes: > I've already noticed that there isn't much variety in the buildfarm animals > available for Darwin. Given the availability of a fairly beefy macOS VM server > here, I've considered hosting some VMs to provide coverage of things like > universal builds and cross-compilation using different versions of the Apple > toolkits for the build farm. This issue provides the motivation to take it further. +1. If you're relying on such edge cases working, hosting a buildfarm animal is by far the best way to ensure they keep working. regards, tom lane ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 15:41 Jakob Egger <[email protected]> parent: Jakob Egger <[email protected]> 0 siblings, 0 replies; 24+ messages in thread From: Jakob Egger @ 2026-05-07 15:41 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; John Naylor <[email protected]>; pgsql-hackers; [email protected]; Tobias Bussmann <[email protected]> > Am 07.05.2026 um 16:38 schrieb Jakob Egger <[email protected]>: >> Am 07.05.2026 um 15:59 schrieb Tom Lane <[email protected]>: >> >> Daniel Gustafsson <[email protected]> writes: >>>> On 7 May 2026, at 14:42, John Naylor <[email protected]> wrote: >>>>> This results in an error: >>>>> checksum.c:57:6: error: call to undeclared function 'x86_feature_available' >> >>>> That's strange. I have an Intel MacBook laying around -- I'll see what I can do. >> >>> I use an Intel MacBook Pro as my daily driver (currently running macOS 14.7) >>> and have not had any issues. >> >> BF member longfin is an Intel Mac Mini and has not shown issues >> (although I think it's one macOS rev behind). I can believe that >> universal builds are busted, because we don't test that. But >> claiming that a plain Intel build is broken is contrary to >> available evidence. > > I've set up a fresh VM with macOS 26.4.1 and command line tools. > Universal builds are still broken, but Intel builds (with Rosetta) work in the new VM. > So the intel issue might be something unique to my machine. I''ll try to figure it out. > > Jakob The Intel build error only occurs when building in Rosetta with Xcode 26.2. I've switched to building with Command Line Tools 26.4 and it builds fine. The Universal build issue on the other hand is reproducible across multiple Xcode and macOS versions. Jakob ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 15:48 Tom Lane <[email protected]> parent: Jakob Egger <[email protected]> 1 sibling, 2 replies; 24+ messages in thread From: Tom Lane @ 2026-05-07 15:48 UTC (permalink / raw) To: Jakob Egger <[email protected]>; +Cc: pgsql-hackers; John Naylor <[email protected]>; Tobias Bussmann <[email protected]> Jakob Egger <[email protected]> writes: > Universal builds are broken since this commit: > 16743db: Centralize detection of x86 CPU features > To reproduce: > export CFLAGS="-arch arm64 -arch x86_64" > ./configure --without-icu > make > This results in an error "cpuid instruction not available" I can reproduce this here. But after looking at it briefly, I think it's purely accidental that universal builds ever worked, and they could have done so only for small values of "work". The fundamental problem is that we make only one probe at configure time to set flags such as HAVE__GET_CPUID. With a single arch selected in CFLAGS, you get the appropriate answer for that arch, and everything's fine. With both selected, one build or the other will fail such probes with errors like In file included from conftest.c:135: /Library/Developer/CommandLineTools/usr/lib/clang/21/include/cpuid.h:14:2: erro\ r: this header is for x86 only 14 | #error this header is for x86 only | ^ so that we end up with essentially no CPU-specific optimizations enabled. That's not a place that you really want to be. (I've not checked into how s_lock.h manages to work at all under these conditions, but maybe it ends up picking a compiler-intrinsic implementation?) The proximate reason that it broke is that v18 and before had code like #ifdef HAVE_X86_64_POPCNTQ #if defined(HAVE__GET_CPUID) || defined(HAVE__CPUID) #define TRY_POPCNT_X86_64 1 #endif #endif and didn't try to compile the cpuid-dependent function unless TRY_POPCNT_X86_64 was set. The code in HEAD doesn't have that guard, and is essentially assuming that every x86 platform wil provide HAVE__GET_CPUID or HAVE__CPUID. To make this actually work well, we'd have to do two sets of configure probes, one for each arch, and somehow apply the correct set of #defines depending on arch. That's a lot of work that I personally have no interest in doing, seeing that the handwriting is on the wall for Apple's support of x86. I wonder whether we shouldn't just disclaim support for multi-arch builds. If it's easy to un-break, then sure we might as well restore the status quo ante, but I don't think it's worth putting a ton of work into. regards, tom lane ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 15:57 Nathan Bossart <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 0 replies; 24+ messages in thread From: Nathan Bossart @ 2026-05-07 15:57 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Jakob Egger <[email protected]>; pgsql-hackers; John Naylor <[email protected]>; Tobias Bussmann <[email protected]> On Thu, May 07, 2026 at 11:48:04AM -0400, Tom Lane wrote: > To make this actually work well, we'd have to do two sets of configure > probes, one for each arch, and somehow apply the correct set of > #defines depending on arch. That's a lot of work that I personally > have no interest in doing, seeing that the handwriting is on the wall > for Apple's support of x86. > > I wonder whether we shouldn't just disclaim support for multi-arch > builds. If it's easy to un-break, then sure we might as well restore > the status quo ante, but I don't think it's worth putting a ton of > work into. +1. I'm not mortally opposed to the idea if someone wants to do the work, but at the moment I can't get excited about it. Perhaps multi-arch builds will become more common down the road, though. -- nathan ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 16:22 Tom Lane <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Tom Lane @ 2026-05-07 16:22 UTC (permalink / raw) To: John Naylor <[email protected]>; +Cc: Jakob Egger <[email protected]>; pgsql-hackers; Tobias Bussmann <[email protected]> I wrote: > ... The code in HEAD doesn't have > that guard, and is essentially assuming that every x86 platform > wil provide HAVE__GET_CPUID or HAVE__CPUID. Independently of whether macOS multi-arch is something we consider supportable, I think the aforesaid assumption is a bad idea. Can't we make pg_cpuid() return zeroes if it doesn't know how to get the info, analogously to what pg_cpuid_subleaf() does? regards, tom lane ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-07 17:33 Lukas Fittl <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Lukas Fittl @ 2026-05-07 17:33 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: John Naylor <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Tobias Bussmann <[email protected]>; Andres Freund <[email protected]> On Thu, May 7, 2026 at 9:22 AM Tom Lane <[email protected]> wrote: > > I wrote: > > ... The code in HEAD doesn't have > > that guard, and is essentially assuming that every x86 platform > > wil provide HAVE__GET_CPUID or HAVE__CPUID. > > Independently of whether macOS multi-arch is something we consider > supportable, I think the aforesaid assumption is a bad idea. > Can't we make pg_cpuid() return zeroes if it doesn't know how to > get the info, analogously to what pg_cpuid_subleaf() does? Having worked in that area in this cycle, I think returning zeroes (or adding a return value that confirms we got data) could work for the TSC related functionality, since we just fall back to the system clock and disallow TSC use if we can't get CPUID data. I'll let John confirm if there are any other optimizations that require CPUID data. CCing Andres as well, since he reviewed some of those patches for pg_cpu_x86.c. Thanks, Lukas -- Lukas Fittl ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-08 03:48 John Naylor <[email protected]> parent: Lukas Fittl <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: John Naylor @ 2026-05-08 03:48 UTC (permalink / raw) To: Lukas Fittl <[email protected]>; +Cc: Tom Lane <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Tobias Bussmann <[email protected]>; Andres Freund <[email protected]> On Fri, May 8, 2026 at 12:34 AM Lukas Fittl <[email protected]> wrote: > > On Thu, May 7, 2026 at 9:22 AM Tom Lane <[email protected]> wrote: > > > > I wrote: > > > ... The code in HEAD doesn't have > > > that guard, and is essentially assuming that every x86 platform > > > wil provide HAVE__GET_CPUID or HAVE__CPUID. > > > > Independently of whether macOS multi-arch is something we consider > > supportable, I think the aforesaid assumption is a bad idea. > > Can't we make pg_cpuid() return zeroes if it doesn't know how to > > get the info, analogously to what pg_cpuid_subleaf() does? Yeah, that seems like a good assumption to remove. Jakob and Tobias, how far do you get with the attached, at least for the target x86 case? And it might help if you sent the configure output in a file so we can see what we're dealing with. Note also that we have two build systems, since we'll likely transition away from autotools: https://wiki.postgresql.org/wiki/Meson > Having worked in that area in this cycle, I think returning zeroes (or > adding a return value that confirms we got data) Speaking of return values, the TSC feature as committed doesn't actually use the return value of pg_cpuid_subleaf(), but it seems like something we could use someday. I don't see such a use for pg_cpuid() beyond having zero values in the 4 variables. > could work for the > TSC related functionality, since we just fall back to the system clock > and disallow TSC use if we can't get CPUID data. I'll let John confirm > if there are any other optimizations that require CPUID data. Yes there are, but everything should still fallback gracefully without it. -- John Naylor Amazon Web Services Attachments: [text/x-patch] dont-assume-cpuid-available.patch (534B, 2-dont-assume-cpuid-available.patch) download | inline diff: diff --git a/src/port/pg_cpu_x86.c b/src/port/pg_cpu_x86.c index f22167da9af..f61f0940431 100644 --- a/src/port/pg_cpu_x86.c +++ b/src/port/pg_cpu_x86.c @@ -63,12 +63,11 @@ mask_available(uint32 value, uint32 mask) static inline void pg_cpuid(int leaf, unsigned int *reg) { + memset(reg, 0, 4 * sizeof(unsigned int)); #if defined(HAVE__GET_CPUID) __get_cpuid(leaf, ®[EAX], ®[EBX], ®[ECX], ®[EDX]); #elif defined(HAVE__CPUID) __cpuid((int *) reg, leaf); -#else -#error cpuid instruction not available #endif } ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-08 08:26 Tobias Bussmann <[email protected]> parent: John Naylor <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Tobias Bussmann @ 2026-05-08 08:26 UTC (permalink / raw) To: John Naylor <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Tom Lane <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> > Am 08.05.2026 um 05:48 schrieb John Naylor <[email protected]>: > > Jakob and Tobias, how far do you get with the attached, at least for > the target x86 case? thanks! I tried the patch and it fixes the universal build that broke with 16743db (and make check passes for both architectures). It remains to be analysed how useful these universal builds are given the lack of optimisations for one of the architectures, but at least they are possible again, as they were previously. > And it might help if you sent the configure > output in a file so we can see what we're dealing with. Please find the configure output attached. > Note also that > we have two build systems, since we'll likely transition away from > autotools: sure, I haven't yet evaluated the meson build system but I plan to do so. Best regards Tobias Attachments: [application/octet-stream] configure.out (18.9K, 2-configure.out) download ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-08 10:04 John Naylor <[email protected]> parent: Tobias Bussmann <[email protected]> 0 siblings, 2 replies; 24+ messages in thread From: John Naylor @ 2026-05-08 10:04 UTC (permalink / raw) To: Tobias Bussmann <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Tom Lane <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> On Fri, May 8, 2026 at 3:26 PM Tobias Bussmann <[email protected]> wrote: > > Am 08.05.2026 um 05:48 schrieb John Naylor <[email protected]>: > > > > Jakob and Tobias, how far do you get with the attached, at least for > > the target x86 case? > > thanks! I tried the patch and it fixes the universal build that broke with > 16743db (and make check passes for both architectures). It remains to be Great! I've pushed that fix. > analysed how useful these universal builds are given the lack of > optimisations for one of the architectures, but at least they are possible > again, as they were previously. Taking a quick look at the configure output you provided, certain optimizations will be lacking on both architectures: checking for _mm_crc32_u8 and _mm_crc32_u32... no checking for __crc32cb, __crc32ch, __crc32cw, and __crc32cd with CFLAGS=... no checking for __crc32cb, __crc32ch, __crc32cw, and __crc32cd with CFLAGS=-march=armv8-a+crc+simd... no checking for __crc32cb, __crc32ch, __crc32cw, and __crc32cd with CFLAGS=-march=armv8-a+crc... no ... checking which CRC-32C implementation to use... slicing-by-8 But compiler builtins seem to work: checking for builtin __atomic int32 atomic operations... yes checking for builtin __atomic int64 atomic operations... yes -- John Naylor Amazon Web Services ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-05-08 13:54 Tom Lane <[email protected]> parent: John Naylor <[email protected]> 1 sibling, 0 replies; 24+ messages in thread From: Tom Lane @ 2026-05-08 13:54 UTC (permalink / raw) To: John Naylor <[email protected]>; +Cc: Tobias Bussmann <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]> John Naylor <[email protected]> writes: > On Fri, May 8, 2026 at 3:26 PM Tobias Bussmann <[email protected]> wrote: >> analysed how useful these universal builds are given the lack of >> optimisations for one of the architectures, but at least they are possible >> again, as they were previously. > Taking a quick look at the configure output you provided, certain > optimizations will be lacking on both architectures: Yeah. I think the main practical problem will be the lowest-common-denominator CRC code. However, AFAICS a universal build would have been selecting that from day one, so if the performance has been acceptable so far it won't be any worse in v19. regards, tom lane ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-02 12:49 Tobias Bussmann <[email protected]> parent: John Naylor <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Tobias Bussmann @ 2026-06-02 12:49 UTC (permalink / raw) To: John Naylor <[email protected]>; +Cc: Lukas Fittl <[email protected]>; Tom Lane <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> > Am 08.05.2026 um 12:04 schrieb John Naylor <[email protected]>: > > Taking a quick look at the configure output you provided, certain > optimizations will be lacking on both architectures: Indeed. The universal builds seem to disable optimisations that are supported natively. I did a comparison on the attached configure outputs with different CFLAGS on arm64 - the first three ran the compiler in x86_64 emulation. I'd expect this to be similar to a native x86_64 environment, but need to verify. 1. configure.x86_64.out: -arch x86_64 using rosetta2 2. configure.universal-cross.out: -arch arm64 -arch x86_64 using rosetta2 3. configure.arm64-cross.out: -arch arm64 using rosetta2 4. configure.x86_64-cross.out: -arch x86_64 5. configure.universal.out: -arch arm64 -arch x86_64 6. configure.arm64.out: -arch arm64 The difference on active features in detail are: Only with arm64 native (6): * svcnt_x * pmull and pmull2 Both with arm64 native (6) and cross (3): * __crc32cb, __crc32ch, __crc32cw, and __crc32cd with CFLAGS Only with x86_64 emulation (1): * assembler supports x86_64 popcntq * _mm512_popcnt_epi64 * _mm512_clmulepi64_epi128 Both with x86_64 emulation/native (1) and arm64 cross (3) and universal cross (2) * AVX2 target attribute support Both with x86_64 cross (4) and emulation (1): * __get_cpuid * __get_cpuid_count * _xgetbv * _mm_crc32_u8 and _mm_crc32_u32 This results in the following CRC-32 decisions: CRC-32C implementation: 1. x86_64 emulation: SSE 4.2 with runtime check 2. universal emulation: slicing-by-8 3. arm64 cross emulation: ARMv8 CRC instructions 4. x86_64 cross: SSE 4.2 with runtime check 5. universal: slicing-by-8 6. arm64: ARMv8 CRC instructions Vectorized CRC-32C: 1. x86_64 emulation: AVX-512 with runtime check 2. universal emulation: none 3. arm64 cross emulation: none 4. x86_64 cross: none 5. universal: none 6. arm64: CRYPTO PMULL with runtime check The differences between cross compilation and emulation(native) may be worth looking into. Cross-compiling arm64 or universal from x86_64 does not work, however: checksum.c:57:6: error: call to undeclared function 'x86_feature_available' if (x86_feature_available(PG_AVX2)) ^ checksum.c:57:28: error: use of undeclared identifier 'PG_AVX2' if (x86_feature_available(PG_AVX2)) This may have to do with the detection of "AVX2 target attribute support" As Sandeep Thakkar just confirmed on the packagers list, this also affects bulding on a native x86_64 host not only an emulated one. The AVX2 target attribute support when ran on x86_64 cross compiling to arm64 (3) (or universal (2)) seems to be wrong. Best regards Tobias Attachments: [application/octet-stream] configure.x86_64.out (19.0K, 2-configure.x86_64.out) download [application/octet-stream] configure.universal-cross.out (19.0K, 3-configure.universal-cross.out) download [application/octet-stream] configure.arm64-cross.out (18.8K, 4-configure.arm64-cross.out) download [application/octet-stream] configure.x86_64-cross.out (18.9K, 5-configure.x86_64-cross.out) download [application/octet-stream] configure.universal.out (18.9K, 6-configure.universal.out) download [application/octet-stream] configure.arm64.out (18.7K, 7-configure.arm64.out) download ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-02 14:48 Tom Lane <[email protected]> parent: Tobias Bussmann <[email protected]> 0 siblings, 2 replies; 24+ messages in thread From: Tom Lane @ 2026-06-02 14:48 UTC (permalink / raw) To: Tobias Bussmann <[email protected]>; +Cc: John Naylor <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> Tobias Bussmann <[email protected]> writes: > Indeed. The universal builds seem to disable optimisations that are supported > natively. Yes, we realized that upthread. It's been like that since day 1, not something that's new in v19. I don't see that we're likely to do anything about it: it'd require a fairly fundamental re-architecting of our configure tests, and the end result would only be to improve support for machines that are going to be dead to macOS within a year. However, it definitely is a regression that the build fails altogether. Too bad nobody tried the x86 -> ARM case earlier. regards, tom lane ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-02 15:10 Tobias Bussmann <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Tobias Bussmann @ 2026-06-02 15:10 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: John Naylor <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> > Am 02.06.2026 um 16:48 schrieb Tom Lane <[email protected]>: > > Tobias Bussmann <[email protected]> writes: >> Indeed. The universal builds seem to disable optimisations that are supported >> natively. > > Yes, we realized that upthread. It's been like that since day 1, > not something that's new in v19. I wasn't intending to say that. I was rather wondering if it would be worth to test doing two independent builds and lipo all the binaries and libraries together manually to construct a proper optimised universal build. > However, it definitely is a regression that the build fails > altogether. Too bad nobody tried the x86 -> ARM case earlier. We already touched the issue briefly before but didn't nail it. I wish I'd have more resources to contribute. Tobias ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-02 15:20 Tom Lane <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 2 replies; 24+ messages in thread From: Tom Lane @ 2026-06-02 15:20 UTC (permalink / raw) To: Tobias Bussmann <[email protected]>; +Cc: John Naylor <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> I wrote: > However, it definitely is a regression that the build fails > altogether. Too bad nobody tried the x86 -> ARM case earlier. I replicated this on longfin's host (x86_64 mac mini). It seems there are two problems: 1. pg_cpu.h believes that x86-specific code can be conditional on #if defined(USE_SSE2) || defined(__i386__) but macOS doesn't define __i386__, only __x86_64__. It works anyway on single-arch builds because the test to set USE_SSE2 succeeds, but not on multi-arch builds. 2. checksum.c believes that it's okay to call x86_feature_available if USE_AVX2_WITH_RUNTIME_CHECK is set. I didn't track down just why that's getting set in a multi-arch build when USE_SSE2 is not, but it is, and that's probably good since it means we get at least some optimization for x86 Macs. But we have to disregard it when we're doing the ARM side. The attached quick hack makes the build work on my machine. I'm hesitant to shove it into the tree though because I'm not too certain whether there could be side-effects on other platforms. I think the way to proceed for now is for EDB to apply this patch in their build of beta1, and we can review the patch at leisure afterwards. regards, tom lane Attachments: [text/x-diff] quick-fix.patch (945B, 2-quick-fix.patch) download | inline diff: diff --git a/src/backend/storage/page/checksum.c b/src/backend/storage/page/checksum.c index 030c44f7308..db8544bca01 100644 --- a/src/backend/storage/page/checksum.c +++ b/src/backend/storage/page/checksum.c @@ -26,6 +26,12 @@ #define PG_CHECKSUM_INTERNAL #include "storage/checksum_impl.h" /* IWYU pragma: keep */ +/* In universal macOS builds, don't try to compile AVX code on the ARM side */ +#if defined(__i386__) || defined(__x86_64__) +#else +#undef USE_AVX2_WITH_RUNTIME_CHECK +#endif + static uint32 pg_checksum_block_fallback(const PGChecksummablePage *page) diff --git a/src/include/port/pg_cpu.h b/src/include/port/pg_cpu.h index 566ed7a16e3..b276d7f1a4e 100644 --- a/src/include/port/pg_cpu.h +++ b/src/include/port/pg_cpu.h @@ -13,7 +13,7 @@ #ifndef PG_CPU_H #define PG_CPU_H -#if defined(USE_SSE2) || defined(__i386__) +#if defined(USE_SSE2) || defined(__i386__) || defined(__x86_64__) typedef enum X86FeatureId { ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-02 16:24 Tom Lane <[email protected]> parent: Tobias Bussmann <[email protected]> 0 siblings, 0 replies; 24+ messages in thread From: Tom Lane @ 2026-06-02 16:24 UTC (permalink / raw) To: Tobias Bussmann <[email protected]>; +Cc: John Naylor <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> Tobias Bussmann <[email protected]> writes: > I wasn't intending to say that. I was rather wondering if it would be worth to test doing two independent builds and lipo all the binaries and libraries together manually to construct a proper optimised universal build. Probably that could be done, but again it seems like an awful lot of work to expend on a build situation that won't be interesting for very much longer. (I do realize that Apple might someday resurrect all this for YA hardware changeover, but surely that's not going to be anytime soon. So I think we have more-pressing things to do.) regards, tom lane ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-02 17:38 Tom Lane <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Tom Lane @ 2026-06-02 17:38 UTC (permalink / raw) To: Tobias Bussmann <[email protected]>; +Cc: John Naylor <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> I wrote: > It seems there are two problems: > 1. pg_cpu.h believes that x86-specific code can be conditional on > #if defined(USE_SSE2) || defined(__i386__) > but macOS doesn't define __i386__, only __x86_64__. It works > anyway on single-arch builds because the test to set USE_SSE2 > succeeds, but not on multi-arch builds. > 2. checksum.c believes that it's okay to call x86_feature_available > if USE_AVX2_WITH_RUNTIME_CHECK is set. After further study, I found that USE_SSE2 is set like this in c.h: #if (defined(__x86_64__) || defined(_M_AMD64)) #define USE_SSE2 ... So my interpretation of what was happening in pg_cpu.h was wrong. I think we ought to s/USE_SSE2/__x86_64__/ there just to make this less confusing, but the part of my quick-hack patch that touched pg_cpu.h was really a no-op. Also, the reason we manage to set USE_AVX2_WITH_RUNTIME_CHECK in a multi-arch build is that what configure actually tests is (a) is $host_cpu x86_64 and (b) does the compiler accept __attribute__((target("avx2"))). On a multi-arch build, the ARM side spits out a warning '+avx2' is not a recognized feature for this target (ignoring feature) but it doesn't actually fail, so configure thinks that's a success. I think that a cleaner solution to this is to fix pg_cpu.h so that x86_feature_available() exists but returns constant false on non-x86 platforms. A proposed patch for that is attached. If we just do that much, a multi-arch macOS build produces the "not a recognized feature" warning while compiling checksum.c, but it works. Given that the build also produces all those ranlib warnings, I don't know that suppressing "not a recognized feature" is worth worrying about. But if we want to, something similar to the checksum.c change I posted before would do it. regards, tom lane PS: does c.h's check for _M_AMD64 really do anything? If we need it there, why not in all the other places that check __x86_64__? Attachments: [text/x-diff] cleanup-x86-tests-1.patch (1.7K, 2-cleanup-x86-tests-1.patch) download | inline diff: diff --git a/src/include/port/pg_cpu.h b/src/include/port/pg_cpu.h index 566ed7a16e3..5b25747e43a 100644 --- a/src/include/port/pg_cpu.h +++ b/src/include/port/pg_cpu.h @@ -13,8 +13,6 @@ #ifndef PG_CPU_H #define PG_CPU_H -#if defined(USE_SSE2) || defined(__i386__) - typedef enum X86FeatureId { /* Have we run feature detection? */ @@ -43,6 +41,8 @@ typedef enum X86FeatureId } X86FeatureId; #define X86FeaturesSize (PG_TSC_ADJUST + 1) +#if defined(__x86_64__) || defined(__i386__) + extern PGDLLIMPORT bool X86Features[]; extern void set_x86_features(void); @@ -58,6 +58,14 @@ x86_feature_available(X86FeatureId feature) extern uint32 x86_tsc_frequency_khz(char *source, size_t source_size); -#endif /* defined(USE_SSE2) || defined(__i386__) */ +#else /* !(defined(__x86_64__)||defined(__i386__)) */ + +static inline bool +x86_feature_available(X86FeatureId feature) +{ + return false; +} + +#endif /* defined(__x86_64__) || defined(__i386__) */ #endif /* PG_CPU_H */ diff --git a/src/port/pg_cpu_x86.c b/src/port/pg_cpu_x86.c index 0405ba19f6f..b050677f717 100644 --- a/src/port/pg_cpu_x86.c +++ b/src/port/pg_cpu_x86.c @@ -19,7 +19,7 @@ #include "postgres_fe.h" #endif -#if defined(USE_SSE2) || defined(__i386__) +#if defined(__x86_64__) || defined(__i386__) #ifdef _MSC_VER #include <intrin.h> @@ -287,10 +287,10 @@ x86_hypervisor_tsc_frequency_khz(void) return 0; } -#else /* defined(USE_SSE2) || defined(__i386__) */ +#else /* defined(__x86_64__) || defined(__i386__) */ /* prevent linker complaints about empty module */ extern int pg_cpu_x86_dummy_variable; int pg_cpu_x86_dummy_variable = 0; -#endif /* ! (USE_SSE2 || __i386__) */ +#endif /* ! (__x86_64__ || __i386__) */ ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-02 18:57 Andrew Dunstan <[email protected]> parent: Tom Lane <[email protected]> 1 sibling, 0 replies; 24+ messages in thread From: Andrew Dunstan @ 2026-06-02 18:57 UTC (permalink / raw) To: Tom Lane <[email protected]>; Tobias Bussmann <[email protected]>; +Cc: John Naylor <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> On 2026-06-02 Tu 11:20 AM, Tom Lane wrote: > I wrote: >> However, it definitely is a regression that the build fails >> altogether. Too bad nobody tried the x86 -> ARM case earlier. > I replicated this on longfin's host (x86_64 mac mini). > It seems there are two problems: > > 1. pg_cpu.h believes that x86-specific code can be conditional on > > #if defined(USE_SSE2) || defined(__i386__) > > but macOS doesn't define __i386__, only __x86_64__. It works > anyway on single-arch builds because the test to set USE_SSE2 > succeeds, but not on multi-arch builds. > > 2. checksum.c believes that it's okay to call x86_feature_available > if USE_AVX2_WITH_RUNTIME_CHECK is set. I didn't track down just > why that's getting set in a multi-arch build when USE_SSE2 is not, > but it is, and that's probably good since it means we get at least > some optimization for x86 Macs. But we have to disregard it when > we're doing the ARM side. > > The attached quick hack makes the build work on my machine. > I'm hesitant to shove it into the tree though because I'm > not too certain whether there could be side-effects on > other platforms. I think the way to proceed for now is for > EDB to apply this patch in their build of beta1, and we can > review the patch at leisure afterwards. > > Sandeep tells me he's going to look at that tomorrow. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Broken build on macOS (Universal / Intel): cpuid instruction not available @ 2026-06-03 20:23 Tom Lane <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 0 replies; 24+ messages in thread From: Tom Lane @ 2026-06-03 20:23 UTC (permalink / raw) To: Tobias Bussmann <[email protected]>; +Cc: John Naylor <[email protected]>; Lukas Fittl <[email protected]>; Jakob Egger <[email protected]>; pgsql-hackers; Andres Freund <[email protected]>; Sandeep Thakkar <[email protected]> I wrote: > Also, the reason we manage to set USE_AVX2_WITH_RUNTIME_CHECK > in a multi-arch build is that what configure actually tests > is (a) is $host_cpu x86_64 and (b) does the compiler accept > __attribute__((target("avx2"))). On a multi-arch build, > the ARM side spits out a warning > '+avx2' is not a recognized feature for this target (ignoring feature) > but it doesn't actually fail, so configure thinks that's a success. After further contemplation, I think the best fix for this problem is to undo configure's mistake globally, not just in checksum.c. checksum.c is the only file using USE_AVX2_WITH_RUNTIME_CHECK today, but that might not be true forever. Also, I think it's pure chance that the test for that symbol succeeds in a universal build while other similar tests don't. So what the attached v2 does is to #undef USE_AVX2_WITH_RUNTIME_CHECK as well as the USE_AVX512 symbols in c.h, if we're not in fact building for x86_64 hardware. We could extend the list of things to #undef there later, if it proves necessary. The shape of this patch is partially dictated by not wanting to create problems for Munro's pending patch [1], but I think it's a reasonable solution even without that consideration. regards, tom lane [1] https://www.postgresql.org/message-id/flat/CA%2BhUKGL8Hs-phHPugrWM%3D5dAkcT897rXyazYzLw-Szxnzgx-rA%4... Attachments: [text/x-diff] v2-fix-macos-universal-build.patch (1.2K, 2-v2-fix-macos-universal-build.patch) download | inline diff: diff --git a/src/include/c.h b/src/include/c.h index 97ed8c63f5e..23db81fa7ca 100644 --- a/src/include/c.h +++ b/src/include/c.h @@ -1340,6 +1340,17 @@ typedef struct PGAlignedXLogBlock PGAlignedXLogBlock; #if (defined(__x86_64__) || defined(_M_AMD64)) #define USE_SSE2 +#else /* ! x86_64 */ + +/* + * In "universal" macOS builds, it's possible for AVX-related symbols to + * get defined if the build host is x86_64, but we mustn't try to build + * that code when cross-compiling to aarch64. + */ +#undef USE_AVX2_WITH_RUNTIME_CHECK +#undef USE_AVX512_CRC32C_WITH_RUNTIME_CHECK +#undef USE_AVX512_POPCNT_WITH_RUNTIME_CHECK + /* * We use the Neon instructions if the compiler provides access to them (as * indicated by __ARM_NEON) and we are on aarch64. While Neon support is @@ -1348,9 +1359,10 @@ typedef struct PGAlignedXLogBlock PGAlignedXLogBlock; * could not realistically use it there without a run-time check, which seems * not worth the trouble for now. */ -#elif defined(__aarch64__) && defined(__ARM_NEON) +#if defined(__aarch64__) && defined(__ARM_NEON) #define USE_NEON #endif +#endif /* ! x86_64 */ /* ---------------------------------------------------------------- * Section 9: system-specific hacks ^ permalink raw reply [nested|flat] 24+ messages in thread
end of thread, other threads:[~2026-06-03 20:23 UTC | newest] Thread overview: 24+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-05-07 11:41 Broken build on macOS (Universal / Intel): cpuid instruction not available Jakob Egger <[email protected]> 2026-05-07 12:42 ` John Naylor <[email protected]> 2026-05-07 12:50 ` Daniel Gustafsson <[email protected]> 2026-05-07 13:59 ` Tom Lane <[email protected]> 2026-05-07 14:38 ` Jakob Egger <[email protected]> 2026-05-07 15:41 ` Jakob Egger <[email protected]> 2026-05-07 14:57 ` Tobias Bussmann <[email protected]> 2026-05-07 15:19 ` Tom Lane <[email protected]> 2026-05-07 15:48 ` Tom Lane <[email protected]> 2026-05-07 15:57 ` Nathan Bossart <[email protected]> 2026-05-07 16:22 ` Tom Lane <[email protected]> 2026-05-07 17:33 ` Lukas Fittl <[email protected]> 2026-05-08 03:48 ` John Naylor <[email protected]> 2026-05-08 08:26 ` Tobias Bussmann <[email protected]> 2026-05-08 10:04 ` John Naylor <[email protected]> 2026-05-08 13:54 ` Tom Lane <[email protected]> 2026-06-02 12:49 ` Tobias Bussmann <[email protected]> 2026-06-02 14:48 ` Tom Lane <[email protected]> 2026-06-02 15:10 ` Tobias Bussmann <[email protected]> 2026-06-02 16:24 ` Tom Lane <[email protected]> 2026-06-02 15:20 ` Tom Lane <[email protected]> 2026-06-02 17:38 ` Tom Lane <[email protected]> 2026-06-03 20:23 ` Tom Lane <[email protected]> 2026-06-02 18:57 ` Andrew Dunstan <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox