Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p23mk-0001Bo-Up for pgsql-hackers@arkaria.postgresql.org; Mon, 05 Dec 2022 05:12:59 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1p23mg-0007XU-QV for pgsql-hackers@arkaria.postgresql.org; Mon, 05 Dec 2022 05:12:54 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p23mg-0007XL-8s for pgsql-hackers@lists.postgresql.org; Mon, 05 Dec 2022 05:12:54 +0000 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p23mc-0005uW-FZ for pgsql-hackers@lists.postgresql.org; Mon, 05 Dec 2022 05:12:53 +0000 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id BD21332005D8; Mon, 5 Dec 2022 00:12:47 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 05 Dec 2022 00:12:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= cc:cc:content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1670217167; x=1670303567; bh=6o4glCZCux CmHcJtR2e0zyYQY9ms1LoL1Pig3dm7qBs=; b=F6V8rVpxCf4L7iUJ4rgBHJ458k 6bBJVvh8M8qLytOT3++ZJlHgdOUA6XwkEO0ZCeVjTJ7ivMqPOR/ThBwFuU7Prmv4 SvIxeJRVpje6MK/ARacBjgugX1RK9W+xGLw08WwCOh97y89/eGnzz3fiSyFrYEOw fjpnsOBRaT6iRHy3CAWLbbn/YmaEXyDU/H7nWDVG9XmJvDHJcaElc/49RXPALLhJ FYcc7pAO7HMYq7DIbEtWnlyV1sNz5nSjK+bRjR6uHfO0BZk0PuXrQPAs6A3sEuMq 0tf/Dnh4Lymxd+tsdHvf1v8Hf3c+BdUsHmjFt9fYf0OebREtVteaCJmYvXPQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1670217167; x=1670303567; bh=6o4glCZCuxCmHcJtR2e0zyYQY9ms 1LoL1Pig3dm7qBs=; b=ZafPQrBuoWJKLKzPRR8s9vO20goO83pKWq2zDLSsXD1T 3WScQummswxOS+nBrsorQebekphRw+QU8+DH2P8h4T6o/MrG4GVpxOxwXK8ITKeY PiZvivRJQX2e0pNsdzfhVZZbcbDFE0aVhKjXPubv7mFStoQnsJr+DWeL/gYdpGEl AV7fjdGG7ia3WEbGcOo2A9B7Oi2469vjEo5M9LsgeFJNVAGDSEoMo42v1ASw0/lo lyZOok1rDvtZb9pcQmCoZfeGubANKnOFYlWebWwOOZrAh4uvF9ZsfZQXYjFZSGCE 2E1eYMDjcBVyFbekprlq+aXa1ahFpMAnAtyb2KvYnQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrudefgdekudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfg hrlhcuvffnffculdefhedmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddt vdenucfhrhhomhepofhitghhrggvlhcurfgrqhhuihgvrhcuoehmihgthhgrvghlsehprg hquhhivghrrdighiiiqeenucggtffrrghtthgvrhhnpeehtdfhvdettddttdetudetjeev heduuedugeetfeehieetkeetgeegfedvuddutdenucffohhmrghinhepshhlphhrohifvg gsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhepmhhitghhrggvlhesphgrqhhuihgvrhdrgiihii X-ME-Proxy: Feedback-ID: i0fe9450f:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 5 Dec 2022 00:12:44 -0500 (EST) Date: Mon, 5 Dec 2022 14:12:41 +0900 From: Michael Paquier To: Niyas Sait Cc: Andres Freund , Ian Lawrence Barwick , Tom Lane , Thomas Munro , Julien Rouhaud , PostgreSQL Hackers Subject: Re: [PATCH] Add native windows on arm64 support Message-ID: References: <271611.1661998926@sss.pgh.pa.us> <20221105183136.hfxyl5dclxdcoyih@awork3.anarazel.de> <7ff7c416-fdfe-517f-a65c-9e5bd8ff8cec@linaro.org> <8d46c46d-66d3-758b-b686-de371e0ba8ca@linaro.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="aTJJx6hfMPZZ/Ldb" Content-Disposition: inline In-Reply-To: <8d46c46d-66d3-758b-b686-de371e0ba8ca@linaro.org> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --aTJJx6hfMPZZ/Ldb Content-Type: multipart/mixed; boundary="DtkylaiSDVEvtVa+" Content-Disposition: inline --DtkylaiSDVEvtVa+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 02, 2022 at 11:09:15AM +0000, Niyas Sait wrote: > I've attached a new revision of the patch (v5) and includes following > changes, >=20 > 1. Add support for meson build system > 2. Extend MSVC scripts to handle ARM64 platform. > 3. Add arm64 definition of spin_delay function. > 4. Exclude arm_acle.h import with MSVC compiler. Hmm. There are still a few things that need some attention here: - USE_SSE42_CRC32C_WITH_RUNTIME_CHECK should not be set for aarch64. - This is missing updates for ICU. Looking at the upstream code, Build.Windows.ProjectConfiguration.props uses libARM64 and binARM64 for the output library and binary paths. - This is missing updates for krb5. For this case, I am seeing no traces of packages for aarch64, so I guess that we could just fail hard until someone cares enough to ping us about what to do here. - There were zero changes in the docs, but we need to update at least the section about architectures supported for the 64-bit builds. - Last comes OpenSSL, that supports amd64_arm64 as build target (see NOTES-WINDOWS.md), and the library names are libssl.lib and libcrypto.lib. Looking at https://slproweb.com/products/Win32OpenSSL.html, there are experimental builds for arm64 with OpenSSL 3.0. Niyas or somebody else, could you look at the contents of lib/VC/ and see what we could rely on for the debug builds after installing this MSI? We should rely on something like lib/VC/sslcrypto64MD.lib or lib/VC/sslcrypto32MD.lib, but for arm64. With meson gaining in maturity, perhaps that's not the most urgent thing as we will likely remove src/tools/msvc/ soon but I'd rather do that right anyway as much as I can to avoid an incorrect state in the tree at any time in its history. - USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK =3D> undef, + USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK =3D> $self->{platform} eq "ARM6= 4" ? : 1 : undef, Did you actually test this patch? This won't work at all with perl, per se the double colon after the question mark. For now, please find attached an updated patch with all the fixes I could come up with. -- Michael --DtkylaiSDVEvtVa+ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v6-0001-Enable-postgres-native-build-for-windows-arm64-pl.patch" Content-Transfer-Encoding: quoted-printable =46rom 9cb858cfdb25c3a119a8b28321cf3218eac282c5 Mon Sep 17 00:00:00 2001 =46rom: Michael Paquier Date: Mon, 5 Dec 2022 14:09:29 +0900 Subject: [PATCH v6] Enable postgres native build for windows-arm64 platform Following changes are included - Extend MSVC scripts to handle ARM64 platform - Add arm64 definition of spin_delay function - Exclude arm_acle.h import for MSVC - Add support for meson build --- src/include/storage/s_lock.h | 10 +++++++++- src/port/pg_crc32c_armv8.c | 2 ++ doc/src/sgml/install-windows.sgml | 3 ++- meson.build | 33 +++++++++++++++++++------------ src/tools/msvc/MSBuildProject.pm | 16 +++++++++++---- src/tools/msvc/Mkvcbuild.pm | 9 +++++++-- src/tools/msvc/Solution.pm | 30 ++++++++++++++++++++++------ src/tools/msvc/gendef.pl | 8 ++++---- 8 files changed, 80 insertions(+), 31 deletions(-) diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index 8b19ab160f..ab6a6e0281 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h @@ -708,13 +708,21 @@ typedef LONG slock_t; #define SPIN_DELAY() spin_delay() =20 /* If using Visual C++ on Win64, inline assembly is unavailable. - * Use a _mm_pause intrinsic instead of rep nop. + * Use _mm_pause (x64) or __isb (arm64) intrinsic instead of rep nop. */ #if defined(_WIN64) static __forceinline void spin_delay(void) { +#ifdef _M_ARM64 + /* + * See spin_delay aarch64 inline assembly definition above for details + * ref: https://learn.microsoft.com/en-us/cpp/intrinsics/arm64-intrinsics= #BarrierRestrictions + */ + __isb(_ARM64_BARRIER_SY); +#else _mm_pause(); +#endif } #else static __forceinline void diff --git a/src/port/pg_crc32c_armv8.c b/src/port/pg_crc32c_armv8.c index 9e301f96f6..981718752f 100644 --- a/src/port/pg_crc32c_armv8.c +++ b/src/port/pg_crc32c_armv8.c @@ -14,7 +14,9 @@ */ #include "c.h" =20 +#ifndef _MSC_VER #include +#endif =20 #include "port/pg_crc32c.h" =20 diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windo= ws.sgml index bbd4960e7b..3f865d7d3b 100644 --- a/doc/src/sgml/install-windows.sgml +++ b/doc/src/sgml/install-windows.sgml @@ -352,7 +352,8 @@ $ENV{MSBFLAGS}=3D"/m"; Special Considerations for 64-Bit Windows =20 - PostgreSQL will only build for the x64 architecture on 64-bit Windows. + PostgreSQL will only build for the x64 and ARM64 architectures on 64-bit + Windows. =20 diff --git a/meson.build b/meson.build index 725e10d815..e354ad7650 100644 --- a/meson.build +++ b/meson.build @@ -1944,7 +1944,13 @@ int main(void) =20 elif host_cpu =3D=3D 'arm' or host_cpu =3D=3D 'aarch64' =20 - prog =3D ''' + if cc.get_id() =3D=3D 'msvc' + cdata.set('USE_ARMV8_CRC32C', false) + cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1) + have_optimized_crc =3D true + else + + prog =3D ''' #include =20 int main(void) @@ -1960,18 +1966,19 @@ int main(void) } ''' =20 - if cc.links(prog, name: '__crc32cb, __crc32ch, __crc32cw, and __crc32cd = without -march=3Darmv8-a+crc', - args: test_c_args) - # Use ARM CRC Extension unconditionally - cdata.set('USE_ARMV8_CRC32C', 1) - have_optimized_crc =3D true - elif cc.links(prog, name: '__crc32cb, __crc32ch, __crc32cw, and __crc32c= d with -march=3Darmv8-a+crc', - args: test_c_args + ['-march=3Darmv8-a+crc']) - # Use ARM CRC Extension, with runtime check - cflags_crc +=3D '-march=3Darmv8-a+crc' - cdata.set('USE_ARMV8_CRC32C', false) - cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1) - have_optimized_crc =3D true + if cc.links(prog, name: '__crc32cb, __crc32ch, __crc32cw, and __crc32c= d without -march=3Darmv8-a+crc', + args: test_c_args) + # Use ARM CRC Extension unconditionally + cdata.set('USE_ARMV8_CRC32C', 1) + have_optimized_crc =3D true + elif cc.links(prog, name: '__crc32cb, __crc32ch, __crc32cw, and __crc3= 2cd with -march=3Darmv8-a+crc', + args: test_c_args + ['-march=3Darmv8-a+crc']) + # Use ARM CRC Extension, with runtime check + cflags_crc +=3D '-march=3Darmv8-a+crc' + cdata.set('USE_ARMV8_CRC32C', false) + cdata.set('USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK', 1) + have_optimized_crc =3D true + endif endif endif =20 diff --git a/src/tools/msvc/MSBuildProject.pm b/src/tools/msvc/MSBuildProje= ct.pm index 58590fdac2..274ddc8860 100644 --- a/src/tools/msvc/MSBuildProject.pm +++ b/src/tools/msvc/MSBuildProject.pm @@ -310,10 +310,18 @@ sub WriteItemDefinitionGroup : ($self->{type} eq "dll" ? 'DynamicLibrary' : 'StaticLibrary'); my $libs =3D $self->GetAdditionalLinkerDependencies($cfgname, ';'); =20 - my $targetmachine =3D - $self->{platform} eq 'Win32' ? 'MachineX86' : 'MachineX64'; - my $arch =3D - $self->{platform} eq 'Win32' ? 'x86' : 'x86_64'; + my $targetmachine; + my $arch; + if ($self->{platform} eq 'Win32') { + $targetmachine =3D 'MachineX86'; + $arch =3D 'x86'; + } elsif ($self->{platform} eq 'ARM64'){ + $targetmachine =3D 'MachineARM64'; + $arch =3D 'aarch64'; + } else { + $targetmachine =3D 'MachineX64'; + $arch =3D 'x86_64'; + } =20 my $includes =3D join ';', @{ $self->{includes} }, ""; =20 diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm index 7e52e9ad0a..c6e8e91f5f 100644 --- a/src/tools/msvc/Mkvcbuild.pm +++ b/src/tools/msvc/Mkvcbuild.pm @@ -123,8 +123,13 @@ sub mkvcbuild =20 if ($vsVersion >=3D '9.00') { - push(@pgportfiles, 'pg_crc32c_sse42_choose.c'); - push(@pgportfiles, 'pg_crc32c_sse42.c'); + if ($solution->{platform} eq 'ARM64') { + push(@pgportfiles, 'pg_crc32c_armv8_choose.c'); + push(@pgportfiles, 'pg_crc32c_armv8.c'); + } else { + push(@pgportfiles, 'pg_crc32c_sse42_choose.c'); + push(@pgportfiles, 'pg_crc32c_sse42.c'); + } push(@pgportfiles, 'pg_crc32c_sb8.c'); } else diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm index c2acb58df0..f7a5c83c8f 100644 --- a/src/tools/msvc/Solution.pm +++ b/src/tools/msvc/Solution.pm @@ -67,8 +67,13 @@ sub DeterminePlatform # Examine CL help output to determine if we are in 32 or 64-bit mode. my $output =3D `cl /help 2>&1`; $? >> 8 =3D=3D 0 or die "cl command not found"; - $self->{platform} =3D - ($output =3D~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32'; + if ($output =3D~ /^\/favor:<.+AMD64/m) { + $self->{platform} =3D 'x64'; + } elsif($output =3D~ /for ARM64$/m) { + $self->{platform} =3D 'ARM64'; + } else { + $self->{platform} =3D 'Win32'; + } } else { @@ -423,7 +428,8 @@ sub GenerateFiles STDC_HEADERS =3D> 1, STRERROR_R_INT =3D> undef, USE_ARMV8_CRC32C =3D> undef, - USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK =3D> undef, + USE_ARMV8_CRC32C_WITH_RUNTIME_CHECK =3D> + ($self->{platform} eq "ARM64") ? 1 : undef, USE_ASSERT_CHECKING =3D> $self->{options}->{asserts} ? 1 : undef, USE_BONJOUR =3D> undef, USE_BSD_AUTH =3D> undef, @@ -438,7 +444,8 @@ sub GenerateFiles USE_PAM =3D> undef, USE_SLICING_BY_8_CRC32C =3D> undef, USE_SSE42_CRC32C =3D> undef, - USE_SSE42_CRC32C_WITH_RUNTIME_CHECK =3D> 1, + USE_SSE42_CRC32C_WITH_RUNTIME_CHECK =3D> + ($self->{platform} eq "x64" || $self->{platform} eq "Win32") ? 1 : und= ef, USE_SYSTEMD =3D> undef, USE_SYSV_SEMAPHORES =3D> undef, USE_SYSV_SHARED_MEMORY =3D> undef, @@ -1013,7 +1020,7 @@ sub AddProject $proj->AddLibrary( $self->{options}->{gss} . '\lib\i386\gssapi32.lib'); } - else + elsif ($self->{platform} eq 'x64') { $proj->AddLibrary( $self->{options}->{gss} . '\lib\amd64\krb5_64.lib'); @@ -1022,6 +1029,10 @@ sub AddProject $proj->AddLibrary( $self->{options}->{gss} . '\lib\amd64\gssapi64.lib'); } + else + { + die "Kerberos not supported with ARM64\n"; + } } if ($self->{options}->{iconv}) { @@ -1037,12 +1048,19 @@ sub AddProject $proj->AddLibrary($self->{options}->{icu} . '\lib\icuuc.lib'); $proj->AddLibrary($self->{options}->{icu} . '\lib\icudt.lib'); } - else + elsif ($self->{platform} eq 'x64') { $proj->AddLibrary($self->{options}->{icu} . '\lib64\icuin.lib'); $proj->AddLibrary($self->{options}->{icu} . '\lib64\icuuc.lib'); $proj->AddLibrary($self->{options}->{icu} . '\lib64\icudt.lib'); } + else + { + # aarch64. + $proj->AddLibrary($self->{options}->{icu} . '\libARM64\icuin.lib'); + $proj->AddLibrary($self->{options}->{icu} . '\libARM64\icuuc.lib'); + $proj->AddLibrary($self->{options}->{icu} . '\libARM64\icudt.lib'); + } } if ($self->{options}->{xml}) { diff --git a/src/tools/msvc/gendef.pl b/src/tools/msvc/gendef.pl index d6bed1ce15..4882d37faf 100644 --- a/src/tools/msvc/gendef.pl +++ b/src/tools/msvc/gendef.pl @@ -120,9 +120,9 @@ sub writedef { my $isdata =3D $def->{$f} eq 'data'; =20 - # Strip the leading underscore for win32, but not x64 + # Strip the leading underscore for win32, but not x64 and aarch64 $f =3D~ s/^_// - unless ($arch eq "x86_64"); + unless ($arch eq "x86_64" || $arch eq "aarch64"); =20 # Emit just the name if it's a function symbol, or emit the name # decorated with the DATA option for variables. @@ -143,7 +143,7 @@ sub writedef sub usage { die("Usage: gendef.pl --arch --deffile --tempdir files-or-directories\n" - . " arch: x86 | x86_64\n" + . " arch: x86 | x86_64 | aarch64\n" . " deffile: path of the generated file\n" . " tempdir: directory for temporary files\n" . " files or directories: object files or directory containing obje= ct files\n" @@ -160,7 +160,7 @@ GetOptions( 'tempdir:s' =3D> \$tempdir,) or usage(); =20 usage("arch: $arch") - unless ($arch eq 'x86' || $arch eq 'x86_64'); + unless ($arch eq 'x86' || $arch eq 'x86_64' || $arch eq 'aarch64'); =20 my @files; =20 --=20 2.38.1 --DtkylaiSDVEvtVa+-- --aTJJx6hfMPZZ/Ldb Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmONfckACgkQnvQgOdby QH2VTQ/8DaaoWcmQBn0o1GOkBwN89Lz2rBL/zQVJ7pAEX1+VRN9HzUDmhY55HUCJ CeU8YBeWahcO9xK4awLQRYCeZMNUNegRKgS7U5kNFGiJ/01Pp+MgPy3HImuIXOjP ZvqqbNqhNtGe/1DEpWkDF+HThaX7Q8Wwbw4RCHsimZDvA2IBbhnTxzA//YDQ5gey FjbnTISnYYRSn+NwtaUv1lCoBgl7lr48fGp4F067VPXyirnPp8LosD7EVsrJafiF ah60zaag4Ey9phPHKd9Wzwbfd9YoP4YFIl2mrHoj1t+SIN/7JfLjj/ed8sHkJLC5 38lJMmaPwD+SvnaZ45o5+eeL6pDxQ/BHrfbDIOhWGKw6BAX0W++lxDsyyEV70Poo jdHGq6+IHEjPyxr8SpJfkNnST0Ddu4d08ETfuxYuB33WcWKFYB8jGH1i2v6vYq0b g+LGPBhZzuZmXTVGWOnVhfn3dV6Y0p1vNNZM6ENqdVR4AIKkiBLYegvCvVVYZXal Ac6WYxNh2XmH78ZIG67xdJxJV3VsXSQZ5vQjSYwY6+Np1Rzx+qauVsPJh5pjI94g b/YW3Csy3UP5c/FWfMRRT7AEZAnoGE5SLhAhoSwT0w+Db8ql48dM7vMSMTgeTdtt UIoi4At/uojQJ0VG99eIlStJo1yq6kz0mcX060W+C6+Ml+Ous18= =/GW0 -----END PGP SIGNATURE----- --aTJJx6hfMPZZ/Ldb--