From: Sergey Shinderuk <s.shinderuk@postgrespro.ru>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: pg_preadv() and pg_pwritev()
Date: Fri, 15 Jan 2021 03:13:17 +0300
Message-ID: <dc9aa033-ad66-d078-ceec-6b5a9de9c401@postgrespro.ru> (raw)
In-Reply-To: <1612230.1610662425@sss.pgh.pa.us>
References: <CA+hUKGJA+u-220VONeoREBXJ9P3S94Y7J+kqCnTYmahvZJwM=g@mail.gmail.com>
<20201220224040.evr5pduibvm6dze5@alap3.anarazel.de>
<CA+hUKG+TCoareSYcAu05yaMh9eUck6Hd8FQZt5VEvotNs3YOMQ@mail.gmail.com>
<CA+hUKGJ8Ww_fjzUzthdxUF6W3JQ3r=a1hWX7Fcfq9uEndZ_+GQ@mail.gmail.com>
<CA+hUKGL9XFxM0ZO8ZADYJepSmjywknhnDkKSEttfmn2j9fg0BQ@mail.gmail.com>
<ed3b8e5d-0da8-6ebd-fd1c-e0ac80a4b204@postgrespro.ru>
<CA+hUKG+L573j4=XWv6S+r-687Yxn0FyXhtObH9_P-Nu_atL2pQ@mail.gmail.com>
<448b3cb5-0ba0-6dc5-63c7-789e4eebc40b@postgrespro.ru>
<1283127.1610554395@sss.pgh.pa.us>
<1286060.1610558607@sss.pgh.pa.us>
<1297389.1610562046@sss.pgh.pa.us>
<1341608.1610580122@sss.pgh.pa.us>
<c2e08e38-e4bd-780d-62df-4a33b537b75e@postgrespro.ru>
<d90658e6-490e-db57-cd64-98868abe79cd@postgrespro.ru>
<1410650.1610638974@sss.pgh.pa.us>
<b2586af5-866f-8290-f068-046fbd1a01b8@postgrespro.ru>
<1418166.1610647514@sss.pgh.pa.u! s>
<1419199.1610649100@sss.pgh.pa.us>
<1612230.1610662425@sss.pgh.pa.us>
On 15.01.2021 01:13, Tom Lane wrote:
> I borrowed my wife's Mac, which is still on Catalina and up to now
> never had Xcode on it, and found some very interesting things.
>
> Step 1: download/install Xcode 12.3, open it, agree to license,
> wait for it to finish "installing components".
>
> At this point, /Library/Developer/CommandLineTools doesn't exist,
> and we have the following outputs from various probe commands:
>
> % xcrun --show-sdk-path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
> % xcrun --sdk macosx --show-sdk-path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> % xcodebuild -version -sdk macosx Path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
>
> Also, cc -v reports
> -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
>
> Unsurprisingly, Xcode 12.3 itself only contains
>
> % ls -l /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs
> total 0
> drwxr-xr-x 5 root wheel 160 Nov 30 07:27 DriverKit20.2.sdk
> drwxr-xr-x 7 root wheel 224 Nov 30 07:27 MacOSX.sdk
> lrwxr-xr-x 1 root wheel 10 Jan 14 15:57 MacOSX11.1.sdk -> MacOSX.sdk
>
> Step 2: install command line tools (I used "xcode-select --install"
> to fire this off, rather than the Xcode menu item).
>
> Now I have
>
> % ls -l /Library/Developer/CommandLineTools/SDKs
> total 0
> lrwxr-xr-x 1 root wheel 14 Jan 14 16:42 MacOSX.sdk -> MacOSX11.1.sdk
> drwxr-xr-x 8 root wheel 256 Jul 9 2020 MacOSX10.15.sdk
> drwxr-xr-x 7 root wheel 224 Nov 30 07:33 MacOSX11.1.sdk
>
> which is pretty interesting in itself, because the same directory on
> my recently-updated-to-Big-Sur Macs does NOT have the 11.1 SDK.
> I wonder what determines which versions get installed here.
>
> More interesting yet:
>
> % xcrun --show-sdk-path
> /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
> % xcrun --sdk macosx --show-sdk-path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> % xcodebuild -version -sdk macosx Path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
>
> and cc -v reports
> -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
>
> So apparently, "xcrun --show-sdk-path" (without any -sdk option)
> is the most authoritative guide to the compiler's default sysroot.
>
> However, googling turns up various people reporting that "xcrun
> --show-sdk-path" returns an empty string for them, and our last
> major investigation into this [1] found that there are some system
> states where the compiler appears to have no default sysroot,
> which I bet is the same thing. I do not at this point have a recipe
> to reproduce such a state, but we'd be fools to imagine it's no
> longer possible. My guess about it is that Apple's processes for
> updating the default sysroot during system updates are just plain
> buggy, with various corner cases that have ill-understood causes.
>
> Also, after re-reading [1] I am not at all excited about trying to
> remove the -isysroot switches from our *FLAGS. What I propose to do
> is keep that, but improve our mechanism for choosing a default value
> for PG_SYSROOT. It looks like first trying "xcrun --show-sdk-path",
> and falling back to "xcodebuild -version -sdk macosx Path" if that
> doesn't yield a valid path, is more likely to give a working build
> than relying entirely on xcodebuild. Maybe there's a case for trying
> "xcrun --sdk macosx --show-sdk-path" in between; in my tests that
> seemed noticeably faster than invoking xcodebuild, and I've not yet
> seen a case where it gave a different answer.
>
> Thoughts?
>
> regards, tom lane
>
> [1] https://www.postgresql.org/message-id/flat/20840.1537850987%40sss.pgh.pa.us
>
Thanks for thorough investigation and sorry for the late reply.
I spent quite some time trying to understand / reverse engineer the
logic behind xcrun's default SDK selection. Apparently, "man xcrun" is
not accurate saying:
The SDK which will be searched defaults to the most recent
available...
I didn't find anything really useful or helpful.
"/Library/Developer/CommandLineTools" is hardcoded into
"libxcrun.dylib". On my machine xcrun scans
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs
and
/Library/Developer/CommandLineTools/SDKs
in that order, and loads "SDKSettings.plist" from each subdirectory. I
looked into plists, but couldn't find anything special about
"MacOSX10.15.sdk".
Okay, here is what I have:
% ls -l
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs
total 0
drwxr-xr-x 5 root wheel 160 Nov 30 15:27 DriverKit20.2.sdk
drwxr-xr-x 7 root wheel 224 Nov 30 15:27 MacOSX.sdk
lrwxr-xr-x 1 root wheel 10 Dec 17 14:25 MacOSX11.1.sdk -> MacOSX.sdk
% ls -l /Library/Developer/CommandLineTools/SDKs
total 0
lrwxr-xr-x 1 root wheel 14 Nov 17 02:21 MacOSX.sdk -> MacOSX11.0.sdk
drwxr-xr-x 8 root wheel 256 Nov 17 02:22 MacOSX10.15.sdk
drwxr-xr-x 7 root wheel 224 Oct 19 23:39 MacOSX11.0.sdk
% xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
% xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
Oh, that's weird! Nevertheless I like you suggestion to call "xcrun"
from "configure".
Adding "--verbose" doesn't really explain anything, but just in case.
% xcrun --verbose --no-cache --find cc
xcrun: note: PATH =
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT =
'/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db =
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: xcrun via cc (xcrun)
xcrun: note: database key is:
cc|/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk||/Applications/Xcode.app/Contents/Developer|
xcrun: note: looking up with
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk
macosx -find cc 2> /dev/null'
xcrun: note: lookup resolved with 'xcodebuild -find' to
'/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
% xcrun --verbose --no-cache --sdk macosx --find cc
xcrun: note: looking up SDK with
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk
macosx -version Path'
xcrun: note: PATH =
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT = 'macosx'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db =
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: lookup resolved to:
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk'
xcrun: note: looking up SDK with
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
-version PlatformPath'
xcrun: note: PATH =
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT =
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db =
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: lookup resolved to:
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform'
xcrun: note: PATH =
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT =
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db =
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: xcrun via cc (xcrun)
xcrun: note: database key is:
cc|/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk||/Applications/Xcode.app/Contents/Developer|
xcrun: note: looking up with
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
-find cc 2> /dev/null'
xcrun: note: lookup resolved with 'xcodebuild -find' to
'/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: s.shinderuk@postgrespro.ru, tgl@sss.pgh.pa.us, thomas.munro@gmail.com
Subject: Re: pg_preadv() and pg_pwritev()
In-Reply-To: <dc9aa033-ad66-d078-ceec-6b5a9de9c401@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox