Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1l0Ck8-0007o7-Am for pgsql-hackers@arkaria.postgresql.org; Fri, 15 Jan 2021 00:13:32 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1l0Ck5-0000LF-Fa for pgsql-hackers@arkaria.postgresql.org; Fri, 15 Jan 2021 00:13:29 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1l0Ck5-0000L8-0U for pgsql-hackers@lists.postgresql.org; Fri, 15 Jan 2021 00:13:29 +0000 Received: from mail.postgrespro.ru ([93.174.131.139]) by makus.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1l0Cjw-0001kQ-Jp for pgsql-hackers@postgresql.org; Fri, 15 Jan 2021 00:13:27 +0000 Received: from Sergeys-MacBook-Pro.local (unknown [62.217.185.248]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mail.postgrespro.ru (Postfix) with ESMTPSA id 06B2B21C36B1; Fri, 15 Jan 2021 03:13:17 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mail; t=1610669598; bh=kXB670LzxPUtK3ALYc1WSLUPE2MN54KRqD/20B6Den8=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=bbn27oX/3/PlalIYHpiHsHcAnTR1Kom7KCZBrJBLV82uZmFqrBhau1HHeIPdHuyIo U7G3PU/ZsEchiiBY3vFgaiNFxG1iZbMN+d8xh9A9vkUGswAD8eousZs+qCJkRw12o2 leZz6NGCY1m9UDPuRbTZ93cOdzZVmxuOlxb95GMk= Subject: Re: pg_preadv() and pg_pwritev() To: Tom Lane Cc: Thomas Munro , pgsql-hackers References: <20201220224040.evr5pduibvm6dze5@alap3.anarazel.de> <448b3cb5-0ba0-6dc5-63c7-789e4eebc40b@postgrespro.ru> <1283127.1610554395@sss.pgh.pa.us> <1286060.1610558607@sss.pgh.pa.us> <1297389.1610562046@sss.pgh.pa.us> <1341608.1610580122@sss.pgh.pa.us> <1410650.1610638974@sss.pgh.pa.us> <1418166.1610647514@sss.pgh.pa.u! s> <1419199.1610649100@sss.pgh.pa.us> <1612230.1610662425@sss.pgh.pa.us> From: Sergey Shinderuk Message-ID: Date: Fri, 15 Jan 2021 03:13:17 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <1612230.1610662425@sss.pgh.pa.us> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 15.01.2021 01:13, Tom Lane wrote: > I borrowed my wife's Mac, which is still on Catalina and up to now > never had Xcode on it, and found some very interesting things. > > Step 1: download/install Xcode 12.3, open it, agree to license, > wait for it to finish "installing components". > > At this point, /Library/Developer/CommandLineTools doesn't exist, > and we have the following outputs from various probe commands: > > % xcrun --show-sdk-path > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk > % xcrun --sdk macosx --show-sdk-path > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk > % xcodebuild -version -sdk macosx Path > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk > > Also, cc -v reports > -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk > > Unsurprisingly, Xcode 12.3 itself only contains > > % ls -l /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs > total 0 > drwxr-xr-x 5 root wheel 160 Nov 30 07:27 DriverKit20.2.sdk > drwxr-xr-x 7 root wheel 224 Nov 30 07:27 MacOSX.sdk > lrwxr-xr-x 1 root wheel 10 Jan 14 15:57 MacOSX11.1.sdk -> MacOSX.sdk > > Step 2: install command line tools (I used "xcode-select --install" > to fire this off, rather than the Xcode menu item). > > Now I have > > % ls -l /Library/Developer/CommandLineTools/SDKs > total 0 > lrwxr-xr-x 1 root wheel 14 Jan 14 16:42 MacOSX.sdk -> MacOSX11.1.sdk > drwxr-xr-x 8 root wheel 256 Jul 9 2020 MacOSX10.15.sdk > drwxr-xr-x 7 root wheel 224 Nov 30 07:33 MacOSX11.1.sdk > > which is pretty interesting in itself, because the same directory on > my recently-updated-to-Big-Sur Macs does NOT have the 11.1 SDK. > I wonder what determines which versions get installed here. > > More interesting yet: > > % xcrun --show-sdk-path > /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk > % xcrun --sdk macosx --show-sdk-path > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk > % xcodebuild -version -sdk macosx Path > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk > > and cc -v reports > -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk > > So apparently, "xcrun --show-sdk-path" (without any -sdk option) > is the most authoritative guide to the compiler's default sysroot. > > However, googling turns up various people reporting that "xcrun > --show-sdk-path" returns an empty string for them, and our last > major investigation into this [1] found that there are some system > states where the compiler appears to have no default sysroot, > which I bet is the same thing. I do not at this point have a recipe > to reproduce such a state, but we'd be fools to imagine it's no > longer possible. My guess about it is that Apple's processes for > updating the default sysroot during system updates are just plain > buggy, with various corner cases that have ill-understood causes. > > Also, after re-reading [1] I am not at all excited about trying to > remove the -isysroot switches from our *FLAGS. What I propose to do > is keep that, but improve our mechanism for choosing a default value > for PG_SYSROOT. It looks like first trying "xcrun --show-sdk-path", > and falling back to "xcodebuild -version -sdk macosx Path" if that > doesn't yield a valid path, is more likely to give a working build > than relying entirely on xcodebuild. Maybe there's a case for trying > "xcrun --sdk macosx --show-sdk-path" in between; in my tests that > seemed noticeably faster than invoking xcodebuild, and I've not yet > seen a case where it gave a different answer. > > Thoughts? > > regards, tom lane > > [1] https://www.postgresql.org/message-id/flat/20840.1537850987%40sss.pgh.pa.us > Thanks for thorough investigation and sorry for the late reply. I spent quite some time trying to understand / reverse engineer the logic behind xcrun's default SDK selection. Apparently, "man xcrun" is not accurate saying: The SDK which will be searched defaults to the most recent available... I didn't find anything really useful or helpful. "/Library/Developer/CommandLineTools" is hardcoded into "libxcrun.dylib". On my machine xcrun scans /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs and /Library/Developer/CommandLineTools/SDKs in that order, and loads "SDKSettings.plist" from each subdirectory. I looked into plists, but couldn't find anything special about "MacOSX10.15.sdk". Okay, here is what I have: % ls -l /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs total 0 drwxr-xr-x 5 root wheel 160 Nov 30 15:27 DriverKit20.2.sdk drwxr-xr-x 7 root wheel 224 Nov 30 15:27 MacOSX.sdk lrwxr-xr-x 1 root wheel 10 Dec 17 14:25 MacOSX11.1.sdk -> MacOSX.sdk % ls -l /Library/Developer/CommandLineTools/SDKs total 0 lrwxr-xr-x 1 root wheel 14 Nov 17 02:21 MacOSX.sdk -> MacOSX11.0.sdk drwxr-xr-x 8 root wheel 256 Nov 17 02:22 MacOSX10.15.sdk drwxr-xr-x 7 root wheel 224 Oct 19 23:39 MacOSX11.0.sdk % xcrun --show-sdk-path /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk % xcrun --sdk macosx --show-sdk-path /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk Oh, that's weird! Nevertheless I like you suggestion to call "xcrun" from "configure". Adding "--verbose" doesn't really explain anything, but just in case. % xcrun --verbose --no-cache --find cc xcrun: note: PATH = '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin' xcrun: note: SDKROOT = '/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk' xcrun: note: TOOLCHAINS = '' xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer' xcrun: note: XCODE_DEVELOPER_USR_PATH = '' xcrun: note: xcrun_db = '/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db' xcrun: note: xcrun via cc (xcrun) xcrun: note: database key is: cc|/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk||/Applications/Xcode.app/Contents/Developer| xcrun: note: looking up with '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk macosx -find cc 2> /dev/null' xcrun: note: lookup resolved with 'xcodebuild -find' to '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc' /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc % xcrun --verbose --no-cache --sdk macosx --find cc xcrun: note: looking up SDK with '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk macosx -version Path' xcrun: note: PATH = '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin' xcrun: note: SDKROOT = 'macosx' xcrun: note: TOOLCHAINS = '' xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer' xcrun: note: XCODE_DEVELOPER_USR_PATH = '' xcrun: note: xcrun_db = '/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db' xcrun: note: lookup resolved to: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' xcrun: note: looking up SDK with '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -version PlatformPath' xcrun: note: PATH = '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin' xcrun: note: SDKROOT = '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' xcrun: note: TOOLCHAINS = '' xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer' xcrun: note: XCODE_DEVELOPER_USR_PATH = '' xcrun: note: xcrun_db = '/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db' xcrun: note: lookup resolved to: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform' xcrun: note: PATH = '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin' xcrun: note: SDKROOT = '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk' xcrun: note: TOOLCHAINS = '' xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer' xcrun: note: XCODE_DEVELOPER_USR_PATH = '' xcrun: note: xcrun_db = '/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db' xcrun: note: xcrun via cc (xcrun) xcrun: note: database key is: cc|/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk||/Applications/Xcode.app/Contents/Developer| xcrun: note: looking up with '/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -find cc 2> /dev/null' xcrun: note: lookup resolved with 'xcodebuild -find' to '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc' /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc