public inbox for [email protected]help / color / mirror / Atom feed
Cygwin cleanup 21+ messages / 5 participants [nested] [flat]
* Cygwin cleanup @ 2022-07-26 04:24 Thomas Munro <[email protected]> 0 siblings, 2 replies; 21+ messages in thread From: Thomas Munro @ 2022-07-26 04:24 UTC (permalink / raw) To: PostgreSQL Hackers <[email protected]>; +Cc: Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> Hi, Continuing a discussion started over at [1]. Moving this to a new thread so that other thread can focus on Unix cleanup, and both threads can get CI coverage... 1. In a few places, it is alleged that both __CYGWIN__ and WIN32 might be defined at the same time. Do you think we should try to get rid of that possibility? I understand that we have to have a few tests for __CYGWIN__ here and there, because eg file permissions don't work quite right and there's not much we can do about that. But it seems a bit unhelpful if we also have to worry about a more-or-less POSIX-ish build taking WIN32 paths at uncertain times if we forget to defend against that, or wonder why some places are not consistent. A quick recap of the three flavours of Windows platform we have to handle, as I understand it: * MSVC: Windowsy toolchain, Windowsy C * custom perl scripts instead of configure * msbuild instead of make * MSVC compiler * Windows C APIs * we provide our own emulation of some POSIX C APIs on top * MSYS: Unixy toolchain, Windowsy C * configure (portname = "win32") * make * GCC compiler * Windows C APIs * we provide our own emulation of some POSIX C APIs on top * Cygwin: Unixy toolchain, Unixy C * configure (portname = "cygwin") * make * GCC compiler * POSIX C APIs (emulations provided by the Cygwin runtime libraries) (The configure/make part will be harmonised by the Meson project.) The macro WIN32 is visibly defined by something in/near msbuild in MSVC builds: /D WIN32 is right here in the build transcripts (whereas the compiler defines _WIN32; good compiler). I am not sure how exactly it is first defined in MSYS builds; I suspect that MSYS gcc might define it itself, but I don't have access to MSYS to check. As for Cygwin, the only translation unit where I could find both __CYGWIN__ and WIN32 defined is dirmod.c, and that results from including <windows.h> and ultimately <minwindef.h> (even though WIN32 isn't defined yet at that time). I couldn't understand why we do that, but I probably didn't read enough commit history. The purpose of dirmod.c on Cygwin today is only to wrap otherwise pure POSIX code in retry loops to handle those spurious EACCES errors due to NT sharing violations, so there is no need for that. Proposal: let's make it a programming rule that we don't allow definitions from Windows headers to leak into Cygwin translation units, preferably by never including them, or if we really must, let's grant specific exemptions in an isolated and well documented way. We don't seem to need any such exemptions currently. Places where we currently worry about the contradictory macros could become conditional #error directives instead. 2. To make it possible to test any of that, you either need a working Windows+Cygwin setup, or working CI. I'm a salty old Unix hacker so I opted for the latter, and I also hope this will eventually be useful to others. Unfortunately I haven't figured out how to get everything working yet, so some of the check-world tests are failing. Clues most welcome! The version I'm posting here is set to run always, so that cfbot will show it alongside others. But I would imagine that if we got a committable-quality version of this, it'd probably be opt-in, so you'd have to say "ci-os-only: cygwin", or "ci-os-only: cygwin, windows" etc in a commit to your private github account to ask for it (or maybe we'll come up with a way to tell cfbot we want the full works of CI checks; the same decision will come up for MSYS, OpenBSD and NetBSD CI support that my colleague is working on). There are other things to fix too, including abysmal performance; see commit message. 3. You can't really run PostgreSQL on Cygwin for real, because its implementation of signals does not have reliable signal masking, so unsubtle and probably also subtle breakage occurs. That was reported upstream by Noah years ago, but they aren't working on a fix. lorikeet shows random failures, and presumably any CI system will do the same... I even wondered about putting our own magic entry/exit macros into signal handlers, that would use atomics to implement a second level of signal masking (?!) but that's an uncommonly large bandaid for a defective platform... and trying to fix Cygwin itself is a rabbithole too far for me. 4. When building with Cygwin GCC 11.3 you get a bunch of warnings that don't show up on other platforms, seemingly indicating that it interprets -Wimplicit-fallthrough=3 differently. Huh? [1] https://www.postgresql.org/message-id/CA%2BhUKGKZ_FjkBnjGADk%2Bpa2g4oKDcG8%3DSE5V23sPTP0EELfyzQ%40ma... Attachments: [text/x-patch] 0001-WIP-CI-support-for-Cygwin.patch (2.9K, ../../CA+hUKG+e13wK0PBX5Z63CCwWm7MfRQuwBRabM_3aKWSko2AUww@mail.gmail.com/2-0001-WIP-CI-support-for-Cygwin.patch) download | inline diff: From 8f300f5b804d5fd2268709d40e31b52c86d6799c Mon Sep 17 00:00:00 2001 From: Thomas Munro <[email protected]> Date: Mon, 25 Jul 2022 23:05:10 +1200 Subject: [PATCH 1/2] WIP CI support for Cygwin. XXX Doesn't get all the way through yet... XXX Needs some --with-X options XXX This should use a canned Docker image with all the right packages installed XXX We would never want this to run by default in CI, but it'd be nice to be able to ask for it with ci-os-only! (See commented out line) XXX configure is soooo slooow, can we cache it?! Compiling is also insanely slow, but ccache gets it down to a couple of minutes if you're lucky XXX I don't know how to put variables like BUILD_JOBS into the scripts XXX I have no idea if crash dump works, and if this should share elements with the msys work in commitfest #3575 --- .cirrus.yml | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/.cirrus.yml b/.cirrus.yml index f23d6cae55..b5238f5f52 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -456,6 +456,57 @@ task: path: "crashlog-*.txt" type: text/plain +task: + name: Windows - Cygwin + + env: + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 8 + + #only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' + + windows_container: + image: cirrusci/windowsservercore:2019 + os_version: 2019 + cpu: $CPUS + memory: 4G + + ccache_cache: + folder: C:\tools\cygwin\tmp\ccache + + sysinfo_script: | + chcp + systeminfo + powershell -Command get-psdrive -psprovider filesystem + set + + setup_additional_packages_script: | + choco install -y --no-progress cygwin + C:\tools\cygwin\cygwinsetup.exe -q -P cygrunsrv,make,gcc-core,ccache,binutils,libtool,pkg-config,flex,bison,zlib-devel,libssl-devel,libreadline-devel,perl,perl-IPC-Run + C:\tools\cygwin\bin\bash.exe --login -c "cygserver-config -y" || EXIT /b 1 + C:\tools\cygwin\bin\bash.exe --login -c "echo 'kern.ipc.semmni 1024' >> /etc/cygserver.conf" || EXIT /b 1 + C:\tools\cygwin\bin\bash.exe --login -c "echo 'kern.ipc.semmns 1024' >> /etc/cygserver.conf" || EXIT /b 1 + C:\tools\cygwin\bin\bash.exe --login -c "net start cygserver" || EXIT /b 1 + + configure_script: + - C:\tools\cygwin\bin\bash.exe --login -c "cd '%cd%' && CCACHE_DIR=/tmp/ccache ./configure --enable-cassert --enable-debug --enable-tap-tests CC='ccache gcc'" || EXIT /b 1 + + build_script: + - C:\tools\cygwin\bin\bash.exe --login -c "cd '%cd%' && CCACHE_DIR=/tmp/ccache make -s -j4 world-bin" || EXIT /b 1 + + test_world_script: + - C:\tools\cygwin\bin\bash.exe --login -c "cd '%cd%' && CCACHE_DIR=/tmp/ccache make -s -j1 check-world -Otarget" || EXIT /b 1 + + on_failure: + <<: *on_failure + crashlog_artifacts: + path: "crashlog-*.txt" + type: text/plain + + always: + upload_caches: ccache + task: name: CompilerWarnings -- 2.35.1 [text/x-patch] 0002-WIP-Do-not-pollute-Cygwin-namespace-with-Windows-hea.patch (6.3K, ../../CA+hUKG+e13wK0PBX5Z63CCwWm7MfRQuwBRabM_3aKWSko2AUww@mail.gmail.com/3-0002-WIP-Do-not-pollute-Cygwin-namespace-with-Windows-hea.patch) download | inline diff: From ea47a8af0332876629b53620788d40bcb7b1e96c Mon Sep 17 00:00:00 2001 From: Thomas Munro <[email protected]> Date: Tue, 26 Jul 2022 12:56:06 +1200 Subject: [PATCH 2/2] WIP Do not pollute Cygwin namespace with Windows headers. Establish that __CYGWIN__ and WIN32 should generally not be defined at the same time. If ever it's necessary to access Windows APIs directly when building for Cygwin, that should be done in an extremely localized way, not allowing Windows macros and declarations to leak into other translation units. dirmod.c initially looked like a potential case for a localized exemption, but on closer inspection it doesn't currently have any reason to include <windows.h> -- so don't. In passing, remove anachronistic comments about ancient Windows versions. --- src/include/pg_config_manual.h | 2 +- src/include/port.h | 17 +++++++------ src/port/dirmod.c | 46 +++++++++++++++++----------------- 3 files changed, 33 insertions(+), 32 deletions(-) diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h index 5ee2c46267..0d1d5d7c9b 100644 --- a/src/include/pg_config_manual.h +++ b/src/include/pg_config_manual.h @@ -148,7 +148,7 @@ * fork()). On other platforms, it's only useful for verifying those * otherwise Windows-specific code paths. */ -#if defined(WIN32) && !defined(__CYGWIN__) +#if defined(WIN32) #define EXEC_BACKEND #endif diff --git a/src/include/port.h b/src/include/port.h index d39b04141f..8cf1112a54 100644 --- a/src/include/port.h +++ b/src/include/port.h @@ -18,10 +18,12 @@ /* * Windows has enough specialized port stuff that we push most of it off * into another file. - * Note: Some CYGWIN includes might #define WIN32. */ -#if defined(WIN32) && !defined(__CYGWIN__) +#if defined(WIN32) #include "port/win32_port.h" +#if defined(__CYGWIN__) +#error "__CYGWIN__ should not be defined at the same time as WIN32" +#endif #endif /* socket has a different definition on WIN32 */ @@ -150,7 +152,7 @@ extern int pg_disable_aslr(void); #define EXE "" #endif -#if defined(WIN32) && !defined(__CYGWIN__) +#if defined(WIN32) #define DEVNULL "nul" #else #define DEVNULL "/dev/null" @@ -276,12 +278,11 @@ extern int pgunlink(const char *path); * Win32 also doesn't have symlinks, but we can emulate them with * junction points on newer Win32 versions. * - * Cygwin has its own symlinks which work on Win95/98/ME where - * junction points don't, so use those instead. We have no way of - * knowing what type of system Cygwin binaries will be run on. - * Note: Some CYGWIN includes might #define WIN32. + * Cygwin has its own symlinks that work where junction points don't, so use + * those instead. We have no way of knowing what type of system Cygwin + * binaries will be run on. */ -#if defined(WIN32) && !defined(__CYGWIN__) +#if defined(WIN32) extern int pgsymlink(const char *oldpath, const char *newpath); extern int pgreadlink(const char *path, char *buf, size_t size); extern bool pgwin32_is_junction(const char *path); diff --git a/src/port/dirmod.c b/src/port/dirmod.c index 7ce042e75d..1bfbead098 100644 --- a/src/port/dirmod.c +++ b/src/port/dirmod.c @@ -6,8 +6,10 @@ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * - * This includes replacement versions of functions that work on - * Win32 (NT4 and newer). + * This includes replacement versions of functions that work on Windows. + * For Cygwin, the purpose of these replacements is to provide retry loops + * around POSIX functions. For native Windows, we also redirect to + * native Windows APIs. * * IDENTIFICATION * src/port/dirmod.c @@ -21,25 +23,24 @@ #include "postgres_fe.h" #endif +#if defined(WIN32) && defined(__CYGWIN__) +#error "WIN32 should not be defined at the same time as __CYGWIN__" +#endif + +#if !defined(WIN32) && !defined(__CYGWIN__) +#error "one of WIN32 or __CYGWIN__ is expected" +#endif + /* Don't modify declarations in system headers */ -#if defined(WIN32) || defined(__CYGWIN__) #undef rename #undef unlink -#endif #include <unistd.h> #include <sys/stat.h> -#if defined(WIN32) || defined(__CYGWIN__) -#ifndef __CYGWIN__ +#if defined(WIN32) #include <winioctl.h> -#else -#include <windows.h> -#include <w32api/winioctl.h> #endif -#endif - -#if defined(WIN32) || defined(__CYGWIN__) /* * pgrename @@ -56,24 +57,24 @@ pgrename(const char *from, const char *to) * someone else to close the file, as the caller might be holding locks * and blocking other backends. */ -#if defined(WIN32) && !defined(__CYGWIN__) +#if defined(WIN32) while (!MoveFileEx(from, to, MOVEFILE_REPLACE_EXISTING)) #else while (rename(from, to) < 0) #endif { -#if defined(WIN32) && !defined(__CYGWIN__) +#if defined(WIN32) DWORD err = GetLastError(); _dosmaperr(err); /* - * Modern NT-based Windows versions return ERROR_SHARING_VIOLATION if - * another process has the file open without FILE_SHARE_DELETE. - * ERROR_LOCK_VIOLATION has also been seen with some anti-virus - * software. This used to check for just ERROR_ACCESS_DENIED, so - * presumably you can get that too with some OS versions. We don't - * expect real permission errors where we currently use rename(). + * Windows returns ERROR_SHARING_VIOLATION if another process has the + * file open without FILE_SHARE_DELETE. ERROR_LOCK_VIOLATION has also + * been seen with some anti-virus software. This used to check for + * just ERROR_ACCESS_DENIED, so presumably you can get that too with + * some OS versions. We don't expect real permission errors where we + * currently use rename(). */ if (err != ERROR_ACCESS_DENIED && err != ERROR_SHARING_VIOLATION && @@ -121,10 +122,9 @@ pgunlink(const char *path) /* We undefined these above; now redefine for possible use below */ #define rename(from, to) pgrename(from, to) #define unlink(path) pgunlink(path) -#endif /* defined(WIN32) || defined(__CYGWIN__) */ -#if defined(WIN32) && !defined(__CYGWIN__) /* Cygwin has its own symlinks */ +#if defined(WIN32) /* * pgsymlink support: @@ -352,4 +352,4 @@ pgwin32_is_junction(const char *path) } return ((attr & FILE_ATTRIBUTE_REPARSE_POINT) == FILE_ATTRIBUTE_REPARSE_POINT); } -#endif /* defined(WIN32) && !defined(__CYGWIN__) */ +#endif /* defined(WIN32) */ -- 2.35.1 ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-26 04:34 Tom Lane <[email protected]> parent: Thomas Munro <[email protected]> 1 sibling, 1 reply; 21+ messages in thread From: Tom Lane @ 2022-07-26 04:34 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]> Thomas Munro <[email protected]> writes: > 3. You can't really run PostgreSQL on Cygwin for real, because its > implementation of signals does not have reliable signal masking, so > unsubtle and probably also subtle breakage occurs. That was reported > upstream by Noah years ago, but they aren't working on a fix. > lorikeet shows random failures, and presumably any CI system will do > the same... If that's an accurate statement, shouldn't we just drop Cygwin support? Now that we have a native Windows build, it's hard to see how any live user would prefer to use the Cygwin build. regards, tom lane ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-26 05:16 Thomas Munro <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Thomas Munro @ 2022-07-26 05:16 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]> On Tue, Jul 26, 2022 at 4:34 PM Tom Lane <[email protected]> wrote: > Thomas Munro <[email protected]> writes: > > 3. You can't really run PostgreSQL on Cygwin for real, because its > > implementation of signals does not have reliable signal masking, so > > unsubtle and probably also subtle breakage occurs. That was reported > > upstream by Noah years ago, but they aren't working on a fix. > > lorikeet shows random failures, and presumably any CI system will do > > the same... > > If that's an accurate statement, shouldn't we just drop Cygwin support? This thread rejected the idea last time around: https://www.postgresql.org/message-id/flat/136712b0-0619-5619-4634-0f0286acaef7%402ndQuadrant.com lorikeet still shows the issue. Failures often involve assertions about PMSignalState or mq->mq_sender. Hmm, it's running Cygwin 3.2.0 (March 2021) and the latest release is 3.3.5, so it's remotely possible that it's been fixed recently. Maybe that'd be somewhere in here, but it's not jumping out: https://github.com/cygwin/cygwin/commits/master/winsup/cygwin/signal.cc (Oooh, another implementation of signalfd...) ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-26 11:40 Tom Lane <[email protected]> parent: Thomas Munro <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Tom Lane @ 2022-07-26 11:40 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]> Thomas Munro <[email protected]> writes: > On Tue, Jul 26, 2022 at 4:34 PM Tom Lane <[email protected]> wrote: >> If that's an accurate statement, shouldn't we just drop Cygwin support? > This thread rejected the idea last time around: > https://www.postgresql.org/message-id/flat/136712b0-0619-5619-4634-0f0286acaef7%402ndQuadrant.com I think maybe we should re-open the discussion. I've certainly reached the stage of fed-up-ness. That platform seems seriously broken, upstream is making no progress on fixing it, and there doesn't seem to be any real-world use-case. The only positive argument for it is that Readline doesn't work in the other Windows builds --- but we've apparently not rechecked that statement in eighteen years, so maybe things are better now. If we could just continue to blithely ignore lorikeet's failures, I wouldn't mind so much; but doing any significant amount of new code development work for the platform seems like throwing away developer time. regards, tom lane ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-26 17:08 Robert Haas <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Robert Haas @ 2022-07-26 17:08 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Thomas Munro <[email protected]>; PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]> On Tue, Jul 26, 2022 at 7:40 AM Tom Lane <[email protected]> wrote: > I think maybe we should re-open the discussion. I've certainly > reached the stage of fed-up-ness. That platform seems seriously > broken, upstream is making no progress on fixing it, and there > doesn't seem to be any real-world use-case. The only positive > argument for it is that Readline doesn't work in the other > Windows builds --- but we've apparently not rechecked that > statement in eighteen years, so maybe things are better now. > > If we could just continue to blithely ignore lorikeet's failures, > I wouldn't mind so much; but doing any significant amount of new > code development work for the platform seems like throwing away > developer time. I agree with that. All things being equal, I like the idea of supporting a bunch of different platforms, and Cygwin doesn't really look that dead. It has recent releases. But if blocking signals doesn't actually work on that platform, making PostgreSQL work reliably there seems really difficult. -- Robert Haas EDB: http://www.enterprisedb.com ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-27 06:44 Justin Pryzby <[email protected]> parent: Thomas Munro <[email protected]> 1 sibling, 1 reply; 21+ messages in thread From: Justin Pryzby @ 2022-07-27 06:44 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: [email protected]; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> On Tue, Jul 26, 2022 at 04:24:25PM +1200, Thomas Munro wrote: > 3. You can't really run PostgreSQL on Cygwin for real, because its > implementation of signals does not have reliable signal masking, so > unsubtle and probably also subtle breakage occurs. That was reported > upstream by Noah years ago, but they aren't working on a fix. > lorikeet shows random failures, and presumably any CI system will do > the same... Reference: https://www.postgresql.org/message-id/20170321034703.GB2097809%40tornado.leadboat.com On my 2nd try: https://cirrus-ci.com/task/5311911574110208 TRAP: FailedAssertion("mq->mq_sender == NULL", File: "shm_mq.c", Line: 230, PID: 16370) 2022-07-26 06:32:35.525 PDT [15538][postmaster] LOG: background worker "parallel worker" (PID 16370) was terminated by signal 6: Aborted > XXX Doesn't get all the way through yet... Mainly because getopt was causing all tap tests to fail. I tried to fix that in configure, but ended up changing the callers. This is getting close, but I don't think has actually managed to pass all tests yet.. https://cirrus-ci.com/task/5274721116749824 > 4. When building with Cygwin GCC 11.3 you get a bunch of warnings > that don't show up on other platforms, seemingly indicating that it > interprets -Wimplicit-fallthrough=3 differently. Huh? Evidently due to the same getopt issues. > XXX This should use a canned Docker image with all the right packages > installed Has anyone tried using non-canned images ? It sounds like this could reduce the 4min startup time for windows. https://cirrus-ci.org/guide/docker-builder-vm/#dockerfile-as-a-ci-environment > XXX configure is soooo slooow, can we cache it?! Compiling is also > insanely slow, but ccache gets it down to a couple of minutes if you're > lucky One reason compiling was slow is because you ended up with -O2. You can cache configure as long as you're willing to re-run it whenever options were changed. That also applies to the existing headerscheck. > XXX I don't know how to put variables like BUILD_JOBS into the scripts WDYM ? If it's outside of bash and in windows shell it's like %var%, right ? https://cirrus-ci.org/guide/writing-tasks/#environment-variables I just noticed that cirrus is misbehaving: if there's a variable called CI (which there is), then it expands $CI_FOO like ${CI}_FOO rather than ${CI_FOO}. I've also seen weirdness when variable names or operators appear in the commit message... > XXX Needs some --with-X options Done > XXX We would never want this to run by default in CI, but it'd be nice > to be able to ask for it with ci-os-only! (See commented out line) > only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' Doesn't this already do what's needed? As long as it doesn't also check: CHANGE_MESSAGE !~ 'ci-os-only', the task will runs only on request. > XXX I have no idea if crash dump works, and if this should share > elements with the msys work in commitfest #3575 Based on the crash above, it wasn't working. And after some changes ... it still doesn't work. windows_os is probably skipping too many things. -- Justin ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-28 22:04 Thomas Munro <[email protected]> parent: Justin Pryzby <[email protected]> 0 siblings, 2 replies; 21+ messages in thread From: Thomas Munro @ 2022-07-28 22:04 UTC (permalink / raw) To: Justin Pryzby <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> On Wed, Jul 27, 2022 at 6:44 PM Justin Pryzby <[email protected]> wrote: > On Tue, Jul 26, 2022 at 04:24:25PM +1200, Thomas Munro wrote: > > 3. You can't really run PostgreSQL on Cygwin for real, because its > > implementation of signals does not have reliable signal masking, so > > unsubtle and probably also subtle breakage occurs. That was reported > > upstream by Noah years ago, but they aren't working on a fix. > > lorikeet shows random failures, and presumably any CI system will do > > the same... > > Reference: https://www.postgresql.org/message-id/20170321034703.GB2097809%40tornado.leadboat.com > > On my 2nd try: > > https://cirrus-ci.com/task/5311911574110208 > TRAP: FailedAssertion("mq->mq_sender == NULL", File: "shm_mq.c", Line: 230, PID: 16370) > 2022-07-26 06:32:35.525 PDT [15538][postmaster] LOG: background worker "parallel worker" (PID 16370) was terminated by signal 6: Aborted Thanks for working on this! Huh, that Cygwin being shipped by Choco is quite old, older than lorikeet's, but not old enough to not have the bug: [04:33:55.234] Starting cygwin install, version 2.918 Based on clues in Noah's emails in the archives, I think versions from maybe somewhere around 2015 didn't have the bug, and then the bug appeared, and AFAIK it's still here. I wonder if you can tell Choco to install an ancient version, but even if that's possible you'd be dealing with other stupid problems and bugs. > > XXX Doesn't get all the way through yet... > > Mainly because getopt was causing all tap tests to fail. > I tried to fix that in configure, but ended up changing the callers. > > This is getting close, but I don't think has actually managed to pass all tests > yet.. https://cirrus-ci.com/task/5274721116749824 Woo. > > 4. When building with Cygwin GCC 11.3 you get a bunch of warnings > > that don't show up on other platforms, seemingly indicating that it > > interprets -Wimplicit-fallthrough=3 differently. Huh? > > Evidently due to the same getopt issues. Ahh, nice detective work. > > XXX This should use a canned Docker image with all the right packages > > installed > > Has anyone tried using non-canned images ? It sounds like this could reduce > the 4min startup time for windows. > > https://cirrus-ci.org/guide/docker-builder-vm/#dockerfile-as-a-ci-environment Yeah, I had that working once. Not sure what the pros and cons would be for us. > > XXX configure is soooo slooow, can we cache it?! Compiling is also > > insanely slow, but ccache gets it down to a couple of minutes if you're > > lucky > > One reason compiling was slow is because you ended up with -O2. Ah, right. > You can cache configure as long as you're willing to re-run it whenever options > were changed. That also applies to the existing headerscheck. > > > XXX I don't know how to put variables like BUILD_JOBS into the scripts > > WDYM ? If it's outside of bash and in windows shell it's like %var%, right ? > https://cirrus-ci.org/guide/writing-tasks/#environment-variables Right. I should have taken the clue from the %cd% (I got a few ideas about how to do this from libarchive's CI scripting[1]). > I just noticed that cirrus is misbehaving: if there's a variable called CI > (which there is), then it expands $CI_FOO like ${CI}_FOO rather than ${CI_FOO}. > I've also seen weirdness when variable names or operators appear in the commit > message... > > > XXX Needs some --with-X options > > Done Neat. > > XXX We would never want this to run by default in CI, but it'd be nice > > to be able to ask for it with ci-os-only! (See commented out line) > > only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' > > Doesn't this already do what's needed? > As long as it doesn't also check: CHANGE_MESSAGE !~ 'ci-os-only', > the task will runs only on request. Yeah I was just trying to say that I was sharing the script in a way that always runs, but for commit we'd want that. This is all far too slow for cfbot to have to deal with on every build. Looks like we can expect to be able to build and test fast on Windows soonish, though, so maybe one day we'd just turn Cygwin and MSYS on? [1] https://github.com/libarchive/libarchive/blob/master/build/ci/cirrus_ci/ci.cmd ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-28 22:23 Justin Pryzby <[email protected]> parent: Thomas Munro <[email protected]> 1 sibling, 2 replies; 21+ messages in thread From: Justin Pryzby @ 2022-07-28 22:23 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: [email protected]; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> On Fri, Jul 29, 2022 at 10:04:04AM +1200, Thomas Munro wrote: > Thanks for working on this! > > Huh, that Cygwin being shipped by Choco is quite old, older than > lorikeet's, but not old enough to not have the bug: > > [04:33:55.234] Starting cygwin install, version 2.918 Hm, I think that's the version of "cygwinsetup" but not cygwin.. It also says this: [13:16:36.014] Cygwin v3.3.4.20220408 [Approved] > I wonder if you can tell Choco > to install an ancient version, but even if that's possible you'd be > dealing with other stupid problems and bugs. Yes: choco install -y --no-progress --version 4.6.1 ccache > > > XXX This should use a canned Docker image with all the right packages > > > installed > > > > Has anyone tried using non-canned images ? It sounds like this could reduce > > the 4min startup time for windows. > > > > https://cirrus-ci.org/guide/docker-builder-vm/#dockerfile-as-a-ci-environment > > Yeah, I had that working once. Not sure what the pros and cons would be for us. I think it could be a lot faster to start, since cirrus caches the generated docker image locally. Rather than (I gather) pulling the image every time. > > > XXX We would never want this to run by default in CI, but it'd be nice > > > to be able to ask for it with ci-os-only! (See commented out line) > > > only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' > > > > Doesn't this already do what's needed? > > As long as it doesn't also check: CHANGE_MESSAGE !~ 'ci-os-only', > > the task will runs only on request. > > Yeah I was just trying to say that I was sharing the script in a way > that always runs, but for commit we'd want that. This is all far too > slow for cfbot to have to deal with on every build. It occurred to me today that if cfbot preserved the original patch series, and commit messages, that would allow patch authors to write things like "ci-os-only: docs" for a doc only patch. I've never gotten cirrus' changesOnly() stuff to work... > Looks like we can expect to be able to build and test fast on Windows > soonish, though, Do you mean with meson ? > so maybe one day we'd just turn Cygwin and MSYS on? I didn't understand this ? -- Justin ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-28 22:29 Thomas Munro <[email protected]> parent: Justin Pryzby <[email protected]> 1 sibling, 0 replies; 21+ messages in thread From: Thomas Munro @ 2022-07-28 22:29 UTC (permalink / raw) To: Justin Pryzby <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> On Fri, Jul 29, 2022 at 10:23 AM Justin Pryzby <[email protected]> wrote: > On Fri, Jul 29, 2022 at 10:04:04AM +1200, Thomas Munro wrote: > > [04:33:55.234] Starting cygwin install, version 2.918 > > Hm, I think that's the version of "cygwinsetup" but not cygwin.. > It also says this: [13:16:36.014] Cygwin v3.3.4.20220408 [Approved] Oops. Ok so we're testing the very latest then, and it definitely still has the bug as we thought. > It occurred to me today that if cfbot preserved the original patch series, and > commit messages, that would allow patch authors to write things like > "ci-os-only: docs" for a doc only patch. I've never gotten cirrus' > changesOnly() stuff to work... Maybe it's time to switch to "git am -3 ..." and reject patches that don't apply that way. > > Looks like we can expect to be able to build and test fast on Windows > > soonish, though, > > Do you mean with meson ? Yeah. Also there are some other things we can do to speed up testing on Windows (and elsewhere), like not running every test query with new psql + backend process pair, which takes at least a few hundred ms and sometimes up to several seconds on this platform; I have some patches I need to finish... > > so maybe one day we'd just turn Cygwin and MSYS on? > > I didn't understand this ? I mean, if, some sunny day, we can compile and test on Windows at non-glacial speeds, then it would become possible to contemplate having cfbot run these tasks for every patch every time. ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-07-28 22:57 Thomas Munro <[email protected]> parent: Robert Haas <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Thomas Munro @ 2022-07-28 22:57 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]> On Wed, Jul 27, 2022 at 5:09 AM Robert Haas <[email protected]> wrote: > On Tue, Jul 26, 2022 at 7:40 AM Tom Lane <[email protected]> wrote: > > I think maybe we should re-open the discussion. I've certainly > > reached the stage of fed-up-ness. That platform seems seriously > > broken, upstream is making no progress on fixing it, and there > > doesn't seem to be any real-world use-case. The only positive > > argument for it is that Readline doesn't work in the other > > Windows builds --- but we've apparently not rechecked that > > statement in eighteen years, so maybe things are better now. > > > > If we could just continue to blithely ignore lorikeet's failures, > > I wouldn't mind so much; but doing any significant amount of new > > code development work for the platform seems like throwing away > > developer time. > > I agree with that. All things being equal, I like the idea of > supporting a bunch of different platforms, and Cygwin doesn't really > look that dead. It has recent releases. But if blocking signals > doesn't actually work on that platform, making PostgreSQL work > reliably there seems really difficult. It's one thing to drop old dead Unixes but I don't think anyone would enjoy dropping support for an active open source project. The best outcome would be for people who have an interest in seeing PostgreSQL work correctly on Cygwin to help get the bug fixed. Here are the threads I'm aware of: https://cygwin.com/pipermail/cygwin/2017-August/234001.html https://cygwin.com/pipermail/cygwin/2017-August/234097.html I wonder if these problems would go away as a nice incidental side-effect if we used latches for postmaster wakeups. I don't know... maybe, if the problem is just with the postmaster's pattern of blocking/unblocking? Maybe backend startup is simple enough that it doesn't hit the bug? From a quick glance, I think the assertion failures that occur in regular backends can possibly be blamed on the postmaster getting confused about its children due to unexpected handler re-entry. ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-08-04 03:38 Justin Pryzby <[email protected]> parent: Thomas Munro <[email protected]> 1 sibling, 1 reply; 21+ messages in thread From: Justin Pryzby @ 2022-08-04 03:38 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: [email protected]; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> On Fri, Jul 29, 2022 at 10:04:04AM +1200, Thomas Munro wrote: > > > XXX We would never want this to run by default in CI, but it'd be nice > > > to be able to ask for it with ci-os-only! (See commented out line) > > > only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' > > > > Doesn't this already do what's needed? > > As long as it doesn't also check: CHANGE_MESSAGE !~ 'ci-os-only', > > the task will runs only on request. > > Yeah I was just trying to say that I was sharing the script in a way > that always runs, but for commit we'd want that. That makes more sense after noticing that you created a cf entry (for which cfbot has been skipping my patch due to my "only_if" line). There's still a few persistent issues: This fails ~50% of the time in recovery 010-truncate I hacked around this by setting data_sync_retry. https://cirrus-ci.com/task/5289444063313920 I found these, not sure if they're relevant. https://www.postgresql.org/message-id/flat/CAA4eK1Kft05mwNuZbTVRmz8SNS3r%2BuriuCT8DxL5KJy5btoS-A%40m... https://www.postgresql.org/message-id/flat/CAFiTN-uGxgo5258hZy2QJoz%3Ds7_Cs7v9%3Db8Z2GgFV7qmQUOwxw%4... And an fsync abort in 013 which seems similar to this other one. data_sync_retry also avoids this issue. https://cirrus-ci.com/task/6283023745286144?logs=cores#L34 https://www.postgresql.org/message-id/flat/CAMVYW_4QhjZ-19Xpr2x1B19soRCNu1BXHM8g1mOnAVtd5VViDw%40mai... And sometimes various assertions failing in regress parallel_select (and then times out) https://api.cirrus-ci.com/v1/artifact/task/5537540282253312/log/src/test/regress/log/postmaster.log https://api.cirrus-ci.com/v1/artifact/task/6108746773430272/log/src/test/regress/log/postmaster.log Or "could not map dynamic shared memory segment" (actually in 027-stream-regress): https://cirrus-ci.com/task/6168860746317824 And segfault in vacuum parallel https://api.cirrus-ci.com/v1/artifact/task/5404589569605632/log/src/test/regress/log/postmaster.log Sometimes semctl() failed: Resource temporarily unavailable https://api.cirrus-ci.com/v1/artifact/task/5027860623654912/log/src/test/subscription/tmp_check/log/... https://api.cirrus-ci.com/v1/artifact/task/5027860623654912/log/src/bin/pg_rewind/tmp_check/log/001_... Some more https://cirrus-ci.com/task/6468927780814848 If you're lucky, there's only 1 or 2 problems, of which those are different symptoms.. Maybe for now this needs to disable tap tests :( This shows that it *can* pass, if slowly, and infrequently: https://cirrus-ci.com/task/6546858536337408 This fixes my changes to configure for getopt. And simplifies the changes to *.pl (the .exe changes weren't necessary at all). And removes the changes for implicit-fallthrough; I realized that configure was just deciding that it didn't work and not using it at all. And adds support for backtraces. And remove kerberos and and add libxml Why did you write "|| exit /b 1" in all the bash invocations ? I think cirrus handles that automatically. -- Justin ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-08-04 04:06 Andres Freund <[email protected]> parent: Justin Pryzby <[email protected]> 1 sibling, 0 replies; 21+ messages in thread From: Andres Freund @ 2022-08-04 04:06 UTC (permalink / raw) To: Justin Pryzby <[email protected]>; +Cc: Thomas Munro <[email protected]>; [email protected]; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> Hi, On 2022-07-28 17:23:19 -0500, Justin Pryzby wrote: > On Fri, Jul 29, 2022 at 10:04:04AM +1200, Thomas Munro wrote: > > > > XXX This should use a canned Docker image with all the right packages > > > > installed > > > > > > Has anyone tried using non-canned images ? It sounds like this could reduce > > > the 4min startup time for windows. > > > > > > https://cirrus-ci.org/guide/docker-builder-vm/#dockerfile-as-a-ci-environment > > > > Yeah, I had that working once. Not sure what the pros and cons would be for us. > > I think it could be a lot faster to start, since cirrus caches the generated > docker image locally. Rather than (I gather) pulling the image every time. I'm quite certain that is not true. All the docker images built are just uploaded to the google container registry and then downloaded onto a *separate* windows host. The dockerfile: stuff generates a separate task running on a separate machine... It's a bit better for non-windows containers, because there google has some optimization for pulling image (pieces) on demand or such. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-08-04 04:16 Thomas Munro <[email protected]> parent: Justin Pryzby <[email protected]> 0 siblings, 3 replies; 21+ messages in thread From: Thomas Munro @ 2022-08-04 04:16 UTC (permalink / raw) To: Justin Pryzby <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> On Thu, Aug 4, 2022 at 3:38 PM Justin Pryzby <[email protected]> wrote: > [train wreck] Oh my, so I'm getting the impression we might actually be totally unstable on Cygwin. Which surprises me because ... wait a minute ... lorikeet isn't even running most of the tests. So... we don't really know the degree to which any of this works at all? > This shows that it *can* pass, if slowly, and infrequently: > https://cirrus-ci.com/task/6546858536337408 Ok, that's slightly reassuring, so maybe we *can* fix this, but I'm one step closer to what Tom said, re wasting developer time... > [lots of improvements] Cool. > Why did you write "|| exit /b 1" in all the bash invocations ? I think cirrus > handles that automatically. Cargo-culted from libarchive. ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-08-04 04:22 Andres Freund <[email protected]> parent: Thomas Munro <[email protected]> 2 siblings, 0 replies; 21+ messages in thread From: Andres Freund @ 2022-08-04 04:22 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: Justin Pryzby <[email protected]>; PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> Hi, On 2022-08-04 16:16:06 +1200, Thomas Munro wrote: > Ok, that's slightly reassuring, so maybe we *can* fix this, but I'm > one step closer to what Tom said, re wasting developer time... It might be worth checking whether the cygwin installer, which at some point at least allowed installing postgres, has download numbers available anywhere. It's possible we could e.g. get away with just allowing libpq to be built. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-08-04 05:19 Thomas Munro <[email protected]> parent: Thomas Munro <[email protected]> 2 siblings, 1 reply; 21+ messages in thread From: Thomas Munro @ 2022-08-04 05:19 UTC (permalink / raw) To: Justin Pryzby <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> On Thu, Aug 4, 2022 at 4:16 PM Thomas Munro <[email protected]> wrote: > On Thu, Aug 4, 2022 at 3:38 PM Justin Pryzby <[email protected]> wrote: > > [train wreck] > > Oh my, so I'm getting the impression we might actually be totally > unstable on Cygwin. Which surprises me because ... wait a minute ... > lorikeet isn't even running most of the tests. So... we don't really > know the degree to which any of this works at all? Hmm, it's possible that all these failures are just new-to-me effects of the known bug. Certainly the assertion failures are of the usual type, and I think it might be possible for the weird parallel query failure to be explained by the postmaster forking extra phantom child processes. It may be madness to try to work around this, but I wonder if we could use a static local variable that we update with atomic compare exhange, inside PG_SIGNAL_HANDLER_ENTRY(), and PG_SIGNAL_HANDLER_EXIT() macros that do nothing on every other system. On entry, if you can do 0->1 it means you are allowed to run the function. If it's non-zero, set n->n+1 and return immediately: signal blocked, but queued for later. On exit, you CAS n->0. If n was > 1, then you have to jump back to the top and run the function body again. ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-08-04 05:23 Tom Lane <[email protected]> parent: Thomas Munro <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Tom Lane @ 2022-08-04 05:23 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: Justin Pryzby <[email protected]>; PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]> Thomas Munro <[email protected]> writes: > It may be madness to try to work around this, but I wonder if we could > use a static local variable that we update with atomic compare > exhange, inside PG_SIGNAL_HANDLER_ENTRY(), and > PG_SIGNAL_HANDLER_EXIT() macros that do nothing on every other system. > On entry, if you can do 0->1 it means you are allowed to run the > function. If it's non-zero, set n->n+1 and return immediately: signal > blocked, but queued for later. On exit, you CAS n->0. If n was > 1, > then you have to jump back to the top and run the function body again. And ... we're expending all this effort for what exactly? regards, tom lane ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-08-04 06:07 Thomas Munro <[email protected]> parent: Tom Lane <[email protected]> 0 siblings, 0 replies; 21+ messages in thread From: Thomas Munro @ 2022-08-04 06:07 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Justin Pryzby <[email protected]>; PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]>; Noah Misch <[email protected]> On Thu, Aug 4, 2022 at 5:23 PM Tom Lane <[email protected]> wrote: > Thomas Munro <[email protected]> writes: > > It may be madness to try to work around this, but I wonder if we could > > use a static local variable that we update with atomic compare > > exhange, inside PG_SIGNAL_HANDLER_ENTRY(), and > > PG_SIGNAL_HANDLER_EXIT() macros that do nothing on every other system. > > On entry, if you can do 0->1 it means you are allowed to run the > > function. If it's non-zero, set n->n+1 and return immediately: signal > > blocked, but queued for later. On exit, you CAS n->0. If n was > 1, > > then you have to jump back to the top and run the function body again. > > And ... we're expending all this effort for what exactly? I'd be almost as happy if we ripped it all out, shut down lorikeet and added it to the list of fallen platforms. I'd feel a bit like a vandal, though. My suggestion is a last-ditch idea for Noah (CCd) and/or Andrew to consider, who (respectively) blocked this last time and run lorikeet. No plans to write that patch myself... ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-10-21 03:40 Justin Pryzby <[email protected]> parent: Thomas Munro <[email protected]> 2 siblings, 1 reply; 21+ messages in thread From: Justin Pryzby @ 2022-10-21 03:40 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: [email protected]; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]> On Thu, Aug 04, 2022 at 04:16:06PM +1200, Thomas Munro wrote: > On Thu, Aug 4, 2022 at 3:38 PM Justin Pryzby <[email protected]> wrote: > > [train wreck] > > Oh my, so I'm getting the impression we might actually be totally > unstable on Cygwin. Which surprises me because ... wait a minute ... > lorikeet isn't even running most of the tests. So... we don't really > know the degree to which any of this works at all? Right. Maybe it's of limited interest, but .. This updates the patch to build and test with meson. Which first requires patching some meson.builds. I guess that's needed for some current BF members, too. Unfortunately, ccache+PCH causes gcc to crash :( -- Justin ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-11-09 01:04 Justin Pryzby <[email protected]> parent: Justin Pryzby <[email protected]> 0 siblings, 1 reply; 21+ messages in thread From: Justin Pryzby @ 2022-11-09 01:04 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: [email protected]; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]>; Andres Freund <[email protected]> On Thu, Oct 20, 2022 at 10:40:40PM -0500, Justin Pryzby wrote: > On Thu, Aug 04, 2022 at 04:16:06PM +1200, Thomas Munro wrote: > > On Thu, Aug 4, 2022 at 3:38 PM Justin Pryzby <[email protected]> wrote: > > > [train wreck] > > > > Oh my, so I'm getting the impression we might actually be totally > > unstable on Cygwin. Which surprises me because ... wait a minute ... > > lorikeet isn't even running most of the tests. So... we don't really > > know the degree to which any of this works at all? > > Right. > > Maybe it's of limited interest, but .. > > This updates the patch to build and test with meson. > Which first requires patching some meson.builds. > I guess that's needed for some current BF members, too. > Unfortunately, ccache+PCH causes gcc to crash :( Resending with the 'only-if' line commented (doh). And some fixes to 001 as Andres pointed out by on other thread. -- Justin ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-12-06 19:31 Andres Freund <[email protected]> parent: Justin Pryzby <[email protected]> 0 siblings, 0 replies; 21+ messages in thread From: Andres Freund @ 2022-12-06 19:31 UTC (permalink / raw) To: Justin Pryzby <[email protected]>; +Cc: Thomas Munro <[email protected]>; [email protected]; Andrew Dunstan <[email protected]>; Tom Lane <[email protected]> Hi, On 2022-11-08 19:04:37 -0600, Justin Pryzby wrote: > From 2741472080eceac5cb6d002c39eaf319d7f72b50 Mon Sep 17 00:00:00 2001 > From: Justin Pryzby <[email protected]> > Date: Fri, 30 Sep 2022 13:39:43 -0500 > Subject: [PATCH 1/3] meson: other fixes for cygwin > > XXX: what about HAVE_BUGGY_STRTOF ? What about it? As noted in another thread, HAVE_BUGGY_STRTOF is defined in a header, and shouldn't be affected by the buildsystem. Pushed this commit. > XXX This should use a canned Docker image with all the right packages > installed? But if the larger image is slower to start, then maybe not... I think once we convert the windows containers to windows VMs we can just install both cygwin and mingw in the same image. The overhead of installing too much seems far far smaller there. > + CONFIGURE_FLAGS: --enable-cassert --enable-debug --with-ldap --with-ssl=openssl --with-libxml > + # --enable-tap-tests I assume this is disabled as tap tests fail? > + C:\tools\cygwin\bin\bash.exe --login -c "cygserver-config -y" I'd copy the approach used for mingw of putting most of this in an environment variable. > +findargs='' > case $os in > freebsd|linux|macos) > - ;; > + ;; > + > + cygwin) > + # XXX Evidently I don't know how to write two arguments here without pathname expansion later, other than eval. > + #findargs='-name "*.stackdump"' > + for stack in $(find "$directory" -type f -name "*.stackdump") ; do > + binary=`basename "$stack" .stackdump` > + echo;echo; > + echo "dumping ${stack} for ${binary}" > + awk '/^0/{print $2}' $stack |addr2line -f -i -e ./src/backend/postgres.exe > + #awk '/^0/{print $2}' $stack |addr2line -f -i -e "./src/backend/$binary.exe" > + done > + exit 0 > + ;; Is this stuff actually needed? Could we use the infrastructure we use for backtraces with msvc instead? Or use something that understands .stackdump files? > +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm > [...] > +++ b/src/test/perl/PostgreSQL/Test/Utils.pm > [...] > +++ b/src/test/recovery/t/020_archive_status.pl > [...] I think these should be in a separate commit, they're not actually about CI. Greetings, Andres Freund ^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Cygwin cleanup @ 2022-12-06 20:35 Thomas Munro <[email protected]> parent: Thomas Munro <[email protected]> 0 siblings, 0 replies; 21+ messages in thread From: Thomas Munro @ 2022-12-06 20:35 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; PostgreSQL Hackers <[email protected]>; Andrew Dunstan <[email protected]> On Fri, Jul 29, 2022 at 10:57 AM Thomas Munro <[email protected]> wrote: > I wonder if these problems would go away as a nice incidental > side-effect if we used latches for postmaster wakeups. I don't > know... maybe, if the problem is just with the postmaster's pattern of > blocking/unblocking? Maybe backend startup is simple enough that it > doesn't hit the bug? From a quick glance, I think the assertion > failures that occur in regular backends can possibly be blamed on the > postmaster getting confused about its children due to unexpected > handler re-entry. Just to connect the dots, that's what this patch does: https://www.postgresql.org/message-id/flat/CA+hUKG+Z-HpOj1JsO9eWUP+ar7npSVinsC_npxSy+jdOMsx=Gg@mail.... (There may be other places that break under Cygwin's flaky sa_mask implementation, I don't know and haven't seen any clues about that.) ^ permalink raw reply [nested|flat] 21+ messages in thread
end of thread, other threads:[~2022-12-06 20:35 UTC | newest] Thread overview: 21+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-07-26 04:24 Cygwin cleanup Thomas Munro <[email protected]> 2022-07-26 04:34 ` Tom Lane <[email protected]> 2022-07-26 05:16 ` Thomas Munro <[email protected]> 2022-07-26 11:40 ` Tom Lane <[email protected]> 2022-07-26 17:08 ` Robert Haas <[email protected]> 2022-07-28 22:57 ` Thomas Munro <[email protected]> 2022-12-06 20:35 ` Thomas Munro <[email protected]> 2022-07-27 06:44 ` Justin Pryzby <[email protected]> 2022-07-28 22:04 ` Thomas Munro <[email protected]> 2022-07-28 22:23 ` Justin Pryzby <[email protected]> 2022-07-28 22:29 ` Thomas Munro <[email protected]> 2022-08-04 04:06 ` Andres Freund <[email protected]> 2022-08-04 03:38 ` Justin Pryzby <[email protected]> 2022-08-04 04:16 ` Thomas Munro <[email protected]> 2022-08-04 04:22 ` Andres Freund <[email protected]> 2022-08-04 05:19 ` Thomas Munro <[email protected]> 2022-08-04 05:23 ` Tom Lane <[email protected]> 2022-08-04 06:07 ` Thomas Munro <[email protected]> 2022-10-21 03:40 ` Justin Pryzby <[email protected]> 2022-11-09 01:04 ` Justin Pryzby <[email protected]> 2022-12-06 19:31 ` Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox