public inbox for [email protected]
help / color / mirror / Atom feedConnection slots reserved for replication
7+ messages / 5 participants
[nested] [flat]
* Connection slots reserved for replication
@ 2018-08-01 12:30 Alexander Kukushkin <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Alexander Kukushkin @ 2018-08-01 12:30 UTC (permalink / raw)
To: pgsql-hackers
Hello hackers,
at the moment it is possible to reserve some amount of connection slots for
superusers and this behavior is controlled by
superuser_reserved_connections configuration parameter with the default
value = 3.
In case if all non-reserved connection slots are busy, replica fails to
open a new connection and start streaming from the primary. Such behavior
is very bad if you want to run postgresql HA clusters
Initially, replication connections required superuser privileges (in 9.0)
and therefore they were deliberately excluded from
superuser_reserved_connections.
Basically that means it has never been possible to reserve come connection
slots for replication connections.
Later (9.1) it became possible to create a user with REPLICATION and
NOSUPERUSER options, but comment in the postinit.c still tells that
superuser is required.
Now I think now it is a time to go further, and we should make it possible
to reserve some connection slots for replication in a manner similar to
superuser connections.
How should it work:
1. If we know that we got the replication connection, we just should make
sure that there are at least superuser_reserved_connections free connection
slots are available.
2. If we know that this is neither superuser nor replication connection, we
should check that there are at least (superuser_reserved_connections +
NumWalSenders() - max_wal_senders) connection slots are available.
And the last question how to control the number of reserved slots for
replication. There are two options:
1. We can introduce a new GUC for that: replication_reserved_connections
2. Or we can just use the value of max_wal_senders
Personally, I more like the second option.
Attached patch implements above described functionality.
Feedback is very appretiated.
Regards,
--
Alexander Kukushkin
Attachments:
[text/x-patch] replication_reserved_connections.patch (4.1K, ../../CAFh8B=nBzHQeYAu0b8fjK-AF1X4+_p6GRtwG+cCgs6Vci2uRuQ@mail.gmail.com/3-replication_reserved_connections.patch)
download | inline diff:
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index d60026d..13caeef 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -2273,6 +2273,10 @@ InitWalSenderSlot(void)
walsnd->applyLag = -1;
walsnd->state = WALSNDSTATE_STARTUP;
walsnd->latch = &MyProc->procLatch;
+
+ /* increment the number of allocated wal sender slots */
+ pg_atomic_fetch_add_u32(&WalSndCtl->num_wal_senders, 1);
+
SpinLockRelease(&walsnd->mutex);
/* don't need the lock anymore */
MyWalSnd = (WalSnd *) walsnd;
@@ -2306,6 +2310,10 @@ WalSndKill(int code, Datum arg)
walsnd->latch = NULL;
/* Mark WalSnd struct as no longer being in use. */
walsnd->pid = 0;
+
+ /* decrement the number of allocated wal sender slots */
+ pg_atomic_fetch_sub_u32(&WalSndCtl->num_wal_senders, 1);
+
SpinLockRelease(&walsnd->mutex);
}
@@ -3022,6 +3030,7 @@ WalSndShmemInit(void)
{
/* First time through, so initialize */
MemSet(WalSndCtl, 0, WalSndShmemSize());
+ pg_atomic_init_u32(&WalSndCtl->num_wal_senders, 0);
for (i = 0; i < NUM_SYNC_REP_WAIT_MODE; i++)
SHMQueueInit(&(WalSndCtl->SyncRepQueue[i]));
@@ -3576,3 +3585,9 @@ LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now)
Assert(time != 0);
return now - time;
}
+
+/* Return the amount of allocated wal_sender slots */
+uint32 NumWalSenders(void)
+{
+ return pg_atomic_read_u32(&WalSndCtl->num_wal_senders);
+}
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 5ef6315..18392c1 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -789,17 +789,25 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
}
/*
- * The last few connection slots are reserved for superusers. Although
- * replication connections currently require superuser privileges, we
- * don't allow them to consume the reserved slots, which are intended for
- * interactive use.
+ * The last few connection slots are reserved for superusers and replication.
+ * Superusers always have a priority over replication connections.
*/
- if ((!am_superuser || am_walsender) &&
- ReservedBackends > 0 &&
- !HaveNFreeProcs(ReservedBackends))
- ereport(FATAL,
- (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("remaining connection slots are reserved for non-replication superuser connections")));
+ if (am_walsender)
+ {
+ if (ReservedBackends > 0 && !HaveNFreeProcs(ReservedBackends))
+ ereport(FATAL,
+ (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
+ errmsg("remaining connection slots are reserved for non-replication superuser connections")));
+ }
+ else if (!am_superuser)
+ {
+ uint32 n = ReservedBackends + max_wal_senders - NumWalSenders();
+
+ if (n > 0 && !HaveNFreeProcs(n))
+ ereport(FATAL,
+ (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
+ errmsg("remaining connection slots are reserved for replication or superuser connections")));
+ }
/* Check replication permissions needed for walsender processes. */
if (am_walsender)
diff --git a/src/include/replication/walsender.h b/src/include/replication/walsender.h
index 45b72a7..9ebcb57 100644
--- a/src/include/replication/walsender.h
+++ b/src/include/replication/walsender.h
@@ -37,6 +37,7 @@ extern int max_wal_senders;
extern int wal_sender_timeout;
extern bool log_replication_commands;
+extern uint32 NumWalSenders(void);
extern void InitWalSender(void);
extern bool exec_replication_command(const char *query_string);
extern void WalSndErrorCleanup(void);
diff --git a/src/include/replication/walsender_private.h b/src/include/replication/walsender_private.h
index 4b90477..3ef8f4d 100644
--- a/src/include/replication/walsender_private.h
+++ b/src/include/replication/walsender_private.h
@@ -14,6 +14,7 @@
#include "access/xlog.h"
#include "nodes/nodes.h"
+#include "port/atomics.h"
#include "replication/syncrep.h"
#include "storage/latch.h"
#include "storage/shmem.h"
@@ -101,6 +102,8 @@ typedef struct
*/
bool sync_standbys_defined;
+ pg_atomic_uint32 num_wal_senders;
+
WalSnd walsnds[FLEXIBLE_ARRAY_MEMBER];
} WalSndCtlData;
^ permalink raw reply [nested|flat] 7+ messages in thread
* [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI
@ 2022-02-21 11:46 Melih Mutlu <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Melih Mutlu @ 2022-02-21 11:46 UTC (permalink / raw)
---
.cirrus.yml | 78 ++++++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 65 insertions(+), 13 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index f23d6cae552..1a06cdcaadb 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -338,13 +338,29 @@ task:
cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores"
+WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
+ env:
+ # Half the allowed per-user CPU cores
+ CPUS: 4
+ # The default working dir is in a directory msbuild complains about
+ CIRRUS_WORKING_DIR: "c:/cirrus"
+
+ # Avoids port conflicts between concurrent tap test runs
+ PG_TEST_USE_UNIX_SOCKETS: 1
+
+ only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
+
+ sysinfo_script: |
+ chcp
+ systeminfo
+ powershell -Command get-psdrive -psprovider filesystem
+ set
+
task:
+ << : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, VS 2019
env:
- # Half the allowed per-user CPU cores
- CPUS: 4
-
# Our windows infrastructure doesn't have test concurrency above the level
# of a single vcregress test target. Due to that, it's useful to run prove
# with multiple jobs. For the other tasks it isn't, because two sources
@@ -354,15 +370,11 @@ task:
# likely can be improved upon further.
PROVE_FLAGS: -j10 --timer
- # The default cirrus working dir is in a directory msbuild complains about
- CIRRUS_WORKING_DIR: "c:/cirrus"
# Avoid re-installing over and over
NO_TEMP_INSTALL: 1
# git's tar doesn't deal with drive letters, see
# https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
TAR: "c:/windows/system32/tar.exe"
- # Avoids port conflicts between concurrent tap test runs
- PG_TEST_USE_UNIX_SOCKETS: 1
PG_REGRESS_SOCK_DIR: "c:/cirrus/"
# -m enables parallelism
# verbosity:minimal + Summary reduce verbosity, while keeping a summary of
@@ -393,12 +405,6 @@ task:
cpu: $CPUS
memory: 4G
- sysinfo_script: |
- chcp
- systeminfo
- powershell -Command get-psdrive -psprovider filesystem
- set
-
setup_additional_packages_script: |
REM choco install -y --no-progress ...
@@ -456,6 +462,52 @@ task:
path: "crashlog-*.txt"
type: text/plain
+task:
+ << : *WINDOWS_ENVIRONMENT_BASE
+ name: Windows - Server 2019, MinGW64
+ windows_container:
+ image: $CONTAINER_REPO/windows_ci_mingw64:latest
+ cpu: $CPUS
+ memory: 4G
+ env:
+ CCACHE_DIR: C:/msys64/ccache
+ BUILD_DIR: "%CIRRUS_WORKING_DIR%/build"
+
+ ccache_cache:
+ folder: ${CCACHE_DIR}
+
+ mingw_info_script:
+ - C:\msys64\usr\bin\dash.exe -lc "where gcc"
+ - C:\msys64\usr\bin\dash.exe -lc "gcc --version"
+ - C:\msys64\usr\bin\dash.exe -lc "where perl"
+ - C:\msys64\usr\bin\dash.exe -lc "perl --version"
+
+ configure_script:
+ - C:\msys64\usr\bin\dash.exe -lc "mkdir %BUILD_DIR% &&
+ cd %BUILD_DIR% &&
+ %CIRRUS_WORKING_DIR%/configure
+ --enable-cassert
+ --enable-tap-tests
+ --with-icu
+ --with-libxml
+ --with-libxslt
+ --with-lz4
+ --enable-debug
+ CC='ccache gcc'
+ CXX='ccache g++'
+ CFLAGS='-Og -ggdb -pipe'
+ CXXFLAGS='-Og -ggdb'"
+
+ build_script:
+ C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s world-bin -j${CPUS}"
+
+ upload_caches: ccache
+
+ tests_script:
+ - set "NoDefaultCurrentDirectoryInExePath=0"
+ - C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s ${CHECK} ${CHECKFLAGS} -j${CPUS} TMPDIR=%BUILD_DIR%/tmp_install"
+
+ on_failure: *on_failure
task:
name: CompilerWarnings
--
2.17.1
--Ca0e2zgpnh8/XhnM
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0002-f.patch"
^ permalink raw reply [nested|flat] 7+ messages in thread
* [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI
@ 2022-02-21 11:46 Melih Mutlu <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Melih Mutlu @ 2022-02-21 11:46 UTC (permalink / raw)
---
.cirrus.yml | 78 ++++++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 65 insertions(+), 13 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index a9193c2c34f..472661c0936 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -339,13 +339,29 @@ task:
cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores"
+WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
+ env:
+ # Half the allowed per-user CPU cores
+ CPUS: 4
+ # The default working dir is in a directory msbuild complains about
+ CIRRUS_WORKING_DIR: "c:/cirrus"
+
+ # Avoids port conflicts between concurrent tap test runs
+ PG_TEST_USE_UNIX_SOCKETS: 1
+
+ only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
+
+ sysinfo_script: |
+ chcp
+ systeminfo
+ powershell -Command get-psdrive -psprovider filesystem
+ set
+
task:
+ << : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, VS 2019
env:
- # Half the allowed per-user CPU cores
- CPUS: 4
-
# Our windows infrastructure doesn't have test concurrency above the level
# of a single vcregress test target. Due to that, it's useful to run prove
# with multiple jobs. For the other tasks it isn't, because two sources
@@ -355,15 +371,11 @@ task:
# likely can be improved upon further.
PROVE_FLAGS: -j10 --timer
- # The default cirrus working dir is in a directory msbuild complains about
- CIRRUS_WORKING_DIR: "c:/cirrus"
# Avoid re-installing over and over
NO_TEMP_INSTALL: 1
# git's tar doesn't deal with drive letters, see
# https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
TAR: "c:/windows/system32/tar.exe"
- # Avoids port conflicts between concurrent tap test runs
- PG_TEST_USE_UNIX_SOCKETS: 1
PG_REGRESS_SOCK_DIR: "c:/cirrus/"
# -m enables parallelism
# verbosity:minimal + Summary reduce verbosity, while keeping a summary of
@@ -398,12 +410,6 @@ task:
cpu: $CPUS
memory: 4G
- sysinfo_script: |
- chcp
- systeminfo
- powershell -Command get-psdrive -psprovider filesystem
- set
-
setup_additional_packages_script: |
REM 3min
REM choco install -y --no-progress --version=1.0.0 visualstudio2022-workload-vctools --install-args="--add Microsoft.VisualStudio.Component.VC.CLI.Support"
@@ -475,6 +481,52 @@ task:
path: "crashlog-*.txt"
type: text/plain
+task:
+ << : *WINDOWS_ENVIRONMENT_BASE
+ name: Windows - Server 2019, MinGW64
+ windows_container:
+ image: $CONTAINER_REPO/windows_ci_mingw64:latest
+ cpu: $CPUS
+ memory: 4G
+ env:
+ CCACHE_DIR: C:/msys64/ccache
+ BUILD_DIR: "%CIRRUS_WORKING_DIR%/build"
+
+ ccache_cache:
+ folder: ${CCACHE_DIR}
+
+ mingw_info_script:
+ - C:\msys64\usr\bin\dash.exe -lc "where gcc"
+ - C:\msys64\usr\bin\dash.exe -lc "gcc --version"
+ - C:\msys64\usr\bin\dash.exe -lc "where perl"
+ - C:\msys64\usr\bin\dash.exe -lc "perl --version"
+
+ configure_script:
+ - C:\msys64\usr\bin\dash.exe -lc "mkdir %BUILD_DIR% &&
+ cd %BUILD_DIR% &&
+ %CIRRUS_WORKING_DIR%/configure
+ --enable-cassert
+ --enable-tap-tests
+ --with-icu
+ --with-libxml
+ --with-libxslt
+ --with-lz4
+ --enable-debug
+ CC='ccache gcc'
+ CXX='ccache g++'
+ CFLAGS='-Og -ggdb -pipe'
+ CXXFLAGS='-Og -ggdb'"
+
+ build_script:
+ C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s world-bin -j${CPUS}"
+
+ upload_caches: ccache
+
+ tests_script:
+ - set "NoDefaultCurrentDirectoryInExePath=0"
+ - C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s ${CHECK} ${CHECKFLAGS} -j${CPUS} TMPDIR=%BUILD_DIR%/tmp_install"
+
+ on_failure: *on_failure
task:
name: CompilerWarnings
--
2.17.1
--lildS9pRFgpM/xzO
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0002-f.patch"
^ permalink raw reply [nested|flat] 7+ messages in thread
* [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI
@ 2022-09-02 20:10 Melih Mutlu <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Melih Mutlu @ 2022-09-02 20:10 UTC (permalink / raw)
CI task on Windows Server with MinGW has been added as optional.
---
.cirrus.yml | 103 ++++++++++++++++++++++++++------
src/tools/ci/cores_backtrace.sh | 17 +++++-
2 files changed, 100 insertions(+), 20 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index a9193c2c34f..60f9ae2b65c 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -34,6 +34,7 @@ on_failure: &on_failure
- "**/*.log"
- "**/*.diffs"
- "**/regress_log_*"
+ - "**/*.stackdump"
type: text/plain
task:
@@ -339,13 +340,33 @@ task:
cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores"
+WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
+ env:
+ # Half the allowed per-user CPU cores
+ CPUS: 4
+ # The default working dir is in a directory msbuild complains about
+ CIRRUS_WORKING_DIR: "c:/cirrus"
+
+ # Avoids port conflicts between concurrent tap test runs
+ PG_TEST_USE_UNIX_SOCKETS: 1
+
+ # git's tar doesn't deal with drive letters, see
+ # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
+ TAR: "c:/windows/system32/tar.exe"
+
+ sysinfo_script: |
+ chcp
+ systeminfo
+ powershell -Command get-psdrive -psprovider filesystem
+ set
+
task:
+ << : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, VS 2019
- env:
- # Half the allowed per-user CPU cores
- CPUS: 4
+ only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
+ env:
# Our windows infrastructure doesn't have test concurrency above the level
# of a single vcregress test target. Due to that, it's useful to run prove
# with multiple jobs. For the other tasks it isn't, because two sources
@@ -355,15 +376,9 @@ task:
# likely can be improved upon further.
PROVE_FLAGS: -j10 --timer
- # The default cirrus working dir is in a directory msbuild complains about
- CIRRUS_WORKING_DIR: "c:/cirrus"
# Avoid re-installing over and over
NO_TEMP_INSTALL: 1
- # git's tar doesn't deal with drive letters, see
- # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
- TAR: "c:/windows/system32/tar.exe"
- # Avoids port conflicts between concurrent tap test runs
- PG_TEST_USE_UNIX_SOCKETS: 1
+
PG_REGRESS_SOCK_DIR: "c:/cirrus/"
# -m enables parallelism
# verbosity:minimal + Summary reduce verbosity, while keeping a summary of
@@ -387,8 +402,6 @@ task:
# currently have a tool for that...
CIRRUS_ESCAPING_PROCESSES: 1
- only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
-
windows_container:
#image: $CONTAINER_REPO/windows_ci_vs_2019:latest
#image: cirrusci/windowsservercore:visualstudio2019-2021.12.07
@@ -398,12 +411,6 @@ task:
cpu: $CPUS
memory: 4G
- sysinfo_script: |
- chcp
- systeminfo
- powershell -Command get-psdrive -psprovider filesystem
- set
-
setup_additional_packages_script: |
REM 3min
REM choco install -y --no-progress --version=1.0.0 visualstudio2022-workload-vctools --install-args="--add Microsoft.VisualStudio.Component.VC.CLI.Support"
@@ -475,6 +482,66 @@ task:
path: "crashlog-*.txt"
type: text/plain
+task:
+ << : *WINDOWS_ENVIRONMENT_BASE
+ name: Windows - Server 2019, MinGW64
+
+ only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-include:[^\n]*mingw.* || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*mingw.*'
+
+ windows_container:
+ image: $CONTAINER_REPO/windows_ci_mingw64:latest
+ cpu: $CPUS
+ memory: 4G
+ env:
+ CCACHE_DIR: C:/msys64/ccache
+ BUILD_DIR: "%CIRRUS_WORKING_DIR%/build"
+
+ ccache_cache:
+ folder: ${CCACHE_DIR}
+
+ setup_additional_packages_script: |
+ REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ...
+
+ mingw_info_script:
+ - C:\msys64\usr\bin\dash.exe -lc "where gcc"
+ - C:\msys64\usr\bin\dash.exe -lc "gcc --version"
+ - C:\msys64\usr\bin\dash.exe -lc "where perl"
+ - C:\msys64\usr\bin\dash.exe -lc "perl --version"
+
+ configure_script:
+ # Try to configure with the cache file, and retry without if it fails, in case the flags changed.
+ - C:\msys64\usr\bin\dash.exe -lc "mkdir %BUILD_DIR% &&
+ cd %BUILD_DIR% && for i in 1 2; do
+ %CIRRUS_WORKING_DIR%/configure
+ --cache-file=${CCACHE_DIR}/configure.cache
+ --enable-cassert
+ --enable-debug
+ --enable-tap-tests
+ --with-icu
+ --with-libxml
+ --with-libxslt
+ --with-lz4
+ CC='ccache gcc'
+ CXX='ccache g++'
+ CFLAGS='-Og -ggdb -pipe'
+ CXXFLAGS='-Og -ggdb' && break;
+ rm -v ${CCACHE_DIR}/configure.cache;
+ done
+ "
+
+ build_script:
+ C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s world-bin -j${CPUS}"
+
+ upload_caches: ccache
+
+ tests_script:
+ - set "NoDefaultCurrentDirectoryInExePath=0"
+ - C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s ${CHECK} ${CHECKFLAGS} -j${CPUS} TMPDIR=%BUILD_DIR%/tmp_install"
+
+ on_failure:
+ <<: *on_failure
+ cores_script:
+ - C:\tools\cygwin\bin\dash.exe --login -c "cd '%cd%' && src/tools/ci/cores_backtrace.sh msys ."
task:
name: CompilerWarnings
diff --git a/src/tools/ci/cores_backtrace.sh b/src/tools/ci/cores_backtrace.sh
index 28d3cecfc67..93e90f284e6 100755
--- a/src/tools/ci/cores_backtrace.sh
+++ b/src/tools/ci/cores_backtrace.sh
@@ -10,11 +10,24 @@ directory=$2
case $os in
freebsd|linux|macos)
- ;;
+ ;;
+
+ msys)
+ # XXX Evidently I don't know how to write two arguments here without pathname expansion later, other than eval.
+ #findargs='-name "*.stackdump"'
+ for corefile in $(find "$directory" -type f -name "*.stackdump") ; do
+ binary=`basename "$corefile" .stackdump`
+ echo;echo;
+ echo "dumping ${corefile} for ${binary}"
+ awk '/^0/{print $2}' $corefile |addr2line -f -i -e ./src/backend/postgres.exe
+ done
+ exit 0
+ ;;
+
*)
echo "unsupported operating system ${os}"
exit 1
- ;;
+ ;;
esac
first=1
--
2.17.1
--TcuvTDpCAASXpu1W
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0002-f.patch"
^ permalink raw reply [nested|flat] 7+ messages in thread
* [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI
@ 2022-09-02 20:10 Melih Mutlu <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Melih Mutlu @ 2022-09-02 20:10 UTC (permalink / raw)
CI task on Windows Server with MinGW has been added as optional.
---
.cirrus.yml | 103 ++++++++++++++++++++++++++------
src/tools/ci/cores_backtrace.sh | 17 +++++-
2 files changed, 100 insertions(+), 20 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index a9193c2c34f..60f9ae2b65c 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -34,6 +34,7 @@ on_failure: &on_failure
- "**/*.log"
- "**/*.diffs"
- "**/regress_log_*"
+ - "**/*.stackdump"
type: text/plain
task:
@@ -339,13 +340,33 @@ task:
cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores"
+WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
+ env:
+ # Half the allowed per-user CPU cores
+ CPUS: 4
+ # The default working dir is in a directory msbuild complains about
+ CIRRUS_WORKING_DIR: "c:/cirrus"
+
+ # Avoids port conflicts between concurrent tap test runs
+ PG_TEST_USE_UNIX_SOCKETS: 1
+
+ # git's tar doesn't deal with drive letters, see
+ # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
+ TAR: "c:/windows/system32/tar.exe"
+
+ sysinfo_script: |
+ chcp
+ systeminfo
+ powershell -Command get-psdrive -psprovider filesystem
+ set
+
task:
+ << : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, VS 2019
- env:
- # Half the allowed per-user CPU cores
- CPUS: 4
+ only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
+ env:
# Our windows infrastructure doesn't have test concurrency above the level
# of a single vcregress test target. Due to that, it's useful to run prove
# with multiple jobs. For the other tasks it isn't, because two sources
@@ -355,15 +376,9 @@ task:
# likely can be improved upon further.
PROVE_FLAGS: -j10 --timer
- # The default cirrus working dir is in a directory msbuild complains about
- CIRRUS_WORKING_DIR: "c:/cirrus"
# Avoid re-installing over and over
NO_TEMP_INSTALL: 1
- # git's tar doesn't deal with drive letters, see
- # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
- TAR: "c:/windows/system32/tar.exe"
- # Avoids port conflicts between concurrent tap test runs
- PG_TEST_USE_UNIX_SOCKETS: 1
+
PG_REGRESS_SOCK_DIR: "c:/cirrus/"
# -m enables parallelism
# verbosity:minimal + Summary reduce verbosity, while keeping a summary of
@@ -387,8 +402,6 @@ task:
# currently have a tool for that...
CIRRUS_ESCAPING_PROCESSES: 1
- only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
-
windows_container:
#image: $CONTAINER_REPO/windows_ci_vs_2019:latest
#image: cirrusci/windowsservercore:visualstudio2019-2021.12.07
@@ -398,12 +411,6 @@ task:
cpu: $CPUS
memory: 4G
- sysinfo_script: |
- chcp
- systeminfo
- powershell -Command get-psdrive -psprovider filesystem
- set
-
setup_additional_packages_script: |
REM 3min
REM choco install -y --no-progress --version=1.0.0 visualstudio2022-workload-vctools --install-args="--add Microsoft.VisualStudio.Component.VC.CLI.Support"
@@ -475,6 +482,66 @@ task:
path: "crashlog-*.txt"
type: text/plain
+task:
+ << : *WINDOWS_ENVIRONMENT_BASE
+ name: Windows - Server 2019, MinGW64
+
+ only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-include:[^\n]*mingw.* || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*mingw.*'
+
+ windows_container:
+ image: $CONTAINER_REPO/windows_ci_mingw64:latest
+ cpu: $CPUS
+ memory: 4G
+ env:
+ CCACHE_DIR: C:/msys64/ccache
+ BUILD_DIR: "%CIRRUS_WORKING_DIR%/build"
+
+ ccache_cache:
+ folder: ${CCACHE_DIR}
+
+ setup_additional_packages_script: |
+ REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ...
+
+ mingw_info_script:
+ - C:\msys64\usr\bin\dash.exe -lc "where gcc"
+ - C:\msys64\usr\bin\dash.exe -lc "gcc --version"
+ - C:\msys64\usr\bin\dash.exe -lc "where perl"
+ - C:\msys64\usr\bin\dash.exe -lc "perl --version"
+
+ configure_script:
+ # Try to configure with the cache file, and retry without if it fails, in case the flags changed.
+ - C:\msys64\usr\bin\dash.exe -lc "mkdir %BUILD_DIR% &&
+ cd %BUILD_DIR% && for i in 1 2; do
+ %CIRRUS_WORKING_DIR%/configure
+ --cache-file=${CCACHE_DIR}/configure.cache
+ --enable-cassert
+ --enable-debug
+ --enable-tap-tests
+ --with-icu
+ --with-libxml
+ --with-libxslt
+ --with-lz4
+ CC='ccache gcc'
+ CXX='ccache g++'
+ CFLAGS='-Og -ggdb -pipe'
+ CXXFLAGS='-Og -ggdb' && break;
+ rm -v ${CCACHE_DIR}/configure.cache;
+ done
+ "
+
+ build_script:
+ C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s world-bin -j${CPUS}"
+
+ upload_caches: ccache
+
+ tests_script:
+ - set "NoDefaultCurrentDirectoryInExePath=0"
+ - C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s ${CHECK} ${CHECKFLAGS} -j${CPUS} TMPDIR=%BUILD_DIR%/tmp_install"
+
+ on_failure:
+ <<: *on_failure
+ cores_script:
+ - C:\tools\cygwin\bin\dash.exe --login -c "cd '%cd%' && src/tools/ci/cores_backtrace.sh msys ."
task:
name: CompilerWarnings
diff --git a/src/tools/ci/cores_backtrace.sh b/src/tools/ci/cores_backtrace.sh
index 28d3cecfc67..93e90f284e6 100755
--- a/src/tools/ci/cores_backtrace.sh
+++ b/src/tools/ci/cores_backtrace.sh
@@ -10,11 +10,24 @@ directory=$2
case $os in
freebsd|linux|macos)
- ;;
+ ;;
+
+ msys)
+ # XXX Evidently I don't know how to write two arguments here without pathname expansion later, other than eval.
+ #findargs='-name "*.stackdump"'
+ for corefile in $(find "$directory" -type f -name "*.stackdump") ; do
+ binary=`basename "$corefile" .stackdump`
+ echo;echo;
+ echo "dumping ${corefile} for ${binary}"
+ awk '/^0/{print $2}' $corefile |addr2line -f -i -e ./src/backend/postgres.exe
+ done
+ exit 0
+ ;;
+
*)
echo "unsupported operating system ${os}"
exit 1
- ;;
+ ;;
esac
first=1
--
2.17.1
--TcuvTDpCAASXpu1W
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="0002-f.patch"
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: "collation" or "collation oder"
@ 2024-12-15 07:58 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Tatsuo Ishii @ 2024-12-15 07:58 UTC (permalink / raw)
To: [email protected]; +Cc: pgsql-hackers
> I think there is a difference. A collation is a database object, a
> collation order is the order produced when invoking that collation.
> For example, in amcheck.sgml:
>
> """
> ... an inconsistency in the collation order between a primary server
> and a standby server ...
> """
>
> If we wrote just "collation" here I think it would be less clear,
> because what's the problem is whether the order is different, not
> whether the collation objects are different.
That makes sense. Thanks for the explanation.
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: improve performance of pg_dump with many sequences
@ 2025-12-29 17:26 Tom Lane <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Tom Lane @ 2025-12-29 17:26 UTC (permalink / raw)
To: Nathan Bossart <[email protected]>; +Cc: Michael Paquier <[email protected]>; Euler Taveira <[email protected]>; pgsql-hackers
Nathan Bossart <[email protected]> writes:
> Committed.
In the no-good-deed-goes-unpunished department: pg_dump's use
of pg_get_sequence_data() (nee pg_sequence_read_tuple()) is
evidently responsible for the complaint in bug #19365 [1]
that pg_dump can no longer survive concurrent sequence drops.
Given that that function already silently returns NULLs if the
sequence isn't readable for other reasons, I think it'd be
sane to make it silently return NULL if the sequence isn't
there anymore. Unfortunately, that looks like it'd require
nontrivial restructuring of init_sequence().
Or maybe we could make it not use init_sequence()? For the moment
a plain try_relation_open and check that it's a sequence should do,
but I'm not sure how that'd fit into people's plans for future
improvement of the sequence API.
There are other reasons not to like use of init_sequence in this
code path, too. pg_dump's session will build a SeqTable entry for
every sequence in the database, which there could be a lot of,
and it will acquire RowExclusiveLock on every sequence and hold
that to the end of the dump, which seems likely to be troublesome
from a concurrency standpoint. Since pg_get_sequence_data is a
read-only operation this lock level feels wrong.
BTW, I'm unconvinced that pg_dump behaves sanely when this function
does return nulls. I think the ideal thing would be for it to skip
issuing setval(), but right now it looks like it will issue one with
garbage values.
regards, tom lane
[1] https://www.postgresql.org/message-id/19365-6245240d8b926327%40postgresql.org
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2025-12-29 17:26 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-08-01 12:30 Connection slots reserved for replication Alexander Kukushkin <[email protected]>
2022-02-21 11:46 [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI Melih Mutlu <[email protected]>
2022-02-21 11:46 [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI Melih Mutlu <[email protected]>
2022-09-02 20:10 [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI Melih Mutlu <[email protected]>
2022-09-02 20:10 [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI Melih Mutlu <[email protected]>
2024-12-15 07:58 Re: "collation" or "collation oder" Tatsuo Ishii <[email protected]>
2025-12-29 17:26 Re: improve performance of pg_dump with many sequences Tom Lane <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox