agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH 2/8] cirrus/freebsd: run with more CPUs+RAM and do not repartition 11+ messages / 2 participants [nested] [flat]
* [PATCH 2/8] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 13213ffd304..21608ac2e0a 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -123,27 +123,25 @@ task: <<: *on_failure_meson cores_script: | mkdir -m 770 /tmp/cores find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; src/tools/ci/cores_backtrace.sh linux /tmp/cores task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST CFLAGS: -Og -ggdb depends_on: SanityCheck only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*freebsd.*' compute_engine_instance: image_project: $IMAGE_PROJECT image: family/pg-ci-freebsd-13 @@ -152,39 +150,36 @@ task: memory: 4G disk: 50 sysinfo_script: | id uname -a ulimit -a -H && ulimit -a -S export ccache_cache: folder: $CCACHE_DIR - # Work around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . mkdir -p ${CCACHE_DIR} chown -R postgres:postgres ${CCACHE_DIR} setup_core_files_script: | mkdir -m 770 /tmp/cores chown root:postgres /tmp/cores sysctl kern.corefile='/tmp/cores/%N.%P.core' setup_additional_packages_script: | #pkg install -y ... # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. + # large enough to make VM startup slow configure_script: | su postgres <<-EOF meson setup \ --buildtype=debug \ -Dcassert=true -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \ -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \ -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ build EOF build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS}' upload_caches: ccache -- 2.34.1 --NvorufYXl2Cnpa+A Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-cirrus-freebsd-define-ENFORCE_REGRESSION_TEST_NAME_R.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 2/8] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 60c0efc2e63..99276481e57 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -123,27 +123,25 @@ task: <<: *on_failure_meson cores_script: | mkdir -m 770 /tmp/cores find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; src/tools/ci/cores_backtrace.sh linux /tmp/cores task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST CFLAGS: -Og -ggdb depends_on: SanityCheck only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*freebsd.*' compute_engine_instance: image_project: $IMAGE_PROJECT image: family/pg-ci-freebsd-13 @@ -152,39 +150,36 @@ task: memory: 4G disk: 50 sysinfo_script: | id uname -a ulimit -a -H && ulimit -a -S export ccache_cache: folder: $CCACHE_DIR - # Work around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . mkdir -p ${CCACHE_DIR} chown -R postgres:postgres ${CCACHE_DIR} setup_core_files_script: | mkdir -m 770 /tmp/cores chown root:postgres /tmp/cores sysctl kern.corefile='/tmp/cores/%N.%P.core' setup_additional_packages_script: | #pkg install -y ... # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. + # large enough to make VM startup slow configure_script: | su postgres <<-EOF meson setup \ --buildtype=debug \ -Dcassert=true -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \ -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \ -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ build EOF build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS}' upload_caches: ccache -- 2.34.1 --Wo0n/IWMQc9EwEbt Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-cirrus-freebsd-define-ENFORCE_REGRESSION_TEST_NAME_R.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 2/9] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index eefc5c21fe6..0c12ca04fd9 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -131,11 +131,9 @@ task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST @@ -160,8 +158,6 @@ task: ccache_cache: folder: $CCACHE_DIR - # Work around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . @@ -175,8 +171,7 @@ task: #pkg install -y ... # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. + # large enough to make VM startup slow configure_script: | su postgres <<-EOF meson setup \ -- 2.25.1 --61jdw2sOBCFtR2d/ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0003-cirrus-freebsd-define-ENFORCE_REGRESSION_TEST_NAME_R.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 3/7] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 858454a3d08..2ba6b7cc2d8 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -131,11 +131,9 @@ task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST @@ -160,8 +158,6 @@ task: ccache_cache: folder: $CCACHE_DIR - # Work around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . -- 2.25.1 --O98KdSgI27dgYlM5 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0004-cirrus-freebsd-define-ENFORCE_REGRESSION_TEST_NAME_R.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 04/10] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 7c6e3691093..6aaaf62d6a9 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -131,11 +131,9 @@ task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST @@ -160,8 +158,6 @@ task: ccache_cache: folder: $CCACHE_DIR - # Workaround around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . -- 2.25.1 --sMkrXc3gAYLRVOjR Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0005-cirrus-clean-up-typos.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 10/11] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index aa381432050..3473d792561 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -63,11 +63,9 @@ task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST @@ -94,8 +92,6 @@ task: fingerprint_key: ccache/freebsd reupload_on_changes: true - # Workaround around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . -- 2.25.1 --I4VOKWutKNZEOIPu Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0011-cirrus-split-linux-and-move-the-only_if-line.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 10/11] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index aa381432050..3473d792561 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -63,11 +63,9 @@ task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST @@ -94,8 +92,6 @@ task: fingerprint_key: ccache/freebsd reupload_on_changes: true - # Workaround around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . -- 2.25.1 --I4VOKWutKNZEOIPu Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0011-cirrus-split-linux-and-move-the-only_if-line.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 04/10] cirrus/freebsd: run with more CPUs+RAM and do not repartition @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index 7c6e3691093..6aaaf62d6a9 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -131,11 +131,9 @@ task: name: FreeBSD - 13 - Meson env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST @@ -160,8 +158,6 @@ task: ccache_cache: folder: $CCACHE_DIR - # Workaround around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . -- 2.25.1 --sMkrXc3gAYLRVOjR Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0005-cirrus-clean-up-typos.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 07/23] cirrus/freebsd: run with more CPUs+RAM and do not repartitiion @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 //-os-only: freebsd --- .cirrus.yml | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index aa3746d0440..c5a7f4d8cc4 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -42,11 +42,9 @@ task: name: FreeBSD - 13 env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir @@ -57,7 +55,7 @@ task: image: family/pg-ci-freebsd-13 platform: freebsd cpu: $CPUS - memory: 2G + memory: 3G disk: 50 sysinfo_script: | @@ -71,8 +69,6 @@ task: fingerprint_key: ccache/freebsd reupload_on_changes: true - # Workaround around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . -- 2.17.1 --tKkaNMvYmhQvRCRK Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0008-cirrus-linux-compile-with-fsanitize.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH 08/25] cirrus/freebsd: run with more CPUs+RAM and do not repartitiion @ 2022-06-24 05:09 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 11+ messages in thread From: Justin Pryzby @ 2022-06-24 05:09 UTC (permalink / raw) There was some historic problem where tests under freebsd took 8+ minutes (and before 4a288a37f took 15 minutes). This reduces test time from 10min to 3min. 4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720 4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008 4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600 6 CPUs https://cirrus-ci.com/task/6678321684545536 8 CPUs https://cirrus-ci.com/task/6264854121021440 See also: https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4hxwp@alap3.anarazel.de#f36c0b1... 8 jobs 7min https://cirrus-ci.com/task/6186376667332608 xi-os-only: freebsd --- .cirrus.yml | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index ea6837d94f9..38cc25fcf7c 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -42,11 +42,9 @@ task: name: FreeBSD - 13 env: - # FreeBSD on GCP is slow when running with larger number of CPUS / - # jobs. Using one more job than cpus seems to work best. - CPUS: 2 - BUILD_JOBS: 3 - TEST_JOBS: 3 + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 6 CCACHE_DIR: /tmp/ccache_dir @@ -57,7 +55,7 @@ task: image: family/pg-ci-freebsd-13 platform: freebsd cpu: $CPUS - memory: 2G + memory: 3G disk: 50 sysinfo_script: | @@ -71,8 +69,6 @@ task: fingerprint_key: ccache/freebsd reupload_on_changes: true - # Workaround around performance issues due to 32KB block size - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh create_user_script: | pw useradd postgres chown -R postgres:postgres . -- 2.17.1 --IS0zKkzwUGydFO0o Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0009-cirrus-freebsd-run-build-check-in-a-make-vpath.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
* [PATCH v2 4/6] Introduce pending flag for GUC assign hooks @ 2025-02-19 16:45 Dmitrii Dolgov <9erthalion6@gmail.com> 0 siblings, 0 replies; 11+ messages in thread From: Dmitrii Dolgov @ 2025-02-19 16:45 UTC (permalink / raw) Currently an assing hook can perform some preprocessing of a new value, but it cannot change the behavior, which dictates that the new value will be applied immediately after the hook. Certain GUC options (like shared_buffers, coming in subsequent patches) may need coordinating work between backends to change, meaning we cannot apply it right away. Add a new flag "pending" for an assign hook to allow the hook indicate exactly that. If the pending flag is set after the hook, the new value will not be applied and it's handling becomes the hook's implementation responsibility. Note, that this also requires changes in the way how GUCs are getting reported, but the patch does not cover that yet. --- src/backend/access/transam/xlog.c | 2 +- src/backend/commands/variable.c | 2 +- src/backend/libpq/pqcomm.c | 8 ++-- src/backend/tcop/postgres.c | 2 +- src/backend/utils/misc/guc.c | 59 +++++++++++++++++++--------- src/backend/utils/misc/stack_depth.c | 2 +- src/include/utils/guc.h | 2 +- src/include/utils/guc_hooks.h | 16 ++++---- 8 files changed, 57 insertions(+), 36 deletions(-) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index f9bf5ba7509..ff82ba0a53d 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -2188,7 +2188,7 @@ CalculateCheckpointSegments(void) } void -assign_max_wal_size(int newval, void *extra) +assign_max_wal_size(int newval, void *extra, bool *pending) { max_wal_size_mb = newval; CalculateCheckpointSegments(); diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c index 4ad6e236d69..f24c2a0d252 100644 --- a/src/backend/commands/variable.c +++ b/src/backend/commands/variable.c @@ -1143,7 +1143,7 @@ check_cluster_name(char **newval, void **extra, GucSource source) * GUC assign_hook for maintenance_io_concurrency */ void -assign_maintenance_io_concurrency(int newval, void *extra) +assign_maintenance_io_concurrency(int newval, void *extra, bool *pending) { #ifdef USE_PREFETCH /* diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c index 61ea3722ae2..cdf21847d7e 100644 --- a/src/backend/libpq/pqcomm.c +++ b/src/backend/libpq/pqcomm.c @@ -1949,7 +1949,7 @@ pq_settcpusertimeout(int timeout, Port *port) * GUC assign_hook for tcp_keepalives_idle */ void -assign_tcp_keepalives_idle(int newval, void *extra) +assign_tcp_keepalives_idle(int newval, void *extra, bool *pending) { /* * The kernel API provides no way to test a value without setting it; and @@ -1982,7 +1982,7 @@ show_tcp_keepalives_idle(void) * GUC assign_hook for tcp_keepalives_interval */ void -assign_tcp_keepalives_interval(int newval, void *extra) +assign_tcp_keepalives_interval(int newval, void *extra, bool *pending) { /* See comments in assign_tcp_keepalives_idle */ (void) pq_setkeepalivesinterval(newval, MyProcPort); @@ -2005,7 +2005,7 @@ show_tcp_keepalives_interval(void) * GUC assign_hook for tcp_keepalives_count */ void -assign_tcp_keepalives_count(int newval, void *extra) +assign_tcp_keepalives_count(int newval, void *extra, bool *pending) { /* See comments in assign_tcp_keepalives_idle */ (void) pq_setkeepalivescount(newval, MyProcPort); @@ -2028,7 +2028,7 @@ show_tcp_keepalives_count(void) * GUC assign_hook for tcp_user_timeout */ void -assign_tcp_user_timeout(int newval, void *extra) +assign_tcp_user_timeout(int newval, void *extra, bool *pending) { /* See comments in assign_tcp_keepalives_idle */ (void) pq_settcpusertimeout(newval, MyProcPort); diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index 1149d89d7a1..13fb8c31702 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -3555,7 +3555,7 @@ check_log_stats(bool *newval, void **extra, GucSource source) /* GUC assign hook for transaction_timeout */ void -assign_transaction_timeout(int newval, void *extra) +assign_transaction_timeout(int newval, void *extra, bool *pending) { if (IsTransactionState()) { diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index 12192445218..bab1c5d08f6 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -1679,6 +1679,7 @@ InitializeOneGUCOption(struct config_generic *gconf) struct config_int *conf = (struct config_int *) gconf; int newval = conf->boot_val; void *extra = NULL; + bool pending = false; Assert(newval >= conf->min); Assert(newval <= conf->max); @@ -1687,9 +1688,13 @@ InitializeOneGUCOption(struct config_generic *gconf) elog(FATAL, "failed to initialize %s to %d", conf->gen.name, newval); if (conf->assign_hook) - conf->assign_hook(newval, extra); - *conf->variable = conf->reset_val = newval; - conf->gen.extra = conf->reset_extra = extra; + conf->assign_hook(newval, extra, &pending); + + if (!pending) + { + *conf->variable = conf->reset_val = newval; + conf->gen.extra = conf->reset_extra = extra; + } break; } case PGC_REAL: @@ -2041,13 +2046,18 @@ ResetAllOptions(void) case PGC_INT: { struct config_int *conf = (struct config_int *) gconf; + bool pending = false; if (conf->assign_hook) conf->assign_hook(conf->reset_val, - conf->reset_extra); - *conf->variable = conf->reset_val; - set_extra_field(&conf->gen, &conf->gen.extra, - conf->reset_extra); + conf->reset_extra, + &pending); + if (!pending) + { + *conf->variable = conf->reset_val; + set_extra_field(&conf->gen, &conf->gen.extra, + conf->reset_extra); + } break; } case PGC_REAL: @@ -2424,16 +2434,21 @@ AtEOXact_GUC(bool isCommit, int nestLevel) struct config_int *conf = (struct config_int *) gconf; int newval = newvalue.val.intval; void *newextra = newvalue.extra; + bool pending = false; if (*conf->variable != newval || conf->gen.extra != newextra) { if (conf->assign_hook) - conf->assign_hook(newval, newextra); - *conf->variable = newval; - set_extra_field(&conf->gen, &conf->gen.extra, - newextra); - changed = true; + conf->assign_hook(newval, newextra, &pending); + + if (!pending) + { + *conf->variable = newval; + set_extra_field(&conf->gen, &conf->gen.extra, + newextra); + changed = true; + } } break; } @@ -3850,18 +3865,24 @@ set_config_with_handle(const char *name, config_handle *handle, if (changeVal) { + bool pending = false; + /* Save old value to support transaction abort */ if (!makeDefault) push_old_value(&conf->gen, action); if (conf->assign_hook) - conf->assign_hook(newval, newextra); - *conf->variable = newval; - set_extra_field(&conf->gen, &conf->gen.extra, - newextra); - set_guc_source(&conf->gen, source); - conf->gen.scontext = context; - conf->gen.srole = srole; + conf->assign_hook(newval, newextra, &pending); + + if (!pending) + { + *conf->variable = newval; + set_extra_field(&conf->gen, &conf->gen.extra, + newextra); + set_guc_source(&conf->gen, source); + conf->gen.scontext = context; + conf->gen.srole = srole; + } } if (makeDefault) { diff --git a/src/backend/utils/misc/stack_depth.c b/src/backend/utils/misc/stack_depth.c index 8f7cf531fbc..ef59ae62008 100644 --- a/src/backend/utils/misc/stack_depth.c +++ b/src/backend/utils/misc/stack_depth.c @@ -156,7 +156,7 @@ check_max_stack_depth(int *newval, void **extra, GucSource source) /* GUC assign hook for max_stack_depth */ void -assign_max_stack_depth(int newval, void *extra) +assign_max_stack_depth(int newval, void *extra, bool *pending) { ssize_t newval_bytes = newval * (ssize_t) 1024; diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h index 1233e07d7da..ce9f258100d 100644 --- a/src/include/utils/guc.h +++ b/src/include/utils/guc.h @@ -187,7 +187,7 @@ typedef bool (*GucStringCheckHook) (char **newval, void **extra, GucSource sourc typedef bool (*GucEnumCheckHook) (int *newval, void **extra, GucSource source); typedef void (*GucBoolAssignHook) (bool newval, void *extra); -typedef void (*GucIntAssignHook) (int newval, void *extra); +typedef void (*GucIntAssignHook) (int newval, void *extra, bool *pending); typedef void (*GucRealAssignHook) (double newval, void *extra); typedef void (*GucStringAssignHook) (const char *newval, void *extra); typedef void (*GucEnumAssignHook) (int newval, void *extra); diff --git a/src/include/utils/guc_hooks.h b/src/include/utils/guc_hooks.h index 951451a9765..3e380f29e5a 100644 --- a/src/include/utils/guc_hooks.h +++ b/src/include/utils/guc_hooks.h @@ -81,12 +81,12 @@ extern void assign_log_timezone(const char *newval, void *extra); extern const char *show_log_timezone(void); extern bool check_maintenance_io_concurrency(int *newval, void **extra, GucSource source); -extern void assign_maintenance_io_concurrency(int newval, void *extra); +extern void assign_maintenance_io_concurrency(int newval, void *extra, bool *pending); extern bool check_max_slot_wal_keep_size(int *newval, void **extra, GucSource source); -extern void assign_max_wal_size(int newval, void *extra); +extern void assign_max_wal_size(int newval, void *extra, bool *pending); extern bool check_max_stack_depth(int *newval, void **extra, GucSource source); -extern void assign_max_stack_depth(int newval, void *extra); +extern void assign_max_stack_depth(int newval, void *extra, bool *pending); extern bool check_multixact_member_buffers(int *newval, void **extra, GucSource source); extern bool check_multixact_offset_buffers(int *newval, void **extra, @@ -141,13 +141,13 @@ extern void assign_synchronous_standby_names(const char *newval, void *extra); extern void assign_synchronous_commit(int newval, void *extra); extern void assign_syslog_facility(int newval, void *extra); extern void assign_syslog_ident(const char *newval, void *extra); -extern void assign_tcp_keepalives_count(int newval, void *extra); +extern void assign_tcp_keepalives_count(int newval, void *extra, bool *pending); extern const char *show_tcp_keepalives_count(void); -extern void assign_tcp_keepalives_idle(int newval, void *extra); +extern void assign_tcp_keepalives_idle(int newval, void *extra, bool *pending); extern const char *show_tcp_keepalives_idle(void); -extern void assign_tcp_keepalives_interval(int newval, void *extra); +extern void assign_tcp_keepalives_interval(int newval, void *extra, bool *pending); extern const char *show_tcp_keepalives_interval(void); -extern void assign_tcp_user_timeout(int newval, void *extra); +extern void assign_tcp_user_timeout(int newval, void *extra, bool *pending); extern const char *show_tcp_user_timeout(void); extern bool check_temp_buffers(int *newval, void **extra, GucSource source); extern bool check_temp_tablespaces(char **newval, void **extra, @@ -163,7 +163,7 @@ extern bool check_transaction_buffers(int *newval, void **extra, GucSource sourc extern bool check_transaction_deferrable(bool *newval, void **extra, GucSource source); extern bool check_transaction_isolation(int *newval, void **extra, GucSource source); extern bool check_transaction_read_only(bool *newval, void **extra, GucSource source); -extern void assign_transaction_timeout(int newval, void *extra); +extern void assign_transaction_timeout(int newval, void *extra, bool *pending); extern const char *show_unix_socket_permissions(void); extern bool check_wal_buffers(int *newval, void **extra, GucSource source); extern bool check_wal_consistency_checking(char **newval, void **extra, -- 2.45.1 --q7xzg7duksj2kgko Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="v2-0005-Allow-to-resize-shared-memory-without-restart.patch" ^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2025-02-19 16:45 UTC | newest] Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-06-24 05:09 [PATCH 10/11] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 04/10] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 3/7] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 10/11] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 07/23] cirrus/freebsd: run with more CPUs+RAM and do not repartitiion Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 08/25] cirrus/freebsd: run with more CPUs+RAM and do not repartitiion Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 2/8] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 2/8] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 2/9] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2022-06-24 05:09 [PATCH 04/10] cirrus/freebsd: run with more CPUs+RAM and do not repartition Justin Pryzby <pryzbyj@telsasoft.com> 2025-02-19 16:45 [PATCH v2 4/6] Introduce pending flag for GUC assign hooks Dmitrii Dolgov <9erthalion6@gmail.com>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox