public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 08/10] Default to zstd..
4+ messages / 3 participants
[nested] [flat]
* [PATCH 08/10] Default to zstd..
@ 2021-03-12 21:35 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Justin Pryzby @ 2021-03-12 21:35 UTC (permalink / raw)
for CI, not for merge
---
configure | 6 ++++--
configure.ac | 2 +-
src/backend/access/transam/xlog.c | 2 +-
src/backend/utils/misc/guc.c | 2 +-
4 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/configure b/configure
index 81e23418b2..253f028fc4 100755
--- a/configure
+++ b/configure
@@ -1582,7 +1582,7 @@ Optional Packages:
use system time zone data in DIR
--without-zlib do not use Zlib
--without-lz4 build without LZ4 support
- --with-zstd build with Zstd compression library
+ --without-zstd build without Zstd compression library
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8740,7 +8740,9 @@ $as_echo "#define USE_ZSTD 1" >>confdefs.h
esac
else
- with_zstd=no
+ with_zstd=yes
+
+$as_echo "#define USE_ZSTD 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index d6f6349067..8d72710fa7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1005,7 +1005,7 @@ fi
# ZSTD
#
AC_MSG_CHECKING([whether to build with zstd support])
-PGAC_ARG_BOOL(with, zstd, no, [build with Zstd compression library],
+PGAC_ARG_BOOL(with, zstd, yes, [build without Zstd compression library],
[AC_DEFINE([USE_ZSTD], 1, [Define to 1 to build with zstd support. (--with-zstd)])])
AC_MSG_RESULT([$with_zstd])
AC_SUBST(with_zstd)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 307eee6626..92023de9f5 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -99,7 +99,7 @@ bool EnableHotStandby = false;
bool fullPageWrites = true;
bool wal_log_hints = false;
bool wal_compression = false;
-int wal_compression_method = WAL_COMPRESSION_LZ4;
+int wal_compression_method = WAL_COMPRESSION_ZSTD;
char *wal_consistency_checking_string = NULL;
bool *wal_consistency_checking = NULL;
bool wal_init_zero = true;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 52f9cd0242..8031e027aa 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -4728,7 +4728,7 @@ static struct config_enum ConfigureNamesEnum[] =
NULL
},
&wal_compression_method,
- WAL_COMPRESSION_LZ4, wal_compression_options,
+ WAL_COMPRESSION_ZSTD, wal_compression_options,
NULL, NULL, NULL
},
--
2.17.0
--jozmn01XJZjDjM3N
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="0009-Add-zstd-compression-levels.patch"
^ permalink raw reply [nested|flat] 4+ messages in thread
* Auto Vacuum optimisation
@ 2024-11-18 10:00 wenhui qiu <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: wenhui qiu @ 2024-11-18 10:00 UTC (permalink / raw)
To: Japin Li <[email protected]>; +Cc: pgsql-hackers; Frédéric Yhuel <[email protected]>
HI hackers and japin li
There was an earlier discussion about auto vacuum optimisation(
https://www.postgresql.org/message-id/6a2ac9b7-6535-4bb1-8274-0647f7c31c82%40dalibo.com),I
referring to the discussion in there, I implemented a parameter selection
triggering the calculation of AUTO VACUUM,This is my first patch for the
community.Hoping for support from the experts
Attachments:
[application/octet-stream] 0001-Add-guc-to-optimize-the-autovacuum-tigger-algorithm.patch (6.0K, ../../CAGjGUAKBLZAeA21ORp+s1S0uU7wAv0Mco2+Wh9uDAVDe3t+_LQ@mail.gmail.com/3-0001-Add-guc-to-optimize-the-autovacuum-tigger-algorithm.patch)
download | inline diff:
From fe09de086939c44de62fbc20acfd874c4389ba16 Mon Sep 17 00:00:00 2001
From: root <[email protected]>
Date: Mon, 18 Nov 2024 14:44:38 +0800
Subject: [PATCH] Add guc to optimize the autovacuum tigger algorithm
---
src/backend/postmaster/autovacuum.c | 34 ++++++++++++++++++++++-----
src/backend/utils/misc/guc_tables.c | 18 +++++++++++++-
src/backend/utils/misc/postgresql.conf.sample | 2 +-
src/include/postmaster/autovacuum.h | 11 +++++++++
4 files changed, 57 insertions(+), 8 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index dc3cf87..ab4020e 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -109,7 +109,7 @@
#include "utils/syscache.h"
#include "utils/timeout.h"
#include "utils/timestamp.h"
-
+#include <math.h>
/*
* GUC parameters
@@ -131,7 +131,7 @@ double autovacuum_vac_cost_delay;
int autovacuum_vac_cost_limit;
int Log_autovacuum_min_duration = 600000;
-
+int autovacuum_algorithm = AUTOVACUUM_ALGORITHM_LINEAR;
/* the minimum allowed time between two awakenings of the launcher */
#define MIN_AUTOVAC_SLEEPTIME 100.0 /* milliseconds */
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
@@ -3038,11 +3038,33 @@ relation_needs_vacanalyze(Oid relid,
/* If the table hasn't yet been vacuumed, take reltuples as zero */
if (reltuples < 0)
reltuples = 0;
-
- vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples;
+ switch (autovacuum_algorithm)
+ {
+ case AUTOVACUUM_ALGORITHM_LOG2:
+ vacthresh = (float4) fmin(vac_base_thresh + (vac_scale_factor * reltuples), vac_base_thresh + vac_scale_factor * log2(reltuples) * 100000.0 );
+ anlthresh = (float4) fmin(anl_base_thresh + (anl_scale_factor * reltuples), anl_base_thresh + anl_scale_factor * log2(reltuples) * 100000.0 );
+ elog(DEBUG2, "Using log2 algorithm for vacuum, vacthresh values is %f,anlthresh values is %f" ,vacthresh, anlthresh );
+ break;
+ case AUTOVACUUM_ALGORITHM_SQRT:
+ vacthresh = (float4) fmin(vac_base_thresh + (vac_scale_factor * reltuples), vac_base_thresh + vac_scale_factor * sqrt(reltuples) * 1000.0 );
+ anlthresh = (float4) fmin(anl_base_thresh + (anl_scale_factor * reltuples), anl_base_thresh + anl_scale_factor * sqrt(reltuples) * 1000.0 );
+ elog(DEBUG2, "Using sqrt algorithm for vacuum, vacthresh values is %f,anlthresh values is %f" , vacthresh, anlthresh );
+ break;
+ case AUTOVACUUM_ALGORITHM_LINEAR:
+ vacthresh = (float4) vac_base_thresh + vac_scale_factor * reltuples;
+ anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
+ elog(DEBUG2, "Using linear algorithm for vacuum, vacthresh values is %f,anlthresh values is %f", vacthresh, anlthresh );
+ break;
+ case AUTOVACUUM_ALGORITHM_POW:
+ vacthresh = (float4) fmin(vac_base_thresh + (vac_scale_factor * reltuples), vac_base_thresh + vac_scale_factor * pow(reltuples, 0.7) * 100.0 );
+ anlthresh = (float4) fmin(anl_base_thresh + (anl_scale_factor * reltuples), anl_base_thresh + anl_scale_factor * pow(reltuples, 0.7) * 100.0 );
+ elog(DEBUG2, "Using pow algorithm for vacuum, vacthresh values is %f,anlthresh values is %f", vacthresh, anlthresh);
+ break;
+ default:
+ elog(ERROR, "Unknown vacuum_algorithm value.");
+ break;
+ }
vacinsthresh = (float4) vac_ins_base_thresh + vac_ins_scale_factor * reltuples;
- anlthresh = (float4) anl_base_thresh + anl_scale_factor * reltuples;
-
/*
* Note that we don't need to take special consideration for stat
* reset, because if that happens, the last vacuum and analyze counts
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 8a67f01..5ef9304 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -474,6 +474,14 @@ static const struct config_enum_entry wal_compression_options[] = {
{NULL, 0, false}
};
+/* vacuum_algorithm guc */
+static const struct config_enum_entry autovacuum_algorithm_options[] = {
+ {"log2", AUTOVACUUM_ALGORITHM_LOG2, false},
+ {"sqrt", AUTOVACUUM_ALGORITHM_SQRT, false},
+ {"linear", AUTOVACUUM_ALGORITHM_LINEAR, false},
+ {"pow", AUTOVACUUM_ALGORITHM_POW, false},
+ {NULL, 0, false}
+};
/*
* Options for enum values stored in other modules
*/
@@ -5206,7 +5214,15 @@ struct config_enum ConfigureNamesEnum[] =
DEBUG_LOGICAL_REP_STREAMING_BUFFERED, debug_logical_replication_streaming_options,
NULL, NULL, NULL
},
-
+ {
+ {"autovacuum_algorithm", PGC_SIGHUP, AUTOVACUUM,
+ gettext_noop("vacuum trigger algorithm."),
+ NULL
+ },
+ &autovacuum_algorithm,
+ AUTOVACUUM_ALGORITHM_LINEAR, autovacuum_algorithm_options,
+ NULL, NULL, NULL
+ },
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, NULL, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 39a3ac2..4c29fe9 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -684,7 +684,7 @@
#autovacuum_vacuum_cost_limit = -1 # default vacuum cost limit for
# autovacuum, -1 means use
# vacuum_cost_limit
-
+#autovacuum_algorithm = linear # autovacuum_algorithm (log2,sqrt,linear,pow) default linear
#------------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index cae1e8b..6fe6732 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -47,6 +47,17 @@ extern PGDLLIMPORT int AutovacuumLauncherPid;
extern PGDLLIMPORT int Log_autovacuum_min_duration;
+/* algorithms for Vacuum */
+typedef enum Autovacuum_algorithm
+{
+ AUTOVACUUM_ALGORITHM_LOG2,
+ AUTOVACUUM_ALGORITHM_SQRT,
+ AUTOVACUUM_ALGORITHM_LINEAR,
+ AUTOVACUUM_ALGORITHM_POW
+} Autovacuum_algorithm;
+
+extern PGDLLIMPORT int autovacuum_algorithm;
+
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
--
1.8.3.1
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Auto Vacuum optimisation
@ 2024-11-18 10:20 wenhui qiu <[email protected]>
parent: wenhui qiu <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: wenhui qiu @ 2024-11-18 10:20 UTC (permalink / raw)
To: Japin Li <[email protected]>; +Cc: pgsql-hackers; Frédéric Yhuel <[email protected]>
HI hackers
These are algorithmic mathematical graphs,By default, vacuum will not be
triggered frequently.Looking at the images the algorithms for pow and sqrt
are close, do we want to cancel one of the algorithms?
log2
[image: image.png]
sqrt
[image: image.png]
pow
[image: image.png]
wenhui qiu <[email protected]> 于2024年11月18日周一 18:00写道：
> HI hackers and japin li
> There was an earlier discussion about auto vacuum optimisation(
>
> https://www.postgresql.org/message-id/6a2ac9b7-6535-4bb1-8274-0647f7c31c82%40dalibo.com),I
> referring to the discussion in there, I implemented a parameter selection
> triggering the calculation of AUTO VACUUM,This is my first patch for the
> community.Hoping for support from the experts
>
Attachments:
[image/png] image.png (126.7K, ../../CAGjGUAKtyAdOjOZ3NuXNG=URp9iXnGpk+vo01GenxuaWbD5maQ@mail.gmail.com/3-image.png)
download | view image
[image/png] image.png (139.8K, ../../CAGjGUAKtyAdOjOZ3NuXNG=URp9iXnGpk+vo01GenxuaWbD5maQ@mail.gmail.com/4-image.png)
download | view image
[image/png] image.png (150.4K, ../../CAGjGUAKtyAdOjOZ3NuXNG=URp9iXnGpk+vo01GenxuaWbD5maQ@mail.gmail.com/5-image.png)
download | view image
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Auto Vacuum optimisation
@ 2024-11-26 13:25 Japin Li <[email protected]>
parent: wenhui qiu <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Japin Li @ 2024-11-26 13:25 UTC (permalink / raw)
To: wenhui qiu <[email protected]>; +Cc: pgsql-hackers; Frédéric Yhuel <[email protected]>
On Mon, 18 Nov 2024 at 18:20, wenhui qiu <[email protected]> wrote:
> HI hackers
> These are algorithmic mathematical graphs,By default, vacuum will not be triggered frequently.Looking at the images
> the algorithms for pow and sqrt are close, do we want to cancel one of the algorithms?
>
> log2
> image.png
>
> sqrt
> image.png
> pow
> image.png
>
Sorry for the late reply.
Did you forget to upload the pictures?
> There was an earlier discussion about auto vacuum optimisation(
> https://www.postgresql.org/message-id/6a2ac9b7-6535-4bb1-8274-0647f7c31c82%40dalibo.com),I referring to the
> discussion in there, I implemented a parameter selection triggering the calculation of AUTO VACUUM,This is my first
> patch for the community.Hoping for support from the experts
I think this is helpful.
One of our customers had large tables that were frequently updated, and their
global configuration did not permit timely VACUUM operations on those tables.
As a result, they had to adjust the vacuum settings for those specific tables,
which unintentionally increased their maintenance burden.
However, I'm not sure we need so many autovacuum algorithms here since it makes
the choice harder (maybe).
--
Regrads,
Japin Li
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2024-11-26 13:25 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-03-12 21:35 [PATCH 08/10] Default to zstd.. Justin Pryzby <[email protected]>
2024-11-18 10:00 Auto Vacuum optimisation wenhui qiu <[email protected]>
2024-11-18 10:20 ` Re: Auto Vacuum optimisation wenhui qiu <[email protected]>
2024-11-26 13:25 ` Re: Auto Vacuum optimisation Japin Li <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox