public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 06/10] Default to LZ4..
4+ messages / 4 participants
[nested] [flat]
* [PATCH 06/10] Default to LZ4..
@ 2021-03-12 21:35 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Justin Pryzby @ 2021-03-12 21:35 UTC (permalink / raw)
this is meant to exercise in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
src/backend/access/transam/xlog.c | 2 +-
src/backend/utils/misc/guc.c | 2 +-
4 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/configure b/configure
index fed440adcf..8d76be00c1 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 8c454128bb..bfcdc88be0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 984ce39cc7..3657f74de9 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -99,7 +99,7 @@ bool EnableHotStandby = false;
bool fullPageWrites = true;
bool wal_log_hints = false;
bool wal_compression = false;
-int wal_compression_method = WAL_COMPRESSION_ZLIB;
+int wal_compression_method = WAL_COMPRESSION_LZ4;
char *wal_consistency_checking_string = NULL;
bool *wal_consistency_checking = NULL;
bool wal_init_zero = true;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index c37a8313d3..52f9cd0242 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -4728,7 +4728,7 @@ static struct config_enum ConfigureNamesEnum[] =
NULL
},
&wal_compression_method,
- WAL_COMPRESSION_ZLIB, wal_compression_options,
+ WAL_COMPRESSION_LZ4, wal_compression_options,
NULL, NULL, NULL
},
--
2.17.0
--XsQoSWH+UP9D9v3l
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="0007-add-wal_compression_method-zstd.patch"
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Truncate logs by max_log_size
@ 2024-09-27 00:30 Euler Taveira <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Euler Taveira @ 2024-09-27 00:30 UTC (permalink / raw)
To: diPhantxm <[email protected]>; [email protected]
On Thu, Sep 26, 2024, at 3:30 PM, diPhantxm wrote:
> I would like to suggest a patch to truncate some long queries. I believe sometimes there is no need to log a query containing some gigabytes of blob, for example. In patch a new parameter, named max_log_size, is introduced. It defines the maximum size of logged query, in bytes. Everything beyond that size is truncated.
I don't know how useful is this proposal. IMO the whole query is usually
crucial for an analysis. Let's say you arbitrarily provide max_log_size = 100
but it means you cannot see a WHERE clause and you have a performance issue in
that query. It won't be possible to obtain the whole query for an EXPLAIN. It
would break audit systems that requires the whole query. I don't know if there
are some log-based replication systems but it would break such tools too.
There are other ways to avoid logging such long queries. The GRANT ... ON
PARAMETER and SET LOCAL commands are your friends. Hence, you can disable
specific long queries even if you are not a superuser.
If your main problem is disk space, you can adjust the rotation settings or have
an external tool to manage your log files (or even use syslog).
--
Euler Taveira
EDB https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Truncate logs by max_log_size
@ 2024-09-27 10:36 Andrey M. Borodin <[email protected]>
parent: Euler Taveira <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Andrey M. Borodin @ 2024-09-27 10:36 UTC (permalink / raw)
To: Euler Taveira <[email protected]>; +Cc: diPhantxm <[email protected]>; [email protected]
> On 27 Sep 2024, at 03:30, Euler Taveira <[email protected]> wrote:
>
> Let's say you arbitrarily provide max_log_size = 100
Consider max_log_size = 10Mb. The perspective might look very different. It’s not about WHERE anymore. It's a guard against heavy abuse.
The feature looks very important for me.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Truncate logs by max_log_size
@ 2024-10-01 11:46 Jim Jones <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Jim Jones @ 2024-10-01 11:46 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; Euler Taveira <[email protected]>; +Cc: diPhantxm <[email protected]>; [email protected]
On 27.09.24 12:36, Andrey M. Borodin wrote:
> Consider max_log_size = 10Mb. The perspective might look very different.
> It’s not about WHERE anymore. It's a guard against heavy abuse.
>
> The feature looks very important for me.
I have the same opinion. As a fail safe it sounds very useful to me.
Unfortunately, the patch does not apply:
$ git apply
~/patches/max_log_query/0001-parameter-max_log_size-to-truncate-logs.patch
-v
Checking patch src/backend/utils/error/elog.c...
error: while searching for:
char *Log_destination_string = NULL;
bool syslog_sequence_numbers = true;
bool syslog_split_messages = true;
/* Processed form of backtrace_symbols GUC */
static char *backtrace_symbol_list;
error: patch failed: src/backend/utils/error/elog.c:114
error: src/backend/utils/error/elog.c: patch does not apply
Checking patch src/backend/utils/misc/guc_tables.c...
Hunk #1 succeeded at 3714 (offset 247 lines).
Checking patch src/backend/utils/misc/postgresql.conf.sample...
Hunk #1 succeeded at 615 (offset 23 lines).
Checking patch src/bin/pg_ctl/t/004_logrotate.pl...
error: while searching for:
check_log_pattern('csvlog', $new_current_logfiles, 'syntax error', $node);
check_log_pattern('jsonlog', $new_current_logfiles, 'syntax error', $node);
$node->stop();
done_testing();
error: patch failed: src/bin/pg_ctl/t/004_logrotate.pl:135
error: src/bin/pg_ctl/t/004_logrotate.pl: patch does not apply
Checking patch src/include/utils/elog.h...
Hunk #1 succeeded at 502 (offset 1 line).
Does it truncate only single queries or also a transaction with many
statements?
Thanks!
Best, Jim
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2024-10-01 11:46 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-03-12 21:35 [PATCH 06/10] Default to LZ4.. Justin Pryzby <[email protected]>
2024-09-27 00:30 Re: Truncate logs by max_log_size Euler Taveira <[email protected]>
2024-09-27 10:36 ` Re: Truncate logs by max_log_size Andrey M. Borodin <[email protected]>
2024-10-01 11:46 ` Re: Truncate logs by max_log_size Jim Jones <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox